Semen is indeed generally harmless, though it can transmit bacteria and minute quantities of HIV since there are small amounts of white blood cells found in semen.
However, urine and blood are also sterile. If the urine isn't sterile, you're probably in for some significant discomfort. If your blood isn't sterile, you're in
serious trouble (on the order of "you're probably gonna die" sort of trouble). Sweat is also sterile, as its acidic pH retards the growth of bacteria. The odor associated with sweat occurs after it's remained on the skin for some time and bacteria have begun to process the organic materials found in sweat. (By the way, soaps are alkaline and can lower the pH of skin secretions, thereby actually promoting the growth of odor-causing bacteria on the skin.)
On the other hand, saliva is unbelievably contaminated with the worst kinds of bacteria, yet we blithely swap spit in so many ways: making out, sharing beverages, etc. The irony is that a human bite is considered far more dangerous than a dog bite. Nevertheless, I've met plenty of people who think nothing of casual making out with virtual strangers -- and yet are repulsed by feet, which are practically havens of sanitation by comparison.
For as large as our brains are, humans can be surprisingly irrational creatures.