Namibia is an arid country that is regularly afflicted by drought. Most urban residents have access to drinking water supply, but access lags behind in rural areas. Although tap water is pretty much always drinkable in larger town and cities, such as Windhoek, it is often recycled and so can taste chlorinated. Most lodges and guest farms in rural areas use bore hole water which is clean and fine to drink. That said many people choose to drink bottled water which is easily obtainable at supermarkets around the country.
Of course the act of drinking water is completely superfluous as Namibia has awesome beer and access to fine wine & gin - so really your major concern should be using water for the purpose of washing. If you do feel the need to wash regularly please remember that water is scarce so take a short shower and save the planet!
"Water, taken in moderation, cannot hurt anybody." Mark Twain