Both travel guides I bought for my trip to Warsaw warned against drinking the tap water there, saying that even most locals avoid drinking it and that many of them get their water from wells. However, I drank lots of water straight out of the tap at my hotel. Not only did it look crystal clear, smell totally normal, and taste just fine, but I certainly never felt any ill effects from it. I stayed in a very cheap hotel, so it's hard to imagine they had a water filtration system. I guess my advice on this is to dig a little deeper before believing everything you read in the tour books.
Don´t drink the tap water, it is of very poor quality. Do it like the Warsovians, buy some bottled water. By the way, you might notice many people walking around with huge plastic bottles. It may sound funny for a European capital, but the people are getting their water for drinking and cooking from wells who have been built in all residential areas. The water from the wells is much healthier than the tap water which comes from the Vistula.