There are lots of horse & carriage rides in both the Old and New Towns. Some have old style stage coaches with the driver sitting up top and the guide at the back. I loved the romance of the little carriage bumping its way through the cobbled streets, built just for two and the only words from the driver being the price = just a few zoltys = but you don't go far!
The horse-drawn tram (charabanc) leaves from Plac Zamkowy near King Zygmunt's Column and goes along Podwale St to the New Town Square and then back along Freta, Dluga and Senatorska to Plac Zamkowy.
Such trams were used in Warsaw in the years 1836 - 1881 so you can get the feel of 19th century Warsaw. Very romantic, unless there are some noisy children on board.
The ride is available only on Saturdays and Sundays in the summer months and the number of places is limited so count on your luck.
Tickets: adults - 12 zl, children under 12 - 6 zl, children under 4 (not occupying a separate seat) - free
Leaves from Plac Zamkowy (Castle Square) every half an hour between 11.00 and 18.30.
Picture from Wikipedia
For around 100 zl per hour you can tour the Old Town by horse drawn carriage. You can find the carriages in Old Town Square or Castle Square and the drivers will be more than happy to offer their opinions in English as they take you around the town.
I didn't do it since I was traveling alone and also because the Old Town isn't that big that walking is a problem, but it might be a romantic way to see the town with the right company.
Horsecarriage, well not exactly, it says "OMNIBUS" on it (isn't an omnibus a collection of stories?)...but it does have a horse and it does have a carriage of sorts. Now I think this is a totally cool way of getting around the old city of Warsaw.
Oh and if you have a dog you know that plastic bag you carry around to collect what your dog "does" on the street...well here the driver has a BIG BAG (actually TWO).
I don't think this is cheap but tourists love to get around the Old Town of Warsaw on a horse carriage. Here's one of them :)