If you follow ul. 3 Maja away from the Rynek end, you'll eventually come to the Old Town Castle. It dates back to the end of the 16th century, but following recent renovation it looks brand new. In the past it's been used as a jail, but it's now the law courts.
I'm afraid I don't know what this building is -- maybe one of the locals can help? -- it's between the end of ul. 3 Maja and the Castle -- but it's a good example of the sort of architecture you can see when wandering the streets.