Arezzo is located on the Firenze-Roma slow railway line (it is called this way in opposition to the direttissima line, where trains travel at 250 km/h, but this doesn't mean this line is slower than others). It is connected to many Tuscan towns with regional trains and to major Italian cities like Milano, Venezia, Roma, Napoli with Intercity trains.
Arezzo is a hub for Tuscan hilltowns. Most of the buses depart for other parts of Tuscany from the main traffic circle in town around the Piazza Guido Monaco. You can buy tickets for most destinations at the Tabacchi on the northwest "slice" of the circle. More information is available from the tourist office in terms of schedules, etc. If you have no idea where you want to go, this would be a great place to just hop on a bus and see where it takes you.
I sounds cliché, but the best way to get around and see the citycentre is on foot. The distances aren't very great and you'll see all this way.
'Though the bikeride of 'La vita è bella' is famous. I wouldn't recommend riding your bike in this town. Not only the piazza grande, but all streets are sloping and steep. In the movie they inly had to go downhill, unfortunately in real life that's different.
I wouldn't recommend to visit the old centre by car either. Most streets are, exept steep, very narrow and small. And there are a lot of one way streets.
So leave your car outside the centre or come by train and discover the centre on foot.