Veracruz is one of Mexico's largest sea ports. As such, there are tons of options available here. There are buses going to most of the larger cities in Mexico and to several of the small ones as well. Just ask around at the main bus station. I took an overnight bus to Oaxaca from Veracruz.
Fort of San Juan de Ulua is a few miles outside Veracruz center. One of the few sights but nothing spectacular ( I did not see the inside...).
Sharing a taxi is more comfortable but a bus is much much more exciting. Reason: the buses are a little old, once they go faster than 20 km/h they fall apart!
So the driver has enough time to read his kinky books and have lunch. And the passengers have time to relax and take a long look at the harbour area of Veracruz.
This is more a comment about Veracruz city buses. Of the places I've taken city buses in so far, they are the worst for comfort in Veracruz. They are like old converted school buses. The seats are hard metal, so if you're in the back of the bus as we were (and the streets are in such bad shape in places) that it is a most uncomfortable ride!
I am told also that at times these buses do not follow their supposed route, if someone pays them to take them to a certain place off-route.
The bus system in Mexico is excelent, especially if you take 1st class or deluxe busses, with lot's of room and if you are lucky the movies will be in English.
The buststion is nice and modern and from there anything is close with the cab, so getting top veracruz, the bus is your best bet.