In the villages there seems to be one price for locals and another for visitors! If you look like a typical tourist, you may charged more for anything that doesn't have a price on it - even for market stall fruit.
The Fort. A really interesting historical tour explains the chain of ownership of the fort (which changed hands several times between the English, and French). It offers great views across Fort de France and the bay. And its still an active military base, so you can see a bit of contemporary military life ;o)
The taxis are terribly expensive. Sometimes it is cheaper to rent a car for 1 day than paying a taxi for a trip from airport to your hotel. Otherwise the people are nice and there are no big dangers for tourists.