Try to visit the Bimini Museum which tells the history of the island(s) including Lerner marine laboratory, the old time wreckers, how sports fishing became popular, the wild children (like Adam Clayton Powell) who vacationed here, smuggling during prohibition, the immigration card of Jimmy Hoffa, and the sports hero records. It is housed the the former government building. At that time the waterfront was right across the street.
The cemetery is on the south end of the island. Bob walks down there every time we go to Bimini. From there you can see the cruise ships, and you can see what the waves are like in the Gulf Stream.
I have never gotten all the way down to the cemetery on foot although I've walked down to the Chalk Airline office.
You can just walk down to the beach off Queen Street and snorkel. It's probably not as good as going out on a dive boat, but it is accessible. For diving, see the Bimini District page.
I usually try to swim off the beach at least once while I'm there. Last time I had a new camera that I wanted to try out (after having lost my former dive camera on a dive - see Cozumel for the story). Bob thinks it's too cold in the winter, but he goes and sits on the beach to see that I don't drown. There are no lifeguards or services.
Take an early morning walk along the beach near all the pine trees. There is nothing quite like the pale morning light of the sea and sky evolving into full bright blue day light.
If there is a slight breeze you are in luck...the sound of wind through pine needles and the smell of fresh salt air mixed with pine is intoxicating. Add in the fact that you will probably be the only person on the beach and you may get the feeling that you are a castaway.