The new Marina Village at Atlantis is finally open. This is part of the hotel complex and is a wonderful place to shop or eat. There's a great ice-cream shop that offers more flavors than I can count. You can sit and hear live music while having a tropical drink or go to one of the many restaurants. This is a fun place to go at night and a great addition to the Atlantis experience.
We were only in Nassau for a few hours, and weren't able to see much except the giant area where the local artists sell their things.
I would suggest limiting yourself to a few items or else you will be wrapped up in the atmosphere and end up with a lot of little trinkets that you will never use or wear again.
I bought a beautiful ironwood sculpture that I keep in our living room, and is a wonderful and simple reminder of our trip not to mention a great conversation piece.
There are many boutique style shops and stalls all over the island selling hand crafted gifts, hats and bags.