A little more personality.
Out of all the places I visited in the Hamptons (during my short stay), I liked Sag Harbor the best. Whereas I found most of the little villages bland and sterile, Sag had much more personality. Basically all of the stores were independent and not chains and really gave a small town main street feel. And hey, the views were very nice as well.