On my last visit we drove out from Queens, early on a Sunday in October. But obviously I’ve been travelling to and from the town for a while.
The traffic will be hellish on summer weekends due to the invasion of city folks.
The LIRR (Long Island Railroad) serves the town. You can get coach services like the Hampton Jitney.
There are many small cab companies in the Hamptons. There are numbers posted at all the convience stores etc. If you are in Montauk there are 2 famous cab companies called "Pink Tuna Taxis" and "Celtic Taxis," the fist having pink cars, the latter having green cars.
They are expensive, not like in the city, but if you don't have a car there isn't much else.
You can bring a bike on the train. Get yourself a local map to get around once you get off the train at East Hampton. You can also get off at Amagamsett, from the Montauk Branch.
There is also a private bus company, The Hampton Jitney. You quite often see actors, newcasters, artists and writers on the bus. Keep your eyes peeled. It's a bit more expensive, to cover that juice and danish you get!