I have only arrived here by bus and by car. Either way can leave you anywhere in town you want to start. There are many public parking lots around town to leave the car and get to walking. We are fortunate to live about an hour and 45 minutes from Newport by car so it can be a one day destination if we don't want a full weekend in Newport.
Sadly the nearest train station is over in Kingston, RI which is approximately $55 each way by taxi from there.