If you want to spend a full day here, renting a bike is a great way to see the island. Getting off at Gruvbryggan, you can easily see the mines on foot, but to get around to the best beaches and desolate places, bike rental is ideal. You pay in the guest harbour and pick the bikes up at in an old harbour barn beyond it as both are run by the company Utömakarna.
There are three different options for getting to Utö without your own boat (if you have your own, there is a nice guest harbour). You can take a boat from Strömkajen in Stockholm city centre, but be prepared to leave early and spend most of the day on the boat (with great views and a cafe!).
For a quicker journey, head for either Dalarö or Årsta where you take a much shorter boat ride (around 40 minutes). To get to those two, catch the commuter trains towards Nynäshamn from Stockholm Central or South stations and change for bus at Haninge or Västerhaninge respectively.