To Karimun: by ferry from Spore (Harbour Front) and P. Gudang (Malaysia).
Ticket from Sproe cost you less than SD 40 (I dont remember) and there are few ferry company provide this service. Mentioned them Tg. Balai, instead of Karimun.
get around: Hired one taxi to bring you ... DEAL before you use their taxi .. its should be cheap (less than Rp 200k for one day).
The best way to get to Karimun Island is by ferry from nearby Batam Island. The ferry leaves at the Sekupang ferry terminal at Batam Island and takes about 1 hour to reach Karimun Island. The cost is very cheap and the surrounding scenery is very beautiful (see the photographs of the VT travelogue on this page). Please visit my VT Batam page to check out how to get from Singapore to Batam.