Public transport will be of absolutely no use to anyone that wants to visit tourist spots in Mount Gambier.
The local buses go in certain loops of the city, taking in mainly residential areas and schools, the local hospitala nd the local UNI/TAFE.
You won't find a local bus going to the Blue Lake, Valley Lakes, Port MacDonnell or any of the caves.
or you can drive .... It takes about 4.5 hours from Adelaide and 5 from Melbourne about (500kms) to get here. There are a number of routes you can take, inland and coast from both directions.
Contact me if you would like some more info.
Easy way is to fly. Regional Express (Qantas) and local airlines O'COnnors offer 2 flights daily to and from mt gambier from Melbourne and Adelaide. You can also get here from most large regional centres, like Port Augusta, Port Lincoln etc.