In addition to bathing in the hot waters, some of the locals swear by drinking the water. It will cure what ails you. Located around the park are many fountains from which you can fill your water jug. If you don't have a jug, they're available at the Fordyce Bathhouse and many of the stores around Bathhouse Row.
There are two kinds of springs outdoors. One type where the water is out in the open is decorative. The other type has a place for you to fill up a jug or other container with the water from the hot springs. Containers are on sale in the Fordyce bathhouse and also in local stores.
It is NOT true that the water is hot enough to kill bacteria.
There is a relatively natural spring - not for decoration, not for bathing, and not for drinking, but just a natural hot spring, behind the Fordyce Bathhouse, which we went to look at after we toured the bathhouse.
Keeping in mind that we were there in the winter time, the only way to tell that it is a hot spring is by the wisps of steam rising off the water. The URL has other peoples pictures of the hot springs.