I think the highlight of any visit to this area would be taking a side trip up the Chihpen river valley. Go couple of kilometers past the tourist center, and on the right side you will find a new outdoor hot spring area that is now under development. From here there is an access trail down to the river.
As you hike up along the riverbed, you will find the scenery gets better and better. There are many places to wade/swim, and even a small waterfall that is perfect for an outdoor shower.
I didn't see another soul when we went up here, though I am sure that with time that will change... (for the worse...)
Chihpen Hot Spring Village is a cluster of hotels that cater to the many hot spring worshipper here in Taiwan. The range of accomodations here range from the good to the bad to the ugly.
Though, I really didnt find much in the way of natural hot springs here where you could take a dip, some of the hotels have outdoor pools in somewhat natural settings where it is pretty enjoyable to take a night time hot soak under the stars...
As with most of the eastern part of Taiwan, there is a fair amount of aboriginal influence in Taitung county.
If you stay at one of the top end hotels in Chihpen, you are likely to be exposed to some sort of pre-packaged aboriginal song and dance routine.