The sheer cliffs just west of Longdong are world famous for technical rock climbing and the like. The cliff dropoff here is near vertical, and falls hundreds of meters down to the ocean.
This area is probably (along with Chingshui cliffs, which are just north of Hualien) the most scenic stretch of coastline in Taiwan.
The most popular trail in the area goes up the cliffs to Bitouchiao Lighthouse. The light house is located about 3 kilometes west of Longdong.
The lighthouse was constructed in 1897 and is the norther-most lighthouse in Taiwan. (and judging by the severly rocky coast around here, I would say it was money well spent....)
There lots of good hiking trails all around Longdong, both to the east and west of the bay, inlcluding the one popular trail that goes up to Bitouchiao Lighthouse.
Careful, though, as in a lot of places it is a sheer drop-off down into the water, so best to mind your footing.