For a short stretch, Hwy 1 leaves the coast and heads inland into the valley of Bis Sur. Rugged, beautiful mountains tower over the valley, which contains the "town" of Big Sur, as well as an information center and Pffeifer Big Sur State Park. You can just drive through the valley, or you can get out of the car and walk around a bit. Many redwoods crowd the area. The valley provides quick access to the Ventana Wilderness in Los Padres NAtional Forest.
Go swimming in the river, if don't mind the cold water. Also, at the northern end of the river is a nice gorge with a waterfall. I don't think fishing is allowed because of the endangered steelhead trout that like to spawn there, but you can fish for crawdads (small freshwater lobsters).
Yes, there is actually a town named Big Sur. But it is very small. It does have a nice little gift shop though.