Once you've crossed the river you can walk back alongside the river and take pictures of the Longmen caves from a distance. If you walk further at one point you can go up East Mountain via a long stone staircase and have a very nice walk on that hillside. One your way you will come across a small temple (where you can strike the big bell) that used to be a summer retreat of Chang Kai Shek (Jiang Jieshi), a residential area with some villa's and a restaurant, some very nice scenery with a small waterfall, and some nice views on the river.
Definitely do this part also and not just the Longmen caves, because it's definitely worth it.
Longmen is about 30 km from the center of Luoyang, so about half an hour driving. It is quite impressive to see, with thousands of holes and caves and thousands and thousands of statues. The biggest one is about 17 meters high, and the smallest are as big as a fingertip. Take your time as you walk from one side of the big rock to the other, because you will have some climbing to do.
If you visit Longmen caves, this restaurant is a good place to go to eat. It's on the other side of the river. On one point you just start climbing up the steps (a beautiful walk past a small buddhist temple and some pitoresque waterfalls), and on the top of the hill there is a hotel with a restaurant. Above the restaurant it says Zhong Canting (Chinese eating place). It's a nice restaurant and when we were there, the place was really calm.
I forgot what I ate, but it was good.