Try one of the free bicykles and the bicykle elevator.
Bicykles can be picked up just about everywhere in the city centre. If you find any of these peculiar looking bikes in the city without anybody on them, you can pick it up. If its atached with a chain, just push 10 NOK (May be more now, this was in 1998) into the lock. You'll get the coin back when you deliver the bike back.
The elevator goes up the hill from bakklandet almost up to the fortress. It works with a key card that you can get from one of the kiosk at the bottom of the elevator. It takes a bit of practice to get up, but you'll manage after a few atempts.