If you stay in Siem Reap for Angkor you should find someone with a motercycle. It is the best way to get around. The temples can be very far apart and although you can go by bicycle, you can see a lot more if you can find a motorcycle-driver to show you around.
For a better environment, and to give a good example...you should take the bike. Maybe you can go one day by motorbike for the further away temples if you decide to take the bike.
There are no public buses to PP, but some guesthouses operate their own buses, for about 9USD.
The bus of the photo leaves from the Rasmei Guesthouse, by the riverside.
The trip is long and the road bumpy... I chose the boat!
You can go from Siem Reap to Phnom Penh by boat through the Tonle Sap lake and river.
There is a speed boat that takes you there in 5 hours (22 USD). Buy the ticket at any guesthouse, and ask them to include a free riding to the pier (otherwise U'd have to pay the taxi there, some 30 minutes). They leave early in the morning.
At the pier there are food stalls to buy supplies, as the boat arrives after noon and makes no stops.
You can find here this kind of tuktuks, which can be useful to visit the temples if you are 2-3 and don't wanna rent 2-3 motorbikes. It is useful too when it rains, as they are covered.
Bargain the price in advance, considering 1 USD will be the maximum charge for a ride within SR.
Cycling is a good way to see SR and a few temples around. If you wanna see all the temples, hire a moto with a driver that will take you there faster. But if you just wanna take a nice ride to Angkor Wat in the afternoon (about 30 minutes) bikes are a perfect choice.
I rented one for a couple of hours in the evening for 1 USD in front of the Rasmei Guesthouse, by the river.
The cheapest way to arrive in SR overland is by bus from KSR, Bangkok. There are there many travel agencies and guesthouses (like Nathan GH, for instance) selling you cheap tickets, about 80 Bahts (2 USD).
You leave KSR in a private bus at about 8 am, arrive to the border in 4 hours, pass the customs (there will be a guide with the group all the time) and then about 4 hours again to SR. So all together it will take you the whole day...
Advantages: Easy, cheap, you avoid the touts in the border.
Disadvantages: Looong, boring, U go in a group, when arriving they will take you directly to the guesthouse where they get comission.
The cheapest way to get to SR from the Thai border (where you can arrive by bus from Bangkok) is to share a pick up with other local travelers. In the one I got there were several cambodian peasants on their way to the market. I paid about 150 bahts (4 USD) for the whole 3 hours ride.
The other way is to come by shared taxis (Toyota Camrys), costing about 300 Bahts (7 USD) if you share it or 1000 Bahts (22 USD) if you take it all for you (more comfort and quicker, as you don't have to wait for more passengers to fill it up).
The airport is north-west of Siem Reap and west of Angkor Wat, and is a short drive from both places. It's got a small modern terminal - but as far as I could see there are no facilities apart from immigration and customs in Arrivals. You'll need 20 US dollars for your visa and some money to get into town, so it's best to arrive with some!
The facilities in the departure lounge are a bit better - there's a cafe, a bookstand and a small gift shop, as well as plenty of seating.
There are currently flights to and from Phnom Penh, Singapore (on Silk Air), Ho Chi Minh (Vietnam Airlines and Siem Reap Airways) and Bangkok (Bangkok Airways); if you're searching for flights the airport code is REP.
From Phnom Penh to Siem Reap one option is to rent a taxi for about $50 and enjoy the ride in the Cambodian countryside.