You can exchange foreign currency for Japanese yen at banks and shops designated as "authorized money changers." They are avaialble all around tokyo. The exchange rate fluctuates daily in accordance with the money market conditions (the present rate is $ 1 = 88 yen, better than in hotel which is at $ 1 = 83 yen) Arriving in Japan without Japanese money should not present problems as foreign currency may be exchanged for yen at banks in the arrival lounge of the Narita or Haneda Airports.
Fondest memory: better change your money in money exchange shops or at post offices as they offer higher rates!
Favorite thing: Despite what your banks says, if you don't have a Japanese ATM card, you may have a hard time withdrawling money in Japan. Nearly all Japanese banks' ATMs do not accept standard foreign ATM cards with logos such as Maestro, Cirrus, and PLUS. There are two places that I am aware of that will accept them: Citibank ATMs, and some of the post office ATM (look for signs in English saying they accept foreign cards). The Citibank ATMs are usually only found in major districts around Tokyo. The post office is scattered throughout Japan and has reasonable opening hours (9am to 8pm or so). Be sure to take out plenty when you have the opportunity. Most likely, you will spend more than you planned to, and the ATMs may not be available on weekends or holidays.