Japan is considered a black hole for global travelers since their mobile standards are different. GSM phones need not apply. But there is a way to go: get a 3G data SIM and use VoIP. Can it be done?
Before flying over I found that SoftBank Mobile would "rent" a 3G SIM for a combined price of ¥1575/day for unlimited data. With the current exchange rate, that prices out over $20/day, a bit pricey, but often a better bargain than for broadband in a hotel. Also, I was to stay at the Imperial - a great hotel, with good broadband, but one which paradoxically did not have WiFi in the rooms. What to do with my iPad, iPhone and NexusOne travel phone? (For that problem, I brought along an Apple Airport Express, which is working great to connect everything; my business partner in the next room over is using it, as the signal seeps to surrounding rooms.)
The SoftBank SIM is something you need to reserve in advance or pick up a the airport. My travel companions were too anxious to spend much time finding the SoftBank booth, and twice when I asked locals at the airport where it was, they sent me to generic phone rental booths, neither of which had the SoftBank SIM deal. In the end we caught the next train to Tokyo, with the Japanese of us assuring me I could find what I needed in the city. It turns out you can, but only at the SoftBank HQ in the Roponggi district, a bit inconvenient.
Chagrined, they lent me one of their MiFi-type units, which cost the same per day and had the benefit of creating a little hotspot for all my devices. (I had planned instead to turn on the WiFi hotspot feature in the Nexus phone to connect my iPad remotely.) You can pick these up at the generic phone booths at the airport. Problem solved, if a bit inelegantly. Next time I will find the SoftBank booth and "rent" their 3G SIM and avoid lugging the extra MiFi unit around.