You can get a postal money order in Japanese yen 
at any post office that carries internation postal 
money orders in USD.  The only problem is that it 
takes a long time to process.  You need to come in 
with the amount in USD with the conversion rate & 
the written amount you need in yen.  Ask for a 
white international postal money order in Japanese 
yen.  It costs about 7$ or so & they send it off 
to some office in the mid US, where it gets 
processed & then sent off.  It can take up to 4 
weeks!