Definitely... 
When my brother and I visited Tokyo this summer,
we had an interesting experience getting around 
with Asian faces (and demeanor) but speaking only
English. ^_^' We actually went with a tour only 
for the first day, then spent six days on our 
own, getting lost, accidentally spending $$$ on a
meal (ho ho, that was fun!), looking for anime 
stores, and discovering we couldn't make it to 
a couple temples without a car (we weren't 
confident enough to get a taxi). Definitely go 
for the cultural experience - it is like nothing 
you've seen/done before. (Well, you know.) We DID,
of course, expect to get some anime shopping in.
Just remember to write down the titles of your 
favorite series in Japanese in case you need help 
finding something. As for cels - yes, I was a bit
disappointed, too, though I did get quite a few 
goodies from older, lesser known series like KnO 
and that Phantom Tribe cel from El Hazard... You 
never know. :P Just make sure you have lots of 
good maps if you plan on striking out on your own
like we did!