You can keep your ID, but just not bid with it. 
Go to my Yahoo Japan Information Page at the URL 
above. Then go to the bottom of the page and 
you'll see a link regarding how to change your 
information.

In the pictures that are under that link, you 
will see a spot where I have noted: "These are 
the links to start and stop charges to your card."
That is the link to the page you're looking for. 
I haven't investigated these links myself, but if 
you run into any trouble, please feel free to 
write me and I'll help where I can. ^_^

Incidentally, you shouldn't need to cancel your 
ID completely. You should only need to tell it to 
stop charging your card. Once you do that, you 
won't be charged and your ID will still work for 
browsing (closed auctions too), but you just 
won't be able to bid or ask a question in the Q&A 
on auctions.

Many Sharp Smiles,
--Drac