I assume that you've successfully separated the 
cel from most of the sketch, and you're talking 
about a shred that's still sticking to the back 
of the cel?  But if the paper is not stuck to the 
cel paint, what's sticking it to the cel?  A drop 
of glue?  A bit of tape or tape adhesive?  Even 
if you get the last of the paper off, that might 
still be visible. 

The paper shouldn't be a problem if it's clear of 
the cel paint: dampen it and gently rub with a 
clean cloth (watch out for dirt that might 
scratch the cel) or, for a little tough spot, 
your fingernail.  But you may still end up with a 
spot of the "something" that was the source of 
the problem.  Dunno if you can get that off 
without using a solvent, and I'd be very 
reluctant to take this step.