Most people recommend you to use the Channels to remove the white background and then fill in the cropped pixels with a black brush. It is however much easier to remove the white background and keep your fine vector lines. Most experts have already found this method out, but yet when they create an tutorial they tell you to use the Channels.
I will tell you how to do this in 3 easy steps that will probably only take max 1 (2 if you're a really slow one or have a slow pc) minute the first time to master.

1) First of all, make SURE you only have a drawing with 1 colored background (preferable white, but other works as well).
Now select the Background Eraser Tool. Check the options to as follow:
Limits: Discontiquous, Tolerance: 100%, Sampling: Once.
Now make sure the brush covers the whole picture by increasing its size (if your pic is really big, can select a brush size at 2000 or something). Name the layer to Lines or something.

2) Ok, now for the fun! Click on a clear white spot on your drawing. The PC will load a bit, but after that it's all good! If you have a bigger picture, click multiple times at white spots to clean it all up. Now create a new layer and make it completly white. Move up the Lines layer up to the top, and tada! You have a clean lineart for coloring without any changes at all!

3) Now you can clean up your art easily. Just select the normal eraser and start erasing. Add lines with the brush tool. Keep note that you should be on the Lines layer when doing so!
You might wanna name the background to something as well... For coloring lines, just lock the layer's transparency and start color.

Now that you know how it works, it shouldn't take more then 30 seconds to get a clean lineart.
Don't go and smack your head saying "Was it this easy!?".. hehe.. Enjoy! And oh, remember to experiment! I always learn new stuff by down this... like this tutorial .