I use a Gesswein ruby stone. It's probably 3000 grit or so, and is hard enough to cut hardened steel. (They're made to dress carbide cutters) If you had to use sandpaper, I wouldn't go any coarser than 1000 grit. You don't need to remove all the tool marks, just take the high spots and roughness off. On most guns, if you've removed all tool marks you've gone too far.