To drill the annealed Lee cutter straight, using my drill press I drilled a 1/8" hole in a 1" piece of hardwood , then using that hole as a pilot, drilled a 15/32" hole about ¾ of the way thru, drove the cutter into the hole as far as it would go using a block of wood on the cutter teeth to protect it from the hammer, then place the cutter face down in the wood block in the hole in the drill press table and put a center mark on the cutter with the 1/8" drill bit, once it's marked, remove the cutter from the block and clamp the cutter in a drill vise, chuck up a #29 drill then align the cutter vertically against the side of the drill bit so the cutter is perfectly straight up and down on two sides of the bit 90º apart, then drill it on the center mark and tap the hole, mine has just a very slight wobble.
Tim