They get better with shooting, not cleaning, just shooting. I take three shots and set it aside, leaving the action open. 10 to 15 minutes later it's ready for another three. One thing to make sure of, did you remove and loc-tite the base screws. Is the scope level and rings tight. Where are you placing the rest, on the forearm, the barrel, or back at the action? Put a 1/16" (or there abouts) O-ring under the forearm to keep the barrel from contacting the forearm. By 1/16th I mean the thickness of the O-ring, not diameter. That's the most common size I use, but sometimes I need a thicker one or thinner one, each gun is different.
I shoot nothing but hand loads in mine. I like the 110gr Nosler Accubond bullet, in front of 57.5 gr Retumbo powder. That gives me 3210fps. For the average shooter 200 yards is a good sight in distance. The 25-06 is a very flat shooting cartridge. I'm shooting a 6" group at 400 yards. It'll tighten up with use.