Don't take offense at this, but the first thing that came to mind was flinch. To determine if this is the case, take a friend with you to the range. Now, for the entire range session, you don't get to load the rifle yourself, your friend does. Make sure you have either cases with just live primers or fired cases for your friend to mix into the groups with the live loaded cartridges. If you are flinching, this process will show it, especially the cases with live primers. This is how I was taught consistency from a bench and field positions, it works very, very well.
Selmer