If your powder/pellets are old or have been exposed to any moisture, it could be the problem, as it'll cause inconsistent ignition.
Are you taking the stock off the gun after the shooting is done, while you're cleaning? You may not be getting the screws tight enough, or tightened equally.
Make sure that your bases and rings are tight, if either of them are loose in the slightest, it could cause your problem.
You might take it to smith and have them check the crown of the barrel below the QLA, but I don't think that's problem. Could be, but I don't think so.
I've got a stainless/synthetic, and it grouped a bit erratically until I hit the 50 round mark, then it settled right down and puts the 250gr SW and 100grs of 777 loose FFg in the same ragged hole all day long at 100 yds.
I don't have a problem getting the primers in with my fingers, but I've got small hands, which helps. My primers start to stick after 3 consecutive shots (without cleaning), and it gets a little worse as I continue to shoot, even if I clean between shots. I get home, pull the plug and clean it, no problems until I get past three shots on the next outing. I had one of the Omega Z5's before I bought the stainless, and I never had any problems with primers sticking. Maybe a new breech plug will take care of it.
I use the T/C dogbone capper in the field, and I have never missed with it, as it holds the primer firmly. You have to get it seated in the recess and then pull the capper off. If the primer isn't seated, it's not going to pull out of the capper. If a primer does fall down in the block, I just make sure that it's open completely as I tilt the rifle upward and over, the primer falls right out.