Kind of sounds like a forend bedding problem, or at least that is the only thing I can think of right now. Is the forend free floated (either by sanding out the barrel channel and adding the washers for the barrel to sit on or by just adding the washers)? This might indeed be the problem in that the forend is not being put on exactly the same way everytime.
Is this on a Contender? If so maybe the pieces in the dovetails in the forend that the forend attaches to are not centered or are loose and moving on that barrel. (I have had to shim some in the past as there were loose).
Other than that I would suspect the scope mount being loose or something, but that should change while you are shooting it.
SD Handgunner