if you try adjusting the forearm screw it may help.i had a similar thing happen with a #1 270 rsi.i also had to relieve the nut on the front of the foreaem that holds the forearm cap in place it was proud if the wood surface by at about .100".a little dremel work on that and just tightening the screw enough to make contact with a little lock-tite i was able to acheive 1/2 groups at 100+ yds,letting the barrel cool between rounds.i have a 30/06 that shoots just as well,never had to touch that one,just found a load it liked.
you may want to try a lighter load/less velocity my, 270 didn't like real hot stuff,and literally it did the same exact thing,would produce two groups out of 4 shots.be sure to clean and cool your barrel between shots,good luck adker.