If all else fails, get the Lee Loader, a loading manual and a powder scale, then load up 5 shot batches of rounds, starting 10% less than the recommended maximum charge. Even lower may be good. Increase the charge by 1/2 grain increments up to max, then take them to the range and shoot each batch by itself. You should see your groups shrinking to what the rifle likes, then opening up again. That will give you your best load for that powder. Vary the types of powder, going through the same routine and compare the groups with what you had before. Eventually you will get the tightest group for the barrrel.
Make sure you hold the rifle in the same position when you shoot. Off the sandbags it is best to support the rifle at the back of the forearm. Check to see if there is anything in the barrel channel of the forearm which is putting pressure on the barrel. I put the NEF plastic stock set on my .30-06 at one time and accuracy went out the window. I tried relieving the channel and putting the rubber washer under it, with no change. Putting the wood back on gave me 1 1/2" groups at 100 yards, with a peep sight.
If you are going to get a new barrel which can be used for varmints but also deer, you might want to look at the 7mm-08. Shucks, the .308 with 125 gr. bullets might work out also....
Lots of things to think about. Keep us posted on what you do.
-WH-