that's one of those "how long is a piece of string" questions. You won't know until you try it. Sad, but if you find the pressure points did make it shoot better, they're a lot harder to put back than remove.
Perhaps you can try this: Make up some washers from biz cards or pop cans and put them on the action screws between the stock and the action. Put enough on that the barrel is lifted clear of the pressure points. (do the dollar bill trick to make sure the barrel isn't touching) You have in effect free floated the barrel. Before you do this, shoot several witness groups with your rifle as it is now set up. Then when you shoot several groups with the barrel not touching, you should get a pretty good idea of which way is better.