I see where your going with this. I'm a Try-and-get-more-accuracy kinda guy too. If I had to chose, I'd definitely say the barrel, since it's the part directing the projectile and more things can be different from barrel to barrel than from frame to frame. The contact points on one frame will be darn near identical to those of another frame saying that there will be little or no difference in accuracy. If I had to guess (which I'm doing by saying this) there could be a slight difference due to lock up and frame to barrel tolerances that could make a bit of difference. If not Ole' Mike Bhellum wouldn't have made such a big deal about those damn oversized hinge pins making such a "dramatic" improvement in accuracy.
I'd put my hard earned jing into a custom barrel before anything. As a matter of fact, I have. My barrel from Bullberry's should be here soon. I just send my frame to TC today to fix my problem with the lock up. 44 mag slams it open every time except using cheapass CCI loads. I got it back today from Fred @ Bullberry's saying that the locking shelf was worn and hammered up badly causing a negative angle. Even with the CCI's I couldn't get better than 4" @ 100 yards. That lock up problem could have contributed to some of that.
As for the grips and forend, where you hold it, how tight, what it rests on and how the recoil is handled all play a part in consistency as well. I don't think style of these has much to do with it.
My two Lincoln heads......