It sound's like you need more lesson's in shooting first. Probally a good idea to get someone that shoot's very well to watch you and tell you what your doing wrong. The best handload's in the world won't make up for a lack of shooting skill. And developing reasonable skill's is not all that hard but there's more than just breath control. I'm guessing your using a scope. What power? Many people will shoot better with a lower power than a high power. 3X vrs 9X. the problem on 9X is that you see your breathing and your heart beat at high magnification. That's what moves the cross hairs around. The same thing happen's at 3X but the low power doesn't magnify it. At 9X the tendicity is to jerk the trigger when the crosshairs pass your aiming point. At 3X you don't see that and consentrate more on a good pull. By the way, I'm a great beliver in low power scopes on hunting rifle's,like 2 3/4 and 4. Either has plenty of magnification for shooting farther than you probally should shoot.
Next, I like to sight in at 100yds. Then I cronograph the load's and sight in for max point blank range at an 8" target. For most all cartridges that'll give you a dead on hold to around 300 yds. That means the bullet will never go more than 4" above the line of sight or 4" below it. I don't think the 308 will get quite 300 yds, don't have my balistics calculator with me, but it'll get you close. Then a dead on hold out to the max point blank range will give you a hit. Make sure you shoot at 100, 200 and 300 yds in practice. You'll be able to see just what's happening then.
But this is all for nothing unless you learn to shoot properly first. It's not that your not a pretty good shot, it's that you just probally never learned how to shoot properly in the first place. I can drop into a sitting position, kneeling position or prone (which I seldom shoot from) and land with all the tension out of my mussel's. You are probally fighting that tension. If you sight your rifle on the target fron a field position and then close your eye's and really relax, when you re-open your eye's, the sight's should still be on the target. If their not, you need to move your body around to release that tension. That tension is caused by you forcing your mussel's to move the muzzle to the target rather than turning your body.
Shooting from prone is real steady but your shoulder witt absorb all the recoil. If you shoot from sitting or kneeling, the recoil will push your shoulder back and your hip's will rotate back slightly absorbing much of the recoil. In the sitting position, your butt makes a shooting platform. Your elbows and knees lock in to form a tripod with your back. Very steady position. Kneeling you'll rest your elbow on a knee to the front and a knee and foot support your upper body. The recoil again goes into your shoulder but is dampened by your hip's. Off hand is a terrible way to shoot and I avoid it at all time's unless I'm right on top of what I'm shooting at. No real support. Some people can shoot remarkably well off hand but they have shooting skills honed beyond fine.
Kinda windy, sorry. Hope it helps some. Learn to shot!