You might want to try a heavy sabot/bullet combo and lighten up the powder. You probably got a 1:28 barrel back as did I. So far, mine seems to like 300 grain or heavier bullets and I NEVER go past 100 grains of powder these days. The White Powerstar 350 grain bullet does pretty well in mine, pain to load but they shoot decent. 300 grain XTPs also show promise in the rifle. I'm trying to find a conical that will do okay and my Whitworths sized to .450 do - sometimes better than okay.
And, if it's any consolation, your first barrel shot MUCH better than mine. It was HORRIBLE! And Don, if you are in a trading mood, I have a good A&H .50 caliber 420 (really) ..... or how about an Optima .45?