My first thought is the patch is not holding as the patch/ball combo is too loose..
Go to a .495 ball & a Oxyoke or pillow ticking patch. Start at about 60 grains & shoot groups of 3 off a bench with sand bags. Go up in 5 grain increments on each group to about 85 grains. Take your time & make sure the Rifles is doing the shooting, not you.
If you are swabbing between shots, do Not swab with a tight patch, but rather with a just barely snug patch like tee shirt material. Get a 2 oz pump/spray bottle (WalMart Cosmetics) and on dry days, wet the tee shirt patch with 3 squirts about 4" from the patch. Swab down & back One Time. (We are swabbing, NOT cleaning. The objective is to keep the barrel in the same fouled condition, not clean the barrel each time.
On the loading patch wet it with 2 squirts. If you find this too wet, got to 2 squirts to swab & 1 squirt to load.
Also it helps if the cleaning jag first land is about .010 undersize of the other lands on it. This lets the patch push past the cruds & then grab the patch & cruds on the way out. Also sharpen the lands of the jag with a 3 corner file & the jag in a drill so that all the lands taper back from the front & have a sharp edge.
No telling how many thousands upon thousands of rounds I have shot with Lehigh & have always had accuracy in all of my rifles with it. I don't use it now since they changed the formula of Lehigh, but I do still have some & I know the new stuff will still work, it just takes more of it to get the same lubrication that the old Lehigh had.
Also, in the past 35 years I have owned over 50 dif. ML's (some store bought and some hand made) & on Almost every single one them the most accurate loads on the rifles from .32 to & including .54 cals., shot the most accurate with a RB usually .005 under bore size. Yes, they are harder to load, but a Great difference in accuracy usually. Once in a while I have found an exception, but normally the tight patch/ball combo shot the best.
Hope it all works out for ya.
Birddog6
PS: If you don't understand how to do the jag with the drill & file, email me & I will send you some photos of modified jags. birddogsix@yahoo.com
www.custommuzzleloaders.com