Like semisane said, a loose fit is not what you want in a muzzle loader. You want soft (pure, or real close to it) lead, and a tight, friction fit. If the fit is too loose, or the mix to hard, the powders blast will not seal behind the bullet. It will cut around the bullet, melting it, causing leading. Improper lubing will make it worse (I like all natural lubes, like "wonder lube"). Another thing to look for is a hollow base. This allows the bullet base to expand upon firing to better fit the bore, and stop gas cutting. Another trick that might help is using a lubed wool / felt wad between the powder and bullet. These days I usually just use the Hornady Great Plains Bullt's in my .50 and .54's. They work well in all respects. I do use a heavier powder charge, usually 100 grains with Pyrodex R/S. I guess it is possible that the 60 grain charge you are using is not enough to expand the base of your bullet upon firing so as to fit the bore porperly.
Larry