If you have a tight constriction mid barrel, say under the rear sight or someplace, it could "swaging" your bullets at that point, then as the barrel opens up, it partially "loses it grip" so to speak to the bore. Slug it, see if you can find any constrictions, if you can, firelapping may help. If you don't have any constrictions, it could be a bit oversized bore and using cast bullets of a little larger diameter might help. Something around .001 or .002" over groove size might work also.