Here's something you might wanna try.
Shoot at a paper target while holding the sights right as you want to hold them. Make a grouping (will be low as you said) on the paper. If you can't see the grouping from where you sit, go blacken it with a marker so it's visible.
Now hold your sights exactly the same way as went you just shot. With both eyes open, you should be able to see the grouping with your weak-side eye. You know, with both eyes open the sight kind of look like a transparent ghost, allowing you to see the previously made bullet holes.
Where those bullet holes appear on your front sight is exactly how far down you need to file the sight to. If the holes are so low that the barrel covers them, you need a higher rear sight.
I had a Pedersoli Kentucky .32 flint. Great little gun. Accept Pedersoli has made a big mistake with their .32s. They drilled the channel for a 5/16" ramrod. That's too snug for a .32. The ramrod works just fine, but it rubs against the bore. There's no room for a muzzle protector. The sand and grit that rides on the ramrod erodes the muzzle.
The gun went from <1" groups to >6" groups over the course of a few years. Pedersoli replacement barrels are outrageous! You can buy top-of-the-line swamped barrels for less. So I filed down the muzzle until I could see the rifling again. Back down to <1" groups.
Pedersoli could remedy this simply by offering these guns in .36 or making them with 1/4" ramrod channels. I doubt they will, some companies are stubborn.