Texgun: a bunch of years ago when I shot silhouette, and then hunted, with a Super Blackhawk I encountered the same problem you did and recently experienced it again with a rifle shooting at a small target at 100 yds. I had forgotten that all I did back then was to raise my rear sight enough to take a 6:00 (six o'clock) hold on the target so that point of impact was on the X ring.
Since I knew where my point of impact was, all I did was to use Kentucky windage and I never missed. In fact, using that technique I had a better sight picture than trying for a dead on hold. Just my 2 centavos worth, but I hope this helps. Mikey.