Try LIGHTER bullets! Heavy slugs have more barrel time, resulting in the muzzle achieving greater height in recoil before the slug exits, at 50 yards drop is going to be negligable.
Irrespective, at 50 yards that is way too high, something isn't kosher with your scope mounting proceedure, I'd try the above suggestion of swapping rings.