Oly probably has it right about poor light. Otherwise if the light is good, the Contender factory sights are very good. I'm 65 and wear glasses and I bought into the common lore that old guys can't shoot iron sights. Then I tried again, for the same reason you want them, a scope clutters up a nice carrying gun. I find that a a set of good post and blade sights like the Contenders have are not impossible for me to shoot.
I have learned to pay attention to what coaches from long ago were saying, focus on the front sight. Let other parts of the sight picture blur a bit. Hard to do, we want to see that target, but going back to basics has revived my interest in the irons.
Another thing that can blow your mind when shooting irons is strong light from the side. I have been ready to throw some of my good shooters away when I get an attack like this, but when I shoot the same gun under different conditions all is well again.