It's the size of the pistol vs. the trigger pull that is causing it, more than anything else.
Bring the trigger right up to where it nearly brakes, pause/concentrate/align sights, and break.
Practice practice practice and you should be able to get "minute of man" at 25yds--it sounds bad, I know...but it's a defense pistol, face it. The PF9 itself is much more that 4ft accurate at 25yds.