I would say to you as I would anyone else handgun hunting. Only shoot an animal at the distance you can keep every shot within a 4 inch circle. If you can't keep every shot in a 4 inch circle, you should not hunt that distance until you can. If you were shooting off sandbags, you should have better groups than that, maybe it was due to you not having your gun sighted in for that distance. Also as drdougrx said, sight your gun in for 50 yards.
Personally I like to keep my handgun shots under 50 yards. I have taken deer out past 140 yards, but that was with my S&W 460 Mag, and a good scope and a good rest. I also practiced out to 150 yards, and kept my groups under 3 inches. I only did it because I wanted to, not to make it something I would always do. If I want to shoot long range, I will take a rifle.
I have been handgun hunting for over 30 yards, and I still prefer up close and personal shots. I compare my handgun hunting to my bow hunting. I use the same tactics and keep my shots close. Practice, practice, practice.