You said that you didn't think a 22 cal would do it but the 22 hornet is king for ihmsa field pistol and NRA Hunter pistol. It has very low recoil, accuracy, inexpensive to load, and shoots very flat.
I have been using a 22 hornet for 3 seasons now and am very pleased with it. This last summer I decided to try shooting cast bullets. I have been using the RCBS 55gr cast bullet with 3.5gr of Win 231, rem brass and CCI 400 primers. It clocks about 1675 fps and I use the same sight setting from chickens to turkeys. I go up 4 clicks for rams. 100 meter groups are 2 inches. I can shoot these rounds for about 4 cents each so I practice more and my scores have gone up an average of 5 points because of the practice.
Other cals that work good are the 32-20, 32H&R mag and 25-20. Some of the guys use the 22 mag also and it seems to work good for them but sometimes they get a ringer.
Sixgun