I think you have gotten some very good advice here.
My first varmint rifle was a 22-250 and I have hardly ever been without one during the last 25 years.It's a great round(as is the .243) but for use in a Handi, it would not be my first pic.
In a Handi rifle, my first choice would be a .223 for all around use .At the 2-300 yard mark, the round will pretty much do everything the larger rounds will with less expense and barrel wear.
I also think for volume shooting, the .223 based rounds are easier on the handi rifle frames due to the reduced trust on the action.Even though the .223 operates at high pressures ,the reduced size of the case head means it is not more stressfull on the frame than mid range 45-70 rounds.L
The .204 is a heck of a round for top performance sniping vermin past 300 yards, but a good .223 load runs a very close second and for your use,I would definitely still with the .223.
For best accuracy in handloads be SURE and try 40,50g and 55g Hornady V max slugs.Over the last 10 years, in .22 centerfires ranging from .22 hornets to .22o Swifts those bullets have consistently outperformed nearly any other brand of slug I have tried in my rifles.The Vmax Hornadys are a heck of a varmint bullet!