When I first saw this post, I started to reccomend the 22 Hornet hands down till I read it thoroughly. The Hornet can be extremely good for your sons purposes, but with the majority of factory loads it can also blow as big a hole in a fox as a 223.
The 22 LR or 22 magnum in my experience are very unreliable on coyotes. Yes, under perfect conditions you might drop one in it's tracks, but most of the time you'll be left looking for a coyote that you may or may not eventually find.
I'm gonna suggest a caliber that you didn't mention, that would likely fill all his needs, and be extremely cheap to shoot even if he's limited to factory ammo.
I think a 357 lever action would be just perfect. I use a Marlin 1894C for a lot of my predator hunting. Factory 357 magnums, especially from a rifle barrel are horrible about blowing a hide apart, but loaded with 38 specials, a good lever action becomes an honest to goodness 100+ varmint gun. It's quiet, plenty powerfull enough for a coyote at calling ranges, but still will usually leave only a small exit on a foxes thin hide with factory 38 special SPs. Trust me, I've shot enough 38s to tell you that it's a very good hunting caliber for everything from squirrels to big bobcats, or coyotes.
The 1894C will hold nine 357s, or ten 38s. They're light, tough, accurate, and if he likes(I don't), he can mount a good scope on it, which will easily let him hit a fox, or coyote in the head at 100 yards. I truly can say, if I was limited to only one gun for all my hunting, I'd pick my 357 rifle. It'll take squirrels(down loaded 38 special SPs) with very little meat damage, or take deer sized game cleanly at 100+ yards with full house 158g SP 357 magnums.
Your son could shoot bulk 38 special ammo like you see advertized in "Cheaper than Dirt" magazines almost as cheap as good 22 LR ammo. Also, if he did start loading, he could load 38s extremely cheap, and get as many as 1750 rounds out of a single pound of Unique.
I absolutely love my 357 rifle, and I'm fairly certain that if he tries one out for hunting, He'll fall in love with it too.