My choice has always been the handloaded 45 colt.
My load is a 300 Gr Speer with a healthy dose of 296 powder. It chronographs 1200fps in my 7.5 Bisley Ruger and 1625 in my Marlin 1894.
I have killed 6-7 hogs up to 350lbs as well as half a dozen or so Whitetails. Thru and thru penatration on all. One pig and one whitetail was shot end to end, brisket to behind. Taking out ribs, shoulders and shattering a ham bone on the pig! This works for me, I see no reason to change.
I have never been a 44 mag fan, but (Have to admit) would work as well. (I dont care for the 30/06 either. Guess I'm different like that?)
I love the 357 but its not a deer caliber. In a rifle it comes close.
You did not mention the 357 Maxi...Now that is a caliber I hold near and dear... 180 Speer flat point, 1675fps from my 10 Contender, 2150fps from my 22" NEF Handi rifle. Both with AA1680 powder. This caliber lays 'em low!!!
The 41 Mag is a good choice, as would the 10MM. (With at least 200gr bullets) I load 41's for my Father. He has had quite a run with Remington & Hornady XTP's in 210gr for Mr. Whitetail. His guns, a 6.5'' Blackhawk and Marlin 1894.
I have no experience with Rugers 480 cal, but paper balistics show it to be another good chioce.
Good luck,
CW