My recommendation would be the .357 Magnum. It's ballistics from a Contender are adequate for whitetails, and the accuracy is outstanding. I'd recommend a 180 Hornady XTP, but Remington and others also offer 180 gr. jacketed bullets. A 158 gr. JHP ( I use the Speer) may offer you less recoil, and can still be adequate if you don't over reach the distance.
A bottleneck for later? I recommend the .257 JDJ in a 14" barrel. It's a wildcat and would require reloading, but it's easy to load for, has low recoil, is extremely accurate, and will work in a Contender. The .243 will not; it would require an Encore, or XP, etc. The ,257 JDJ will run a 100 gr. bullet @ 2620 fps. from a 14". It's excellent cousin, the 6.5 JDJ runs a 120 at about 23-2400, but the recoil is correspondingly greater.
I use a 14" .45/410 with 2 3/4" high brass shells, rather than 3". It's the cat's meow, or something like that.
So, a Contender in .22 LR, .357 Mag., 14" .257 JDJ ( if you didn't need a straight wall, this could serve all deer/ antelope type needs) and a 10, 12 or 14" 410 should serve all your needs nicely, except for elk... although in the hands of an expert shot, the .257 with Nosler Partitions would work on cow elk. I'd have to think hard about the terrain, likely distance, and such before I talked myself into using it on a bull, even if that's all I had.