I wouldn't even estimate the number of deer I have killed with a .243, but it has averaged about 4-5 deer per year for the past 45 years. I shoot a mild hand load, 33grs of IMR3031, with a 100 gr bullet. It is mild, but very accurate in my gun.
I managed a large spread for many years, during that time I leased to hunters and saw just about every caliber from a .17 up. Many deer were taken with some of the smaller calibers, but more deer were lost with the calibers smaller than the .243 than I can count. After a period of several years where many deer were lost to the .22 class guns, I put a stop to using them, and set a .243 minimum limit, the number of deer lost was reduced significantly.
One point that was brought out in one of the above post, Caliber makes little difference if bullet placement is not in the correct place. I also found that a lot of people could hit the x ring on a regular basis when shooting from a bench, but when looking at a game animal they couldn't hit the side of the barn. That said, it is my opinion that caliber does make a difference in killing deer sized animals, A bad shot with a larger caliber will often allow for another shot, while the .22 class guns don't do enough damage to slow the animal down, it jumps the fence and is lost forever to the vultures..............