Many years ago, with the components available at that time, the .25-06 was an outstanding long range varmint cartridge. Built on accurate rifles it could deliver better than most other cartridges....but today that has changed.
Actually the .25-06 has gotten even better, but many other cartridges have caught up. For shooting at 500 yards today you can use a hot .22 centerfire or a 6mm and get almost-comparable performance for several reasons. First, modern components give flatter trajectory and more remaining velocity on-target, and second they are easier to hit with. Recoil is important in long range field accuracy, and the shooter's ability to deal with it makes the difference between hitting and missing. It is easier to control a .243 in the field than a .25-06 - simple physics. Low drag bullets and rifle technology have made huge strides in bringing lesser cartridges closer to parity.
But I'm not dissing the .25-06 at all. I've varmint hunted with my .257 Weatherbys and they are amazing....but they are not ideal varmint rigs. Hitting a chuck at 500 yards with a 100-grain .257" BTip does 'move' you more than the same hit with an 80-grain 6mm bullet though.