Choose your shot and it should work fine. My Dad killed three separate cows at 300 yards. All of them were shot with a .308 and 150 grain bullets. It'll work.
That said, I personally prefer a heavier bullet. I'm not looking for a bullet to shoot through the south end of a north bound elk, or anything else, unless the animal has been wounded. Even so, you may see if you can load a heavier bullet in your .30-06 and see how it does. Even if your groups open up to say, 1.25" or so, you've still got a rifle that will put those into a group of less than 4" at 300 yards. That's plenty good enough for whitetail shooting and an elk has an even larger kill zone. I think, at times, we as shooters get too caught up in group size. When shooters become hunters, there is a need to reevaluate the group size and perhaps forsake some bragging rights for a bullet that performs on the game we wish to hunt at the ranges we wish to hunt them.