You'll be fine using the .312" in your .30-06 from a .32 ACP. The only problem would be either using .312" bullets in a .30-06 case with the same max or near max loads for .308" bullets, or that they would not even chamber if the .30-06 chamber were tight. Otherwise the amount of difference is minimal; now accuracy is a whole other topic, but strictly speaking safety, there will be no problem.