Personally, I just whale away with my 223 Savage. It takes A LOT to heat up and it can be rebarrelled easily enough for cheap price. The Handi, the M70 seems prone to heating up very fast (is 270), the Remington in 7-08 heats up slowly, but it has an outstanding bedding job on it and remains very consistent.
The 223 Handi that I formerly owned had a lug that was too tight on the FRONT of the forearm hole. As it got hot it would walk shots up the page. I relieved the wood with a knife and file and it shot much better.
My take has always been that it is mainly an issue of erosion in a properly bedding/constructed rifle. The SKS's limited accuracy goes to pot very quickly, for example, because it has a gas tube, piston, poor bedding/inletting, etc, that heat/cool and expand/contract at different rates. I agree with the explanation from Oneshotonekill, but I will qualify it by saying that 3-5 shot groups are not a big deal IMHO. Automatic weapons are a different animal. Sustained fire weapons have open bolts, changeable barrels, etc.