OK, I'm no expert and have been waiting on one to reply, but since they haven't I'll make a few observations:
The difference between POI on the first outing and second outing could easily be due to the difference in ambient temperature. You say it was warm the first outing, and freezing the second. That's probably at least a 40 degree temperature change which could easily make a 1MOA change in POI.
As to the groups opening up, I suspect one of two things, or maybe a combination of the two: Barrel fouling and heating.
You don't say whether or not you broke in the barrel. I know there is some controversy about breaking in a barrel, but I know for me it made a difference with the .243 Pro Hunter barrel I bought. Cleaning got easier and easier, fouling less and less, groups more an more consistent as I broke it in. I used the "clean every shot for 5, every 2 shots for 10, 3 shots for 15, 4 shots for 20, 5 shots for 30" method. Then just cause I had 100 rounds, I did 10, cleaned, then 10 more and cleaned.
I know you think the barrel was cooling off between groups, but in my experience there is still plenty of heat left in a barrel 10 minutes after firing a group. If I want to do a "cold barrel" test, I wait at least 30 minutes between groups, usually more. Sure, being 32 degrees outside helps, but not to where the barrel is cold after 5 or 10 minutes.
Anyway, I hope this points you in the right direction. Either that, or someone smarter than I am will hopefully post the right answer.
Just tryin' to help...
Just a Shooter