Cast bullets do not change in diameter AT ALL during the hardening cycle, whether air cooled or quenched, nor does sized diameter changed if bullets are sized soon after casting, before the hardening cycle has taken place.
The hardening cycle will go on as if not sized, if bullets are sized soon after casting. Also, you will read about bullets being softened by sizing, so I'll enlighten you on this matter also. The only portion of a bullet which is softened is a very thin skin of the surface where sizing actually occures, and this thin slightly softened skin has no effect on cast bullet performance. It is the overal hardness which controls obturation in the barrel, and expansion on game.
The thin softened skin of the sized surface, is so thin that very special hardness testing equipment is required to measure it.
Regarding any concern about bullet hardness, effect from sizing etc. Judge by bullet performance ONLY. If you do some new trick and get results doing it, do it again, regardless of what you might read contrarywise. ie. If changing something cause leading to increase or accuracy to fall off, or bullets to come apart too easy on impact, that trick is a bad one. If there are no negative effects, the trick is a good one.