I normally hit delete for answers that come in ahead of mine, fellas, but will leave the above two in place, with the following comments. (Also, the word theory, and all theory is forbidden on this site!) Just the facts and facts only! My 'theory' is, better to say I don't know and seem ignorant than to give a theory and be known as wrong!
Fillers work fine, like cornmeal and grits, if they are really dried well, but they are hydroscopic and the moisture they contain in damp weather, if not dried with heat before use can mess things up with the powder. Because of this I recommend granulated plastic shot buffer. However I don't recommend filling more than 1/3 of the case with it. If the powder being used takes up less than 2/3 of the case, use it without filler, if it's and easy to ignite flake like Unique or the other pistol powders, or switch to a powder that better fills the case, to at least 2/3 of capacity and filler that.
Do NOT USE DACRON OR ANY FUZZ BALL FILLER! THEY ALMOST GUARANTEE A RINGED AND RUINED BARREL!
My main use for shot buffer filler is when the gas check of the bullet wanting to be used, must protrude into the powder room, or if leading problems arise before I reach the velocity level I want. The plastic filler keeps the gas checks from inverting and wipes up any leading, when used as stated above.
Whenever using a filler, ALWAYS use some compression so there is no mixing of powder and filler. Also, don't just ad filler over a charge that is developing some pressure signes and pressures will absolutely rise dramatically when filler is added, due to much faster ignition and powder burning rate from the powder confinement. Back the charge off a few grains, filler it and try the load. Work up as seems prudent by heeding pressure signs.
My experiance with the Lyman cast bullet manual. I've found VERY FEW loads that could be worked up even close to their maximum without heavy leading and wild inaccuracy, with any bullet, but especially with any of the Lyman designs. The fast powder loads prooves the worst offenders for me, except with the lightest loads given, or close to the lightest. If you have lots of Unique to burn, and want to do it with a rifle, do it with light plinker loads. Buy some slower powder, preferably ball type, for heavy loads.
A neighbor who owned a garbage service 20 plus years ago brought me two large cans of Unique someone threw out. I liked it because it was free, but used it slowly because I don't like it good enough to buy it for anything. I think I proabably still have most of it, if it wasn't stolen back there when calamity hit us 11 years ago. Still don't have all that sorted out yet.