As Microboomer said, you need an intimate mixture, and that requires the aluminum to be finely powdered.
This accomplishes two things: firstly, it ensures that the metal and the oxidizer have enough physical proximity to each other for the reaction to initiate, and secondly, that there is enough surface area for the heat to initiate combustion.
To visualize this second point, imagine trying to set fire to wood with a cigarette lighter.
If the wood is sawdust, then it will be easy to light.
If the wood is a log, it will be practically impossible to light.
We light logs by using tinder (very large surface area/volume ratio) to light kindling (large surface area/volume ratio) which is then used to light the wood (low surface area/volume ratio)
Of course, if you want to light a log with a lighter, it is possible, but given the same size flame, it will take much longer.
Imagine you had tried to make your flash powder without using your sister's rock tumbler. It wouldn't have worked, because the particle size of the reagents wouldn't have been small enough.
The reason for the fire warning signs around the magnesium machining equipment was that, when the magnesium is machined, 'fines' (small particles) are produced, giving the high surface area/volume ratio required for combustion.
It's the same as my wood analogy above, the magnesium billets are the 'log', and the fines are the sawdust.
I bet there were no fire warning signs in the area where the magnesium billets were stored before use.
Similarly, aluminum fines can be very dangerous, but you don't see such warning signs on a roll of aluminum foil.
A good example of this effect is a swedish firesteel.
These are not made of steel, but a substance called ferrocerium, which is pyrophoric, ie. it ignites spontaneously in air.
This reaction does not take place until the firesteel is scraped, shaving off fine slivers. These slivers have a large enough surface area/volume ratio for the spontaneous ignition to take place, and they burn, producing the sparks.
