Loading cap & ball revolvers is not a safety hazard. Shooters often load behind the line at the unloading table while checking guns of the other shooters as they finish. Another option often used is to load on a cylinder loading stand/press attached to or on the shooting cart or the unloading table. If you have spare cylinders, load several at the range, or at home the night before the match.
Cap & ball cylinders are not considered loaded until they are capped. Capping MUST be done at the loading table only, or on the line if there is a reload required in the stage, as only 5 chambers are allowed to be capped, but one can be loaded. It is also permitted to use a second staged uncapped revolver for a required reload on the firing line. Capping also should NOT be done with the cylinder removed from the revolver, as convenient as it might appear, as a capped cylinder is a hazard if dropped.
Speed is not a problem with spare cylinders, and even with one cylinder, a loading stand like the "Tower of Power" is much faster than loading with the gun's own loading lever. Actually, under a lot of use, the loading stand will save bending loading levers and rammer linkage.
Loading cylinders off the gun also allows black powder fouling to be wiped off between stages.