I've had up to 9 from shooting outdoors. My recollection is that mercury poisoning used to be treated by sweating it out of people, and, I think, lead as well.
The various imitations of a chemical hood described here are the right idea for casting. They will put an end to fumes. Lead on hands is an issue. Wash after handling, including casting, loading, shooting and bore cleaning. Use a brush. It is when the particles get into your stomach acid and becomes a water soluble salt that it gets into your system. Symptoms include sterility (end of the Roman empire (lead drinking cups), and insanity if you really get a lot. Children get developmental problems.
I built a hood over my workbench at the back end of the garage. Nothing else happens there. Been soldering wires for decades and handled a lot of the stuff. Still didn't get above the current limit. Common sense.
Nick