One should be real careful about breathing lead vapors from the melt pot, as vapors do come off any time the alloy is molten, and especially at high temps. Just vent it well or keep the wind at your back if outside and you'll do fine. Wash your hands before eating after handling lead, and relax. Take plenty of calcium suppliment also, as it's a buffer against lead toxity.
Perhaps of interest here is that a good sized area in Missouri there is so much lead in the ground that well drillers have problems with bits plugging up with it. But when they hit water they drink it and no one gets sick from it. Also, most of the old cities in the US were plumbed with lead pipe water mains, which have never been removed and are still in use, with no toxity problems.
The most toxic problem we have with lead is the fact that our government is trying to get it out of the reach of bullet casters.