...I do my casting outside.  . .
Yeah you should. 
Lead fumes are very easy to suck up 
in an enclosed area, even with fans 
and such. A fellow I knew years back 
used to cast bullets on a large scale 
with his dad, and the cumulative effects 
of doing it indoors for so long nearly 
killed both of them. Messed his dad's 
brain up