I've only taken one deer (smallish doe) with a BP handgun. Broadside shot @ about 30 yds, both lungs, full penetration. Deer ran just a short distance before piling-up.
It was a CVA "Kentucky" pistol, .45. It shoots extremely well with the 250 gn TC Maxi, crisco/gulf-wax lube over 22 Gns of Pyro P.
Velocity is only 830 fps (chronied), but that Maxi penetrates extremely well (even though it doesn't always expand).
Sorry to sound redundant, as this has been said so many times before, but if you hunt within your weapon's limitations and make a good shot, your deer will fall.
Your .50 should be plenty capable if you work with it.
Skillet