You didn't mention much about your expectations for the use of the pistol on your deer hunt. What sort of ranges do you expect to shoot? Do you want a cartridge that will double for some other purpose?
I'll set some of the parameters and say that we'll limit shots <200 yards, which seems to be fairly normal for many people's hunting experiences. The calibers that immediately jump to my mind as great deer cartridges are the 7-30 Waters & .30-30. You could certainly gain some velocity by going to a .260, 7-08, or .308 and you would also get some more muzzle blast. Personally I think cases based on the .308 would be about the max for a 12" barrel as I think larger cases would only buy you more muzzle blast without a huge difference in velocity. I would think that a 7-08 would easily be capable of taking a deer at 250 yards if you are skillful to deliver the goods on the target.
If your distances are <125 yards, then I'd probably be more inclined to pick a big bore cartridge with a flat nosed cast bullet. Something with a bore size of at least .41" like a .44 mag, .454 Casull, .480 Ruger, .444, .45-70 and similar type rounds. These loads eliminate the whole variable of bullet performance by cutting a big channel through both sides. Their only limitation tends to be trajectory as you have to factor in lots of drop as the ranges get out there. Good luck!