You will have a completely different feeling in the woods hunting deer if you leave the rifle at home or in the truck IMO. I have carried both at the same time but it is a lot more exciting when the rifle is not with you. Try it and I'm sure you will see what I mean when that chip shot on a deer with a rifle is just a little to far to take an ethical shot with the handgun 
And yes that 44 is awesome for deer!
Good luck hunting with whatever you carry 
Bulletstuffer
Couldn't have said it better. I tried carrying both for a couple days, and discovered the challenge of the pistol was much more satisfying than the "if ya can see it ya can kill it" rifles I had hunted with. It put the HUNT back in deer hunting for me.
I also wish you the best of luck
