I think that if they don't smell you or see you, that they will make a wide circle and sneak back in to try and figure out what spooked them!
If they see you or smell you, it's hard to figure when or if they would return!
A deer never doubts it's nose or eyes, especially if both senses are working together!