I totally agree with wejim. I have hunted deer for years while being able to see two bear baits from my stand. One about 60 yards away, and one about 110 yards. In fact my deer stand is used by the bear hunters (rifle) for the 60 yard bait.
Sometimes the seasons overlap which causes some people a little concern while walking out after dark in an area where they know bear are being baited. I always hum a little Irish tune just to let Brother bear know that I'm there. (He probably already knows without my song) He also knows Irishmen are too tough and stringy for good eating. In either case I wouldn't worry about its bothering the deer or you.
Good Hunting
williek