Boy, talk about opening up a bucket of worms.
I own several sets of camo, jeans, shirts, jackets and coveralls. When I hunt, I usually wear them, I even like to wear camp when bass fishing. (they say the bass can't see ya as well?) . As for wearing them hunting, I'm sure they give you an advantage when bow or pistol hunting, but I am not convinced they are necessary when rifle hunting. I would never consider a solid white shirt, but other than that I have worn every color of the rainbow at some point. As far as I know, it never stopped me from making a kill.
If you are wearing coveralls, there is no reason you can't wear any thing you want under them.
Scent........thats a different story. I do think scent can make a big difference in your success, but here again, it is not as important when rifle hunting as it is when bow or pistol hunting. I have set in a blind puffing on a cigarette and still managed to see plenty of game. I have smoked a butt, walked to a ground blind with my bow and seen deer sniff me out and turn away. Since I quit and started using scent cover things have gotten better.