I grew up shooting a recurve. I think anything over 160 fps is fast. Even at that speed, my arrow will shoot through a deer, so worrying about whatever small speed loss could theoretically happen with a WB is, in my opinion, worrying about stuff that doesn't matter.
I use a fall-away, not a WB. Personal choice is all it is. I don't like anything contacting my arrow once I've released it.