The cheap plastic safety selector on my twenty-bore New Haven 600 (house-brand Mossberg 500) broke during a quail hunt when it only had a season and half of use. Fortunately, I had middle-school metal shop class in which to make a new one out of brass. The brass one has been going strong for nearly thirty years.
I had to remove the trigger group and the bolt/bolt carrier when I fixed mine. I removed the cartridge stop, cartridge interrupter, and elevator, too. The saftey slide button screw is affixed to an interrupter block on the inside of the receiver, and you need to be able to hold this in position while you position the safety selector slide so that the detent is oriented to the spring-loaded detent ball (that bearing thingy). It isn't a hard job to do, but you need the receiver pretty much stripped of all the other parts in order to have room to get your hand and fingers inside to hold the safety interrupter block in position while you line everything up and get the screw started and torqued enough to keep things from going "sproing", if you know what I mean.
The M-500 is a pretty simple gun to detail strip and it is equally easy to re-assmeble. This is one of the many things about it that I like. I've had mine down to the bare receiver many, many times during the thrity years I've owned the gun.
You're about to replace the only part of the Mossberg 500 that I really don't care for -the cheezy, chintzy, failure-prone, cheap plastic safety selector slide, and I think you'll find that it is much easier to do than it is to write about!
-JP