As CW stated, it's safe, but on a rifle or slug gun it will make a difference in accuracy, just depends on what accuracy level you're willing to accept, a well fitted barrel will have no movement. Aluminum isn't a good shim material, it will hammer out eventually and be loose again, just depends on how much you shoot it. A thicker shim fitted to the pivot is the best way to get there, depending on headspacing of the chamber, IE if the rim cut is on the deep side, you can remove material from the chamber face down to the face of a chambered round to allow a thicker shim, but be aware that all ammo doesn't have the same rim thickness, I'd only do this on a shotgun barrel if you have several loads to check, and then only with the ammo with the thickest rim. This is all covered in the barrel fitting info in the FAQs, although not specifically for shotguns, the techniques still apply.
Tim