I wouldn't waste time JBing an aluminum shim, it will just hammer out and you'll be right back where you started. Use steel shims made from a feeler gauge set or regular steel shim stock. Shimming the spacer only tightens the barrel when the action is open, being loose when open has no bearing on how well it shoots, the barrel being tight to the action without the forend on is when it counts, it will feel tighter when locked up with a shimmed spacer, but the gap is still there.
Tim