If theres contact I'd take it off.
There's movement of the barrel metal and everything else when the gun goes boom, I wouldn't want anything in the way to interfere with that natural movement.
All it would need do is push just a little the wrong way at the right time, an you'll wonder why you can't keep the scope zeroed.