I'm not a fan of swapping barrels, and probably can be accused of the pot calling the kettle black, because all three of my rifles are mixed and matched, but now that I have them on receivers that fit, and also shoot well, that is where they will stay.
I had to do the Handi two-step a couple times with barrels that didn't fit, and it's not as much fun as having a barrel that simply fits like it should, and when you do have one consider yourself fortunate.
It's more the rule than the exception, but still there will be a time when you swap one and it won't work for you.
What I have found is just because a barrel fits, it may not shoot as well from one receiver to the other, so you won't know until you shoot them as I think anytime you mate two seperate parts together they have to develope that fit for each other, and eventually over time wear into "that fit," which will only come from staying mated to the same receiver.
You could do that swapping back and forth between the two, but that would mean each time you did that, it would be another trip to the range just to make sure your still at zero.
Consistency is the key to accuracy, and that goes for the rifle, as well as the ammunition!