Not wanting rain on your parade......but most of us who went the swap barrel route have gone to dedicated frames. The only swap set I have really left is the 22LR/22Mag and the old 12ga. has been staying the slug Buck barrel, I use my 20s for shot, (the Buck barrel in cyl, bore shoots shot really well too, if I 'need' to).
The novelty of swapping changed for me when it always seemed I had a different barrel on than the one I wanted to play with, or needed to grab quick for the issue at hand.
I suggest a Sportster 22 in a short bag (mine fits a Daisy BB gun bag nicely), not much longer than a barrel with a scope hanging off the back end.
BTW, shotguns are soooo cheap complete that there is no reason not to just have one.
It's more about portablility than anything else, weight reduction. I'm thinking I should know what I'm hunting for and barrel up as needed, and that some defensive sidearm would be my resort for problems like bears and bad people. Hence the idea of the sidearm being .357 or .44.
The scenario I'm thinking of is that of being on foot for an extended time.
But yes, given that you're in the preferable situation of being able to store several firearms, yeah, having dedicated frames makes sense. Seems folks that use the Rossi Trifecta complain of accuracy changing with successive barrel swaps. Though other folks seem to get tack drivers. Not sure what's up with that.
Anyway, the overall idea was, "What's the smallest 'covers everything' arrangement." One direction of research was swappable barrels for hunting. (.22/12 gauge/30-06 should be good for all of lower 48) and one good "trouble gun" on the hip.
The other thought was, .22 handgun, .44 handgun and Pardner Pump. Me personally, I'm good to 100 yards with a Super Blackhawk. The .22 covers small game and the Pardner can be made to cover anything. And in this arrangement, everything is a repeater, which is a plus at times. But it's costlier as well. It also leaves out the 30 caliber rifle ranges. But to be honest, most of the hunting I've done in my life was in places where you couldn't see 100 yards. So I've never really found myself saying... "Ya know, what I need here is a scoped, belted magnum so I can hit." I've pretty much always found myself in the, "I need to be a better hunter, find 'em and get closer" camp.