am I wearing down more rifling with every shot?
Of course you can wear one out, but if you clean it and oil it after each use to prevent corrosion, it would take a long time. Maybe your grandkids will wear it out someday? The high wear part would be the breech plugs flash hole. You may have to replace a plug every 500 to 1000 shots for optimum accuracy, but that is a feature that has been designed into these rifles. It used to be nipples with #11 and musket caps, now it is breech plugs or vent liners with 209 ignition.
There just isn't a one answer fit's all. You have different grades of steel, types of rifling, cleaning methods, cleaning solutions, cleaning rods, gun oils, storing practices, powders, primers, bullets, and several other factors to consider. Conventional wisdom tells us that anytime you introduce a corrosive powder along with heat and pressure to barrel steel, something is happening. You might not be able to see it with the naked eye, but over time with a bore scope, I am sure you could. I just don't worry about it, that is why I have over a dozen rifles and SEVERAL breech plugs, I don't think I could wear them all out in 10 lifetimes.
Now to answer your question. Yes!