Personaly, I shoot many 38s for every round of 357 and no, this is not an issue.
I do shoot cast bullets in both calibers, and after prolonged firing of 38s in the 357 chamber there can be a ring of lube and powder in the chamber.
AS already stated above, this is not a big thing and as is easy to take care of with just "normal" cleaning.
However, at least in my case I need to do a lot of shooting before before this cleaning is required.
I suppose some lube and powder combinations might be worse, but all in all, nothing to really be concerned about.
keep em coming!
CDOC