Given that I have no experience at all with ported barrels, I'll nonetheless venture an opinion.
No problem. If there were a problem, S&W would not have done it. Must be at least half of all .44s are shot with cast bullets anyway.
Just keep the bore clean - that's not an easy thing. You might want to take a close look at your bore before doing any shooting and think about hot-lapping it. Maybe reaming the throats. Somewhere below I wrote a long sad story about the nasty factory bore of a Model 21 .44 Special and all the trouble I had with leading, and how I fixed it.
-Don