I don't have any raspberrys planted, but I have seen black and red planted side by side other places and the color didn't change in the berry.
Whether the taste changes is another story and I don't have that answer. It is possible.
I do have wild ones growing next to my garden and they aren't all that prolific. I like having them around and it may be time to get some others. As long as they don't over-run the place, which is easy for raspberrys to do.