I have never owned anything other than Lee pots.
Like you I started out with a homemade pot 40 years ago, a few years later I went with the Lee dipper pot, a few years latter I bought my first Lee bottom pour.
It was the Lee 10 pound, I believe it was a better design than the 4-20, in that the handle was weighted, you still needed to turn the spout control rod occasionally too stop the dripping.
The Lee 4-20 I think the first thing everyone needs do, is remove the wood knob handle, and place a large fender washer on the screw for added weight, and replace the wood knob.
Bull plate lube also works great to lubricate bind points on the pot.
The Lee is the cheapest pot on the market.
I have used this one for years, and I believe over time, heat and use has sprung the control arm, this is why I needed to make the modifications.
All that I have read on the Lyman and RCBS pots, both are a great improvement over the Lee with the RCBS getting a slightly better rating.
But both are $300.00 more than the Lee.
I have thought about an up grade myself, But the almost $400.00 will buy a lot of molds or components.
The Lee works ok you just need to know up front its going to be a pain at times.
If you are young and just getting started, yes sell the Lee and get one of the better pots.
At my age I am also Leary of buying green bananas, so I will continue my battle with the Lee.