You first need to assess your hunting situation.
I'd say if your shots are normally no more than 100-125 yds, the 20 ga is probably fine using the 1,900 fps rounds. Beyond that I would opt for the 12 gauge.
I have the 12 ga USH and couldn't be happier. Every bit as accurate as everyone claims. It's a bit on the heavy side but my only concern is getting it to my blind. If I intended to do any walking at all, the 12 is not the way to go.
As for recoil, I guarantee you if you are shooting a 12 ga now, that the recoil out of the USH will be noticeably less using the same load that you're using now. The weight of the USH absorbs it!
My main reason (and not necessarily a good reason) for not buying the 20 ga was that I or nobody I knew had ever used the 20. I just couldn't convince myself at that time that that was the way to go. In addition, to achieve the same amount of energy and effective range as a 1500 fps 12 ga sabot, you'd need to shoot one of 1,900 fps 20 ga rounds. End result is you're still probably going to have a dead deer one way or the other, but the 12's going to leave behind a bigger hole and maybe a better blood trail by virtue of the sabots size (438 gr vs 260 gr).
There's alot of success stories using the 20 which can't be denied. My son has the 20 ga USH youth model. When he grows out of it I intend to keep him in a 20 by either buying the adult stock and forend or just a new gun itself. We'll have the best of both worlds....one of each....a 12 and a 20!
It's a tough decision. Been there and know what you're going through.
Let me just say there's no wrong answer. Either way you're going to have a fine shooting slug gun.