I know my old Buck model 12ga. (smoothie) will rock my world (Im 6'1", 205), but the 20ga Topper with bird barrel and slugs is more civilized. The rifled barrel will result in more recoil than a smoothbore, but if it weighs more than 5.25# that will help. You can add some weight in various ways. I think you are on the right track with the 20 on that weight gun.
Be prepared to have to spend some pretty good money on the ammo; it seems fully rifled bores prefer sabot type loadings. If you havent, go to the Slug Gun forum here on GBO.
If you are going to sit more than still hunt you might consider the USH in 20, or even 12. The extra weight right off will really help tame the recoil and it is ready for scope.
Another option for you, since you want to scope it, would be to get a std model 20 (with the recoil pad!) and have it D/Ted for the scope mount, and add some weight. See how it shoots with the bird barrel and Fosters and/or Brennecke and if need be cut it down a tad to a cylinder bore.