You are right on with your felt vs. actual recoil. There are so many factors to consider.
Grip design, material, and how you hold your gun makes a big difference. I used to shoot a 6" 629 S&W with fairly narrow gripper grips. It was not fun to shoot much, and really pounded the web of your hand. Now I have a 5.5" Ruger SBH with smooth faux ivory grips. I find it is a pussycat even with hot Corbons and handloads. I also have a SRH in .480 with a Hogue monogrip on it. With cast handloads, it can definitely sting a bit. I'm betting I could put the same loads in a single action FA, and it would be most pleasant by comparison. But, that's just me. Other folks don't like the SA grips. Placement of the grip location vs the bore line can change how recoil comes back at you vs. up and away. That can greatly impact perceived recoil.
Some people are sensitive to the muzzle blast and perceive the recoil worse on those rounds. A perfect example is the 180 gr .44 mag. It has a huge muzzle flash, and some people think it is a horrible thing. I've had people in lanes beside me back away from the line and watch. The heavy bullets kick a lot more, but have far less muzzle blast/flash. Similarly, a comp'ed barrel will have less actual recoil, but will have more muzzle blast, causing some shooters to flinch. A good comp can cut actual recoil a very large percentage.
If you can shoot both rounds in similar guns, that is the best way to have a perfect comparison. For example, you could shoot the .44, the .480, and the .454, all in the SRH platform and have a good comparison of the recoil by itself.
All that being said, I'm sure we are pretty safe to assume that the 500 will definitely kick more than the .44 mag!! Corbon and Buffalo Bore's hottest hunting ammo for the .44 mag runs about 1200 FTLBS of energy at the muzzle. Buffalo Bore lists a special P+ .44 for limited firearms that launches a 340 gr LBT-LFN-GC at 1480 fps for 1640 FTLBS of energy. By comparison, The 500 S&W loads from Corbon and Buffalo Bore run around 2500 FTLBS of energy. The highest energy load is a 440 gr hard cast at 1625 FPS, with 2580 FTLBS of energy. Good enough for a fair sized mastodon!!
If you really want that 500, but want a load you can shoot more easily, Corbon also offers two 500 S&W Special loads that are similar to the 1200 FTLBS .44 mag energy range. If you handload, you will have lots of other options.