Handi's are notorius for throwing the first shot from a cold barrel wild. I don't know how to tell you how to diagnose the gun, but I do wonder what primer you are using. It is possible that the primer is not up to the task (cold) change primers around. Also it sounds as though you are loading the XTP's longer than the first cannalure, perhaps in an attempt ot get closer to the lands. Forget that. Seat in the cannalure, and give the load a good crimp, consistant ignition is far more imprtant than bullet jump. This ain't no rifle cartridge, it is a pistol casrtridge and should be treated as such. 44 Mags commonly have a long jump across the cylinder and barrel throat in revolvers. Don't affect accuracy as much as you would think. Many other guns have long throats and shoot very well, the Weatherbys are just one example. Seating to the lands is not a hard fast rule. In this case by seating long, you are increasing the case volumn and altering the pressure curve. This may well be the cause of the flyer. Powders like H110/WW296, AA1680 will brun best in the 44 slightly compressed and with some good resistance that gets the powder charge burning completely.
Also, good 100 yards groups from a 44mag rifle look different than 100 yard groups from a 308. 2" is quite good. The measure of a good 44 Mag rifle group is not how small but how consistant. If your load will always put 5 shots in 2" you have a very good rifle. !" groups will be more the exception than the rule. Remember, this is a hunting rifle, not a bench rifle. A different set of rules apply.