Yes, the barrel is essentiallly the only difference, at least in an Italian repro, the original Colt used different gripframe materials in some models, and some details (like the capping groove), depending upon the Model (Year it was made). The '51 was made from 1850/51 until 1873.
However, the barrel, unless you are lucky, is not a drop-in. Any new barrel, even the 51 will often require some fitting at the arbor hole, which might sometimes be too tight. The new barrel should get its own dedicated wedge, which might need fitting. The cylinder gap might be tight and need adjustment. Nothing ourtrageous.
I have fitted a 51 barrel of a different make (marked Colt, probably 3rd gen.) to a 61 Uberti to cut it down to a 5 inch barrel, not wanting to cut down the original 61 barrel.
Check the photo, one is a 51 and one is a 61, both converted with a Kirst (ungated at the time of the photo):
http://i13.photobucket.com/albums/a293/rcflint/pair.jpgI suggest fitting a spacer at the end of the arbor to correctly fit the arbor length to the barrel. Don't permanently attach it if you interchange barrels. See Pettifoggers articles on tuning open-top Colt types.