The most explicit recognition of private property rights comes in the Fifth Amendment which states "Nor shall [anyone] be deprived of life, liberty, or
property without due process of law; Nor shall
private property be taken for public use without just compensation.
Slaves were private property so ownership of slaves (just one form of private property) is indeed protected by the Constitution. The last time I checked, the Fifth Amendment hadn't been rescinded at the time of the War. And no slaveholder, North or South, ever got compensated for the government taking their private property without due process.
The federal government claimed it didn't owe Robert E. Lee for seizure of Arlington because Mrs. Lee owed $92.07 in property taxes (on property that they illegally occupied). Remember, Virginia was not part of the United States, but Mrs. Lee sent an agent to pay the taxes anyway. The government refused payment from an agent, requiring Mrs. Lee pay the taxes in person (also an illegal act by the gov't.).
Neither Robert E. Lee, nor his wife, as title holder, ever attempted to publicly recover control of Arlington House. They were buried at Washington University (later renamed Washington and Lee University) where Lee had served as president. The couple never returned to the home George Washington Parke Custis had built and treasured.
After General Lee's death in 1870, George Washington Custis Lee brought an action for ejectment in the Circuit Court of Alexandria (today Arlington) County, Va. Custis Lee, as eldest son of Gen. and Mrs. Lee, claimed that the land had been
illegally confiscated and that, according to his grandfather's will, he was the legal owner.
In December 1882, the U.S. Supreme Court, in a 5-4 decision, returned the property to Custis Lee, stating that it had been confiscated without due process.In 1883, Custis Lee sold the mansion and property to the U.S. government for $150,000 at a signing ceremony with Secretary of War, Robert Todd Lincoln, son of Abraham Lincoln. How ironic is that?
SBG