Technically, all the troops belonged to the States from which they enlisted. Their units may have been assigned to Army-this or Corps-that, for command and control purposes, but they were enlisted by the individual states.
Most states would accept a privately-raised "unit" into their "regular service". If someone wanted to be a captain, for example, he could equip & uniorm, and enlist to him a group of at least 50 men (assuming he had the money). If they were found to be acceptable for service, they would then be incorporated into the state's regular troop list with him as the commander. Commisions could also be purchased from the states by paying for them, and suitably connected individuals of the higher levels of society would often be commissioned "gratis". Some folks enlisted as private soldiers and were commisioned because of individual merit.
Late in the war, states sent out "enrolling officers" to enlist soldiers and enforce the draft. States also had things similar to "home guard" units, made up of those otherwise unfit to fight, but even they are considered veterans, and rightly so.
If your gggp was a captain, there should be some record of his service somewhere. But he or his widow would ordinarily be paid their pensions by the state for whom he served. Some widows were paid from the US Treasury after 1900 or so, but the application by the soldier would be sent to his state.
