I keep a small bench hammer and a 8" wooden dowel in my barrel box so I never touch the frame with the hammer, just the dowel. Don't have to worry about missing or messing anything up.
Since my hammer head is hard plastic (hobby hammer) I sometimes tap the pin directly in after aligning everthing. My theory is never force it. When things are aligned, it goes in. If not, it hangs up.
Hope this helps!
Mike