Nope, they're real tough to knock out for a trigger job, can't imagine one being loose, must be defective and undersized or the hole in the frame is oversize. Is the splined end of the pin on the left side of the frame as it should be? That's the latch pin, so I'd contact H&R and see about getting it fixed.
Tim