I think youre right on track about the little ones. The good ones don't mess around, they grab your bait and try to rip the rod out of your hands. Its always the little ones that are hard to get the hook into. Nothing scientific, just my observation. Next time hook one of those 10" channels through the tail and throw him back out there. I know it looks and feels rediculous, but that feeling goes away when you're trying to land a catfish that's bigger than your average 1st grader. I'm serious, give it a try.
good luck
SHB