I have Handis in both .308 and .30-06. I shoot and hunt with the .308 more than the '06, and it has a steady diet of 165 - 168 gr. boat tails. I don't mind the recoil at all, and it is on a synthetic stock. The '06 on a wood stock seems to kick more with a comparable load. Of the two, I prefer the .308. I've had a bolt action .308 for years and really like the caliber and what it can do. When I got the Handi, it just about retired the bolt gun. If you want some long range clout and light recoil, you may consider the 7mm-08.
Pete