I have shot alot of 7-08, and IMO you would be better served with using a 140 gr bullet over the lighter ones. I do not think there is enough recoil difference to matter, but I am a big guy, so it may be different for a smaller frame person. I have an aversion to heavy recoil, and much prefer shooting small caliber guns, but going down 20 gr in bullet weight never seemed to make any difference to me. I think most of the 100-120 7mm bullets are more geared towards varmints, and would prefer a 140 class for deer. If varmints or paper are what you are looking for, go for it.
For powder, I would use Varget. My dad has shot tons of it through his 7-08 silhouette rifles, and it has always produced. Check your manual, but 40 gr should be a good starting point with a 140, slightly more for a 120. Always seems to shoot great, and is less temp sensitive than the ball type powders we used to use, like H414. Personally this is where I would start, and if I could not find acceptable accuracy, I would keep trying different bullets until I found something that showed promise. Haven't seen a 7-08 yet that wouldn't shoot this way, but I've never had one in a Handi, so YMMV. If that didn't work, I would start working on the forearm and following the accuracy tips in the FAQ, then start over with the best load I had previously shot. Of course if your accuracy is really bad, it may be best to start there and then work on different loads. HTH, Billy