Manuals are guide lines. And each use different "test" barrels with different lengths, different primers and different brass.
Each load needs to be worked up to what the individual needs for his rifle.
That said, it gets easier as you get to know your gun, what it likes and what it doesn't. I haven't used "starting loads" for quite some time now. I can start mid-range of the load data and work up and I have a few favored loads that have gone over book max with-out pressure signs. But when I get close to max I go really slow and carefully look for any signs of problems.
The Lyman manual comes highly recommended on this site, but it's actually my least looked at book. I'm glad I have it for the "cast" data, as it might come in handy if/when I move in that direction, but as you say the data in some cases is very conservative and of little use to me. It's good to look at several differant sources.