I have a W&H and typically shoot 265g FPGC's (which actually weigh out to 289 grains). I have found decent accuracy with IMR-3031, IMR-4198, and 2400 powders in combination with these bullets. However, recently I tried pushing the loads to higher pressure and velocity and started to get scatter and keyholes. This was especially present with the 2400 powder.
When I first received my rifle I slugged the barrel at .379 and pushing near .3795. The bullets I purchased are supposed to be .379 in diameter, so it would follow that maybe the bullets are not getting enough spin on them because they are not fully engaging the rifling however, what seems odd is that at the lower velocities they shoot great. One would think that at lower velocity they would have even less spin and worse stability in flight.

If you're wondering why I purchased these bullets; well, when I heard all of the talk about narrow chambers etc, I figured I'd start out loading this round with a few .379's and hope that obturation would lend a hand as opposed to reaming a chamber before I ever even shot the thing. As it turned out, I got good accuracy from the start and decided to stay with the .379's as opposed to changing anything.
Anyway, in a nutshell, maybe you should try cutting your load back to see what happens. Perhaps at a lower pressure/velocity you'll get better results.
Another idea might be to seat your bullets out further and almost resting on the lands. Perhaps the jump between the bullet and the lands is causing your problem.