Most on here have reported better groups with the lighter 32 gr. rounds.
Mine shoots 32 and 40 gr. bullets equally well. However, it is a little different than most, it's a 22" standard barrel factory installed on my .45-70 Buffalo Classic.
You'll love the .204. There is just something magical about watching the hole appear in the target (or the crow exploding) before recoil moves the rifle!