Understand that the reason for using soft tin lead alloys with black powder is to provide bullets which are soft enough to slump and fill the barrel when the blast hits. This being mandatory because the bullet design has a majority of the bullet way undersize so it will chamber with a fouled bore. In other words. If the bullet nose has a relitively long bore ride section which fits loosely in the muzzle of the gun, a soft alloy which lets the nose slump to fill the bore will make this style bullet shoot best. Since black is always compressed, the traditional method of finding top accuracy is to play with the alloy (hardness) until to find what shoots best. If the shooter recovers fired bullets which aren't damaged by impact, such as bullets which land in snow, the most accurate loads with this style bullets will always be a straight cylinder with rifling full length, except for the nose form. Roundnose shoots better than pointed types with loose br noses because the roundnose has little weight going off center due to slumping which is never consistent. On pointed types the point will wobble enough to hurt accuracy.
So, don't anneal unless the bullets have an undersize bore ride nose, and if you do anneal, you may not get them soft enough to get the slump require for optimum accuracy.
Depending on the alloy used, annealing #2 alloy will result in approximately 8 bhn immediately after the cast, with hardness gradually increasing to 12 to 16 over a 2 to three week period.