I am not a fly fishing expert, but I'll give some input.
I have a #5 and it works well for small trout flies, but doesn't work very well with larger flies, streamers and poppers like you may use for panfish or larger fish.
I also have a #7, it will cast the larger more wind resistant flies much better than the #5. It will cast the smaller flies, but the presentation is not nearly as graceful as the #5.