I wouldn't think it was the die as much as differences in the brass hardness and the amount of "spring back" it may cause. The die "should" be repetitive in dimensions, the reaction of the brass not so much. Each firing/resizing cycle will change it slightly, the use of annealing.I measure a number of my cases (headspacing) with a RCBS headspace micrometer. My die headspaces very consistently. I believe the worse I've seen is about 0.002" and more typically right on or within 0.001".
It's hard to believe a match grade die is sizing so erratically that it's producing 100 or so fps spreads. But, it's certainly worth looking at. I'll do 25 cases and measure the headspace - that seems like a worthwhile thing to do/know!
I've read of annealing being more for extending case life than neck tension, etc., but I imagine it would have an effect on all aspects of the shoulder/neck area. I think annealing after every firing is a bit obsessive. I've always felt it was cheaper to buy more brass than invest in a quality annealing machine, but for those who are really into that, they probably feel differently.