Here is Strat O Matic's argument, and from playing both Strat's and Diamond Mind's computer version, I agree. They are both realistic and statistically accurate, and you can't really say one is better at replicating teams and players better. I have tested this too with multiple replays of individual seasons (I don't work for Strat), and am amazed how accurate the stats compare to real life.
WHAT MAKES STRAT-O-MATIC THE BEST?
Why should you play Strat-O-Matic computer baseball and what makes it better than the competition? These are questions that many people have asked and in this short essay we're going to try to answer these questions.
First off, let us state our position (and the position of many people who play our game): Strat-O-Matic is the most statistically accurate and most realistic representation of baseball on the market.
Now how does one go about proving this. Well, the first part (the most statistically accurate simulation on the market) is quite easy. Does this mean that Strat-O-Matic's game generates the exact, or near exact batting averages for all players? No, this is not what we mean. Nobody we know would want to play a game where every player's batting average came out exactly correct. If you are interested in that you really want to be using an electronic baseball encyclopedia instead of playing a baseball simulation. What we mean by statistical accuracy is how much variance there is from a player's real life and replay totals.
And how does one prove the second part of our statement (that Strat-O-Matic is the most realistic representation of baseball on the market)? That will take a little looking into, and we're confident once you start looking, you're going to like what you see.
STATISTICAL ACCURACY
Just about every baseball simulation that hits the market is advertised with the claim that it is "the most statistically accurate game ever developed". We at Strat-O-Matic sometimes laugh amongst ourselves when a new game comes out because the claim is so often used that it has become a clich�.
So what is the story with statistical accuracy? The truth is that there is an expected (and measurable) range of statistical variation that you should see when you run a baseball simulation. Anyone who has taken an entry level course on statistics and probability would know about Standard Deviation and could easily compute what that range should be.
If a computer simulation is performing better then that range then it is "rigged" so that the replay "comes out the way that it should". Certain events would not occur because of a random generated outcome but because the batter was "behind his pace" and needed a little "help" to catch up. A few computer baseball games have come and gone that use this fallacious method of generating outcomes. They are not popular because gamers instinctively know when a model is not providing enough variation. After all, what is the point of starting a replay if you already know what every batter is going to hit?
Strat-O-Matic is as accurate or more accurate than any simulation that uses a truly random model. That is because when you run a Strat-O-Matic replay you will see a distribution of variance that is perfectly in line with what you should receive in a truly random model. For any given number of real-life at-bats and average you can compute the expected statistical variance you should see if you ran many replay. Strat-O-Matic hits that mark perfectly. However, it is important to note that not every game out there does.
When preparing this article we tested other simulations for statistical accuracy. We ran three replays with each of the following products. Here is a chart of the most accurate of the three 1996 replays using both of these products:
1996 AL AVG. 1996 AL E.R.A. 1996 NL AVG. 1996 NL E.R.A.
ACTUAL .277 4.99 .262 4.21
Strat-O-Matic .278 5.05 .261 4.20
Competitor "DM" .278 4.95 .264 4.22
Competitor "AP" * .259 4.36 .244 3.77
*Note: Competitor "AP" offers add-on tools that might improve statistical accuracy. These tools were not used for this test.
Since the Strat-O-Matic and "DM" competitor were so close we decided to perform a detailed study of these two games. In this study we compared the replay totals of all batters who had 400 or more at-bats with their real-life batting averages and home run percentages. There were 172 batters who fell into this category and here is the result of the comparison:
Average batting average error Average hr % error
Strat-O-Matic .015 .007
Competitor "DM" .016 .007
So the average batter (with 400 or more at-bats) varied by sixteen points from their actual batting average using DM and fifteen points using Strat-O-Matic. Clearly both games are excellent at reproducing statistics and are in the expected range of statistical deviation. However there are other games available that, despite the claims, are not nearly as accurate.