Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Alternate Baseball Universes

kdawson posted more than 6 years ago | from the say-it-ain't-so-joe dept.

Math 229

Jamie found a NYTimes op-ed by a grad student and a professor from Cornell, outlining some research they did into alternate baseball universes. The goal was to find out how unlikely in fact was Joe DiMaggio's 56-game hitting streak, played out in the 1941 season. No one since has even come close to that record. The math guys ran simulations of the entire history of baseball from 1885 on — 10,000 of them. For each simulation they put each player up to the plate for each at-bat in each game in each year, just like it happened; and they rolled the dice on him, based on his actual hitting stats for that season. (Their algorithm sounds far simpler than whatever the Strat-O-Matic guys use.) The result: Joltin' Joe's record is not merely likely, it's basically a sure thing. Every alternate universe produced a streak of 39 games or better; one reached 109 games. Joe DiMaggio was not the likeliest player in the history of the game to accomplish the record, not by a long shot.

cancel ×

229 comments

How likely is it that the researchers have sex? (-1, Troll)

Anonymous Coward | more than 6 years ago | (#22914992)

I just flipped a coin a few times, and it seems they never do.

(For those wondering, tails is sex, and heads is no sex, per Bill Clinton rules)

Re:How likely is it that the researchers have sex? (-1, Offtopic)

CRCulver (715279) | more than 6 years ago | (#22915012)

I thought it was sad how the lead researcher is constantly tormented by voices whispering, "If you build it, they will come."

Re:How likely is it that the researchers have sex? (-1, Troll)

Anonymous Coward | more than 6 years ago | (#22915280)

I think you're fucking sad Christopher. Check out how often you post on Slashdot. Get a fucking life. I rarely come here yet I see posts from you all the fucking time. Face facts - you're a loser and none of these fucking posts you make mean anything. You are wasting your life with trivial bullshit. This is the only time I will offer you help.

Either find something meaningful to do with your life or go fucking kill yourself. Asshole.

Someone please mod parent Troll. (1, Troll)

Enderandrew (866215) | more than 6 years ago | (#22915902)

Someone please mod parent Troll.

If its so likely, they why hasn't it happened? (4, Interesting)

quanticle (843097) | more than 6 years ago | (#22915004)

I know the statisticians among you are going to bash me with a cluestick for such a naive question, but I'll ask anyway - if this event is so likely to occur, then why hasn't it happened again?

Re:If its so likely, they why hasn't it happened? (4, Insightful)

Anonymous Coward | more than 6 years ago | (#22915018)

Clearly they aren't factoring in the stress and nerves the average ballplayer would be dealing with as they got closer to the mark.

Re:If its so likely, they why hasn't it happened? (0)

EdIII (1114411) | more than 6 years ago | (#22915056)

It won't occur again. That's not their point. What they are saying is that if you took those games and played them out with other player's performance stats, then it is actually highly likely that another player would do the same thing or better.

They are just re-running 1941 over and over and over again and putting different players in DiMaggio's spot. So Joe was not necessarily good, just in the right place at the right time according to the research.

Re:If its so likely, they why hasn't it happened? (3, Informative)

Martin Blank (154261) | more than 6 years ago | (#22915094)

No, they reran 1871-2005 through the simulator a total of 10,000 times. This is clear not only from the statement that says as much ("Using a comprehensive collection of baseball statistics from 1871 to 2005, we simulated the entire history of baseball 10,000 times in a computer"), but from the mention that the record was set most often in 1894.

Re:If its so likely, they why hasn't it happened? (1)

EdIII (1114411) | more than 6 years ago | (#22915146)

Okay fair enough and I stand corrected, but my point is still valid. The research is not about predicting future games that have not occurred, but about predicting different outcomes or possibilities or past ones.

At least that is what I understand from the article, it is mostly about how Joe DiMaggio could have been substituted by another player of similar stats and the same records would likely be achieved.

Re:If its so likely, they why hasn't it happened? (1)

maxume (22995) | more than 6 years ago | (#22915592)

The other poster was wondering why it didn't happen in 1929, and then with DiMaggio in 1941, and then again in 1942 and again in 1955, not if it was going to happen in 2008.

Re:If its so likely, they why hasn't it happened? (2, Interesting)

Martin Blank (154261) | more than 6 years ago | (#22915062)

It was likely to occur early in the history of baseball, and fell off dramatically after the 1930s. The early years tended to be batting competitions (in some ways like today's) rather than pitching competitions, and a pitcher's repertoire was limited to about a half-dozen pitches, plus whatever grease, oil, jelly, file, sandpaper, thumbtack, or razor blade he could conceal.

Re:If its so likely, they why hasn't it happened? (1)

Opie812 (582663) | more than 6 years ago | (#22915952)

...plus whatever grease, oil, jelly, file, sandpaper, thumbtack, or razor blade he could conceal.

Don't forget snot*. The snot ball.

*Major League, circa 1989.

Re:If its so likely, they why hasn't it happened? (5, Interesting)

Anonymous Coward | more than 6 years ago | (#22916064)

The early years tended to be batting competitions (in some ways like today's) rather than pitching competitions
If by "early years", you mean 1920 and later, yeah.

Otherwise, buddy, you're way off base.

NL year-by-year stats. [baseball-reference.com]

Look at those ERAs pre-1920. Before 1920, the ERA on the NL never significantly exceeded 3.00. After 1920, it never dropped below 3.3 or so, with the exception of a 2.99 in 1968, after which MLB made changes to the rules, amongst them lowering the acceptable height of the pitcher's mound.

The time prior to 1920 was marked by pitchers such as Cy Young, Mordecai Brown, Walther Johnson, Ed Walsh, Christy Mathewson. You've probably heard of most of them.

Here are the single-season MLB ERA leaders. [baseball-reference.com] Outside of Bob Gibson in the aforementioned 1968, you have to go all the way to Greg Maddux in 1994 at #48 all time to find a season after 1920 on the list. Barely 10 of the 100 lowest single-season ERAs in MLB history occurred after 1920. And that's only because Pedro Martinez in 2000 and Ron Guidry in 1978 tied with 9 others for #100 on the list. So only 8 of the best single-season ERAs happened after 1920.

You need to research "dead ball era", and the response by baseball to "Black Sox". (Hint: just like the response to the 1994 strike, it involves the ball...)

The fact that you got a +5 out of such a demonstrably incorrect post is a major indictment of the baseball knowledge of the Slashdot faithful.

Re:If its so likely, they why hasn't it happened? (5, Insightful)

hedwards (940851) | more than 6 years ago | (#22915080)

The most likely reason is that statistics isn't the appropriate method by which to study this problem.

This sort of a study is really more about curiosity, it doesn't deal with things like changes to the way in which the game is played. For instance early on, and for quite a while later, it was common for a pitcher to pitch 9 innings every game, and in many cases to pitch both games out of a double header. Meaning more opportunity for errors and since batters get time to rest up, there's a bit of an edge under that style of play to the batter which doesn't exist today.

That also doesn't include the variety of pitching which players see today or the fact that a player might get to see 3 different pitchers in a single game.

Even the length of the season has an effect on how players play. None of those things are easily quantified, much less analyzed by statisticians.

Re:If its so likely, they why hasn't it happened? (1, Insightful)

Anonymous Coward | more than 6 years ago | (#22915262)

While it might be true that statistics is not an appropriate technique to study baseball, none of the things you mention are evidence against it being useful. All of the factors you mention influence a player's batting average, and the hypothesis they are using is that once you know the batting average you can calculate a set of possible histories of hits for that player, with the right statistical weight. They are assuming that the probability of a batter getting a hit in any game is uncorrelated with his performance in previous games.

The answer to the question in the subject line is "It has happened." They're not claiming that in any given year there is likely to be a 56 game or longer hitting streak. What they calculated was the probability of the longest streak in the entire history of baseball having a particular value, and found that the most likely longest streak is 51 games, about what is observed in the universe we live in.

Re:If its so likely, they why hasn't it happened? (2, Interesting)

hedwards (940851) | more than 6 years ago | (#22915370)

While it might be true that statistics is not an appropriate technique to study baseball, none of the things you mention are evidence against it being useful. All of the factors you mention influence a player's batting average, and the hypothesis they are using is that once you know the batting average you can calculate a set of possible histories of hits for that player, with the right statistical weight. They are assuming that the probability of a batter getting a hit in any game is uncorrelated with his performance in previous games.
Actually, they are reasons why statistical analysis is not appropriate in this instance.

Statistical analysis isn't inappropriate in terms of studying baseball, it is just inappropriate to use it in this manner.

What you are suggesting is a good example of the gambler's fallacy. And it breaks down in this case for the reasons that I mentioned, the underlying conditions in which those batting averages were collected has changed in such a way that they no longer accurately reflect the present conditions.

The GP was asking if the occurrence is that common, why hasn't it happened since, and the answer I gave was that there was a fundamental change in the way that the game is played which changed who has the advantage. It's similar to why nobody has had a .400+ season since 1930.

Re:If its so likely, they why hasn't it happened? (0)

Anonymous Coward | more than 6 years ago | (#22915478)

The result given in the summary doesn't address the question "What is the probability that you will have a hitting streak of a given length in any given year?" but rather "What is the probability of having a longest hitting streak of a given length in the entire history of baseball?" I'm not falling for any gambler's fallacy: the say quite clearly that they use the statistics for each year to construct the history of hits for each player in that year.

It is true that they find that it was more likely that the streak would come early in the history of baseball, possibly for the reasons you mention, but that doesn't mean that the quoted result is in any way an inappropriate use of statistics.

Maybe we're not disagreeing, if what you're saying to GGP is that the result doesn't mean what he thinks it means.

Ted Williams might be surprised... (1)

emaja (1264772) | more than 6 years ago | (#22915916)

(if he was still alive) to read that no one has hit .400+ since 1930, since he hit .406 is '41.

Baseball is one of those things that if your father didn't like, you probably won't either. It seems to be a passion that is passed down from generation to generation.

Today, it seems a bit archaic or even boring to some because of the slower pace, but some of us appreciate it for those very same qualities.

Re:If its so likely, they why hasn't it happened? (1)

Reziac (43301) | more than 6 years ago | (#22916112)

Having shamefully RTFA, ISTM that the study assumed that baseball players are rather more homogenous in their performance, and in who gets to see whom in the field, and how that affects individual players, than occurs in reality. I wonder what sort of numbers it would come up with if it accounted for how managers juggle players around?

Gene Mauch's strategy when he managed the Angels was so unvarying and consistent that he probably COULD be reduced to an algorithm.

Re:If its so likely, they why hasn't it happened? (5, Insightful)

ByteSlicer (735276) | more than 6 years ago | (#22915254)

Because baseball players aren't dice?

Re:If its so likely, they why hasn't it happened? (3, Insightful)

GoodbyeBlueSky1 (176887) | more than 6 years ago | (#22915558)

I wish my mod points hadn't just expired, because you just summed it up perfectly. Silly study with no basis in reality.

In other news, I've just started a fund of stocks that are held and traded based on historical data. If you invest in it, I guarantee a large return, because complex systems that rely heavily on myriad human variables are of course determined entirely by statistics.

Re:If its so likely, they why hasn't it happened? (1)

Colonel Korn (1258968) | more than 6 years ago | (#22915988)

The parent should be modded insightful, not just funny.

Re:If its so likely, they why hasn't it happened? (4, Funny)

Kamineko (851857) | more than 6 years ago | (#22916076)

Like Einstein said: "God does not play baseball!"

I think.

Re:If its so likely, they why hasn't it happened? (5, Informative)

Frequency Domain (601421) | more than 6 years ago | (#22915432)

No bashing, it's not a bad question. The answer is because it still qualifies as a "rare event". The thing that's kind of counter-intuitive, but easy to demonstrate, is that having a particular rare event happen is rare, but having some rare event happen is common.

A good illustration of this is the so-called "birthday paradox", which asks what's the probability of having duplicate birthdays in a group of n people (whose birthdays are independent of each other). Think of adding the people to the room one by one. The first person doesn't have any chance of having a duplicate birthday, because there's nobody else in the room. The second person has 1/365 chances of duplicating, 364/365 of missing the first one. Let's follow up on the misses, they're easier to work with. In general, if we've got k people in the room without a duplicate, that means they've used up k of the 365 days in the year, and the next person we introduce to the room has to miss all of those days to avoid a duplication. So the probability of everybody missing everybody else, by the time we get up to n people in the room, is (365/365)*(364/365)*(363/365)*...*((365-n+1)/365), which starts diving towards zero really fast. The probability of having one or more duplicates is 1 - P(no duplicates), which correspondingly climbs to one really fast. If you write a short program to do the exact calculations, you'll find that by the time you have 23 people in the room the probability is greater than 0.5 of having a duplicate, and by the time you get 57 people it's greater than 0.99!

If you pick one particular person and ask what's the probability of duplicating that birthday it remains quite small. That's the difference between having a particular rare event rather than having some rare event. For a large enough group, some pair of people will almost surely share a birthday but the odds of it being you (or any other designated person) remain quite small.

Just to preserve my computing geek cred, this is why you need collision resolution for hashing algorithms. You don't know which entries will share hash values, but collisions are almost certain to happen by the time you've loaded 3 * sqrt(Hash Table Capacity) values, e.g., if your hash table has capacity 10000 you will almost surely see a duplicate within the first 300 entries.

Re:If its so likely, they why hasn't it happened? (0)

Anonymous Coward | more than 6 years ago | (#22915986)

It's likely to occur once in the 100 odds year history of baseball, the probability for it occurring twice is correspondingly lower.

The statement isn't "it should be happening all the time" but "if not him someone else would have done it by now."

Nerves (4, Insightful)

digidave (259925) | more than 6 years ago | (#22915028)

This doesn't take into account that once a player achieves an impressive hit streak he gets more media attention, people start asking him about Dimaggio's record, and every time he steps up to the plate he's a bit more nervous about it than the last time, making it slightly less likely that he'll get a hit.

Re:Nerves (2, Interesting)

p0tat03 (985078) | more than 6 years ago | (#22915096)

That kind of error can be accounted for by tracking their batting averages over time. If we have a model for batting average deterioration due to stress, then the simulation will still work as a good approximation.

Re:Nerves (1)

EdIII (1114411) | more than 6 years ago | (#22915102)

That's like saying that a famous player would be more likely to be popular with the women and get the "clap" thereby depriving him of a few games.

In any case, what you are talking about would affect all players equally, therefore it would cancel itself out in their research.

Re:Nerves (3, Interesting)

Kjella (173770) | more than 6 years ago | (#22915390)

In any case, what you are talking about would affect all players equally, therefore it would cancel itself out in their research.
Not when they use it the way they use it, and say streaks of 39 to 109 is to be expected. If the difficulty increases by the length of the streak, 56 could be a far more exceptional streak than their research indicates.

Re:Nerves (1)

The One and Only (691315) | more than 6 years ago | (#22916024)

You know, for a lot of successful athletes stress either doesn't affect them much or it actually works the opposite way: it makes them more successful. These people are who we call "clutch performers". Of course, a lot of talented but non-clutch performers would be eliminated were this compensated for, but a lot of clutch performers would do better.

Re:Nerves (1)

sykodoc (763810) | more than 6 years ago | (#22916216)

So all the players after Demaggio are displaying the Heisenberg principal of sports players!! They are being observed by the public, with feedback of that observation being provided by the media, and so the outcome becomes unknowable.

awesome.

Deistic proof? (0)

Anonymous Coward | more than 6 years ago | (#22915034)

I think this is basically a form of statistical proof that there is, indeed, a God.

Re:Deistic proof? (3, Funny)

Loadmaster (720754) | more than 6 years ago | (#22915060)

What would God need with an impressive hitting streak? Besides, Jesus can't hit a curve ball so he'd never make it anyway.

Re:Deistic proof? (2, Funny)

TheRealMindChild (743925) | more than 6 years ago | (#22915196)

Apparently, Joeboo can't either :(

Re:Deistic proof? (1)

spisska (796395) | more than 6 years ago | (#22915240)

Besides, Jesus can't hit a curve ball so he'd never make it anyway.

The curve he's alright with, and can manage a bit of opposite field power. It's the backdoor slider that gives him trouble. And the knuckleball.

Those pitches are a bit too deceitful. I heard that after 40 days of batting practice he was offered some kind of deal that would let him see the pitches before they were thrown but he turned it down. Oh well. I guess that's why he's still playing in Iowa.

Or is it proof that... (1)

clonan (64380) | more than 6 years ago | (#22916262)

Giving grad students unlimited computational power does NOT provide quality science.

How to Make Baseball Even MORE Boring? (3, Funny)

morari (1080535) | more than 6 years ago | (#22915054)

Talk about the statistics of anyone at bat..

Re:How to Make Baseball Even MORE Boring? (5, Funny)

pchan- (118053) | more than 6 years ago | (#22915168)

You don't understand. Baseball is so boring, the fans find the statistics exciting!

Re:How to Make Baseball Even MORE Boring? (2, Funny)

jd (1658) | more than 6 years ago | (#22915538)

Do you have the statistics to prove that?

just slow down (1)

plopez (54068) | more than 6 years ago | (#22916000)

You just have to slow down. Put away the x-box and the crackberry. It has it's own pace and ebb and flow. drink a few cold ones, fire up the grill. A nice saturdy or sunday relaxing watching a game is never wasted. Or go out to a park and see a real live game. Get out of your parent's basement and get some sun. Even better, call in to work for a mental health day and go to a park. If you don't have access to a mjor league park a minor league game can be fun too.

In an ADD culture, a nice relaxing baseball game can be great.

Re:How to Make Baseball Even MORE Boring? (5, Funny)

garett_spencley (193892) | more than 6 years ago | (#22915200)

I was once at a friend's BBQ and a lot of the other guests were really into sports and talking a lot about their various sporting events etc. I made a comment about how baseball was one of those sports that is fun to play but boring as hell to watch. One of the guys responded with, simply, "I disagree". To which I replied "You're right. It's pretty boring to play too." He wasn't very amused.

Talk about a great way to make an awkward social event even more awkward :(

Re:How to Make Baseball Even MORE Boring? (1)

davolfman (1245316) | more than 6 years ago | (#22915396)

It's decent to watch provided you are in the stands, reasonably close to the field. Otherwise you're just looking through a straw.

Re:How to Make Baseball Even MORE Boring? (1)

rob1980 (941751) | more than 6 years ago | (#22915430)

It's definitely one of those sports that is boring to watch on TV, at the very least. Going to a college or minor league game is a good way to spend an afternoon with the kids or some friends, as long as you eat beforehand so you don't talk yourself into paying $4 for a hot dog, $6.50 for a bag of peanuts, etc etc etc. I've heard hockey's the same way, but where I live it's pretty tough to see a live hockey game let alone one on TV.

Re:How to Make Baseball Even MORE Boring? (1)

zach_d (782013) | more than 6 years ago | (#22916122)

... maybe it's my nationality talking, but I fail to see how hockey's boring on TV.

now, if I was defending curling, I know it's my nationality, but hockey's got some pace to it.

Cricket (1)

flyingfsck (986395) | more than 6 years ago | (#22915574)

Actually, baseball is very exciting compared to cricket.

Re:Cricket (1)

Alpha Whisky (1264174) | more than 6 years ago | (#22915898)

Actually, the English invented baseball, but we gave it up to play the vastly superior game "Rounders". Rounders is now only played by schoolchildren and even then only when they don't have enough space in their timetable for the infinitely superior game of Cricket. We consider it quaint that you colonials still play that obsolete game. It's nice that someone preserves the old ways.

Re:Cricket (-1, Troll)

The One and Only (691315) | more than 6 years ago | (#22916134)

That's interesting, we did the same thing. We took your "football" and called it "soccer". These days, soccer is only played by schoolchildren, and even then only when they don't have enough equipment for the infinitely superior game of American football. It amuses us that you folks riot and kill each other over a childrens' game.

How to make STATISTICS even more boring? (1)

sayfawa (1099071) | more than 6 years ago | (#22915214)

Involve baseball. There, fixed.

Also, go Jays.

Of course it's not that simple (0)

Anonymous Coward | more than 6 years ago | (#22915078)

A batter with, say, a .300 batting average does not have a 30% chance of getting a hit each time he's up to bat. There are inevitably going to be days when you face someone like Randy Johnson or Roger Clemens or (cross fingers for 2008) Felix Hernandez. Couple that with some fantastic defenses and it's not surprising that a good batter goes a day without a hit.

So players have some days with a smaller chance of getting a hit than others (say, when the 2007 Mariners were running Horacio Ramirez out there with our god-awful defense). There's a reason advanced baseball statistics are more complex than what these guys did.

Re:Of course it's not that simple (1)

glitch23 (557124) | more than 6 years ago | (#22916208)

A batter with, say, a .300 batting average does not have a 30% chance of getting a hit each time he's up to bat.

There is a legitimate reason for that. A batting average is not a prediction for the future but a record of the past. Of course, the same applies to all other statistics in any subject for that matter.

typo (0)

Anonymous Coward | more than 6 years ago | (#22915086)

i found a typo streak!!!

"very alternate universe produced a steak f 39 games or better"

2 typos in a row!!!

Re:typo (1)

Chris Burkhardt (613953) | more than 6 years ago | (#22915956)

Pfft. That's nothing. I onc mitsyped, lik, fivve wrds in a row.

i came here to make an insightful comment (4, Funny)

circletimessquare (444983) | more than 6 years ago | (#22915118)

unfortunately, not many of my comments are insightful, so with my batting average, you will have to refer to a parallel universe

there you will find that this comment contains something worthwhile reading. sorry

Changing game of baseball (4, Interesting)

kingmundi (54911) | more than 6 years ago | (#22915144)

One of the key points mentioned in this article is when does the hitting game streak occur? They mention that it was much more likely to occur during the early 1900's which is known as the deadball era. The baseball wasn't as springy and they tended to use the same ball during the entire game. During that time it was more efficient to try and knock the ball between the holes in the fielders and get a double or single then to try and hit it out of the park.

I think it would be more impressive to take a subset of the data, and compare from 1930 up until the present. Of course, there have been other major changes to; glove sizes, introduction of the slider for a pitch, steroid use.

too simplistic (5, Insightful)

ndenissen (1201635) | more than 6 years ago | (#22915154)

From reading the article (which is light on the details) it seems like they used nothing but batting average, at bats, and games played.

The problem is this doesn't control for variances in the quality of pitching. The chances of going that many games without running into a hot pitcher isn't accounted for.

Imagine you average a 75% chance of getting a hit in any individual game. If you face three average pitchers, your chances are (.75)^3 but if you face a good pitcher an average pitcher and a bad pitcher it might be (.5)(.75)(1.0) which gives a different probability, despite the same average number of hits.

In order to be realistic the calculation would need to account for the deviation from average in the ability of the pitchers (which would likely be higher 100 years ago because of fewer player and segregation, and now because of expansion, as compared to the 1950s)

What they don't report is how often there are long (but not record) streaks in their model, so there is no way of knowing how accurately it reproduces reality.

Something like the modding on /. (0)

Anonymous Coward | more than 6 years ago | (#22915728)

The parent and the post immediately after it make the same point. The parent is modded 1 and the one after is modded 5.

The point is that the statisticians didn't accurately model the chance of a hitless game.

Re:too simplistic (2, Interesting)

DannyO152 (544940) | more than 6 years ago | (#22915982)

On the other hand, one doesn't get the benefit of running into the belly-itchers. My feeling is that, on average, the superstars, the ones with above 340 career averages, generally feasted on the mediocre to minor pitchers.

What this study doesn't take into account is how long it takes to live through a streak. DiMaggio needed two months. Besides the strain of day to day playing (and if it's a pennant race, you know the hot hitter is going to be in the lineup) there's also the way the weather and the light changes during the season. There used to be more day games and double-headers back in the 30s-40s-50s when batting averages were highest. Travel was by train and by bus and took longer. There seems to be a week every season when a cold or flu is making the rounds of the club. Then there's situational issues. 7th inning and behind, man on second base, the hitter is 0-3 and 30 games into the streak. I say the pitcher semi-intentionally walks the batter and amid a chorus of boos the streak goes poof. Here's another consideration, the opposing players and pitchers know the hitter has a streak when it gets past 20 games and the pitching gets a bit more careful and the batter has to extend the streak via pitchers' mistakes, and that makes it less likely.

if what I say is true, it should follow that the incidence of any consecutive games with a hit streak beyond 15 in a MLB season should be lower than the probability suggested by the league batting averages (which are depressed in the NL by pitchers and the other bottom 4 from the lineup.)

Too many assumptions? (4, Informative)

kevinatilusa (620125) | more than 6 years ago | (#22915156)

From the descriptions I've seen of their research, it seems that they're treating all games identically for the purpose of determining a typical season's behavior. While this may me necessary to make the computation tractable, it's not realistic, and introduces a sizable bias towards long hitting streaks.

In reality, a league is typically very imbalanced from team to team and from pitcher to pitcher (probably even more so in the game of the early 20th century than now). It's easier to get hits off of two successive average pitchers than it is to get hits both off of a very good and a very bad pitcher. For example (to oversimplify a good deal):

Say the league is split 50/50 between "good" pitchers (pitchers you'll get a hit off of 50% of games) and "bad" pitchers (pitchers you'll get a hit off of 80% of games). In a typical 20 game stretch, you'll encounter 10 good pitchers and 10 bad ones, and your odds of getting a hit in all 20 games would be (0.50)^10(0.80)^10, about 1/9537.

Under their analyis as I understand it, they'd replace all the pitchers by mediocre pitchers who you'd get a hit off of 65% of the time, and your odds would be (0.65)^20, about 1/5517.

This one assumption almost doubled your chances of getting a hit in all 20 games.

There are other biases as well going the other way (ignoring the effect of hitting slumps, for example), but this one jumped out at me.

Re:Too many assumptions? (1)

mortonda (5175) | more than 6 years ago | (#22915434)

Wait. You're saying that you can make statistic lie? Say it ain't so!
</sarcasm>

why the assumptions (0)

Anonymous Coward | more than 6 years ago | (#22916148)

In reality, a league is typically very imbalanced from team to team and from pitcher to pitcher (probably even more so in the game of the early 20th century than now).
The reason the imbalance decreases with time is the rise of competition. In his book "Full House", Stephen J. Gould detailed this effect for both baseball (in particular, but he noted sports in general) and in the rise of and success of complex life forms. The effect of a random walk from zero athletic achievement (or size or complexity or etc.) is limited still by the wall of maximum human ability (or etc.). You're bound to find records being advanced by large amounts and often at first, trailing off to smaller advances less frequently as time goes on. The book cites the example of the present slow growth of men's sporting achievements compared to the relatively faster growth of women's sporting achievements in the same athletic endeavors where the difference is simply the length of time of women's participation and records having been kept.

Among all batters and at-bats in the major leagues, the batting average hovers right around .260 overall and for every sub-interval, with (IIRC) a handful of moderate single-season excursions. Note that that spans first 60 years, last 60, deadball, steroid era, everything.

It describes "the death of the .400 hitter". The best hitters aren't getting any better because there's only so well a human can perform given the laws of physical reality. However, the worst hitters are getting better because of the "efficient market" effect: there's no room in the league for lousy hitters because more high-quality hitters are available from larger pools (from human population in an absolute sense and from the popularity of a sport in particular). Same with pitchers. That's why the best hitters' batting average falls with time and why the best pitchers' opposing batting average goes up with time. It's also why the worst hitters' and pitchers' numbers improve with time; all the numbers converge to steady-state optimality as quality improves in an efficient market.

A top-quality hitter in an early era would have many more mediocre pitchers to exploit than in a later era. Similarly, a top-quality pitcher in an early era would have many more mediocre hitters to exploit than in a later one.

It's easier to get hits off of two successive average pitchers than it is to get hits both off of a very good and a very bad pitcher. For example...
[pertinent example] ...
True enough (and your numbers sound reasonable; I'm not certain what the likelihood of an n-game hitting streak is...), but that's not a big enough difference to surmount the sheer number of chances at hitting streaks throughout the history of baseball. Take even your most pessimistic example (or one much more pessimistic, even), and sum it over the number of possible 56-game-batter-hitting-streaks over the last 100 years of games played and the result will be bigger than unity. In fact (I think this is the case...), if each individual player in the history of baseball, no matter how great or terrible, had exactly one shot at a 56-game hitting streak, given the aforementioned .260 overall average, you would expect at least one of them to get it and probably more, and you would expect the largest streak to be somewhat larger still.

Your complaint about the inhomogeneity's affecting the outcome of the calculation would be more valid if the distribution of player abilities, match-ups, and the like were not uniform; but in fact they are. They do indeed make that assumption as you note, but it is a justified assumption.

What baffles me is why they are running "simulations" on this sort of thing rather than dealing with closed-form probability. It's cool to get the computers to do simulate baseball, and there's a market for that entertainment value, but you don't need to plink the pseudo-random number generator millions of times to find your way around the problem.

Bogus (1, Insightful)

DoofusOfDeath (636671) | more than 6 years ago | (#22915166)

Shouldn't we say that the probability of it happening was 1.0, because it did happen?

It seems to me that if their experiments report anything else, then either their models are erroneously inaccurate, or they got something else wrong.

Re:Bogus (3, Interesting)

Miseph (979059) | more than 6 years ago | (#22915270)

No, because the probability for ANYTHING, given enough chances, is 1.

What they are actually saying is that reality appears to follow a probability bell curve.

You could also say that, in 1,230,000 years of baseball games, we could be almost certain of a hitting streak longer than 56 games.

Re:Bogus (0)

Anonymous Coward | more than 6 years ago | (#22915790)

"No, because the probability for ANYTHING, given enough chances, is 1."

ehhh... no.

I'm pretty sure that is not the case for rolling a 13 with a pair of standard dice or drawing 5-of-a-kind from a standard deck.

Re:Bogus (1)

Reziac (43301) | more than 6 years ago | (#22916304)

Hell, in 1,230,000 years of baseball games, *I* could get a 56 game hitting streak. ;)

True story: I am the world's worst volleyball server. I'm lucky to whack the ball in the correct direction, let alone get it over the net. One day in the 9th grade, I scored 14 spikes in a row. Everyone (including the gym teacher) is staring at me like "WTF?! What have you done with the real Rez??" Needless to say it was a one-time freak event. :?~

Re:Bogus (1)

Anonymous Coward | more than 6 years ago | (#22915360)

Your lack of understanding of statistics is horrifying. I just flipped a coin ten times. The most likely outcome is 5 heads, and the fact that I got 7 does not show that the usual model is in any way inaccurate.

Re:Bogus (1)

DoofusOfDeath (636671) | more than 6 years ago | (#22916080)

I just flipped a coin ten times. The most likely outcome is 5 heads, and the fact that I got 7 does not show that the usual model is in any way inaccurate.

I disagree on two counts. First, when you say the likely outcome is 5 heads, your asserting that the mean of the PDF for the coin is 0.5. I.e., your model includes a mean of 0.5 as a detail. Because you're taking that as an established fact, no matter how many heads you got in a row could change your mind about the PDF's true mean - you've already taken a mean of 0.5 as a given.

But I was making a very different point in my OP. The particular baseball stat is a "state of nature". It happened. If we assume that the universe is deterministic, then it's nonsensical to assign a probability other than 1.0 to that event. Any model that produces an answer other than 1.0 for that event is probably replacing a variable that in the real world has a fixed value, with a random variable. Thus, the researchers had to introduce an inaccuracy in their model in order to make anything about their outcome be uncertain. It's more of a modeling issue than a probability issue.

Clearly... (1)

Corpuscavernosa (996139) | more than 6 years ago | (#22915180)

... they didn't take into account my 162 game hitting streak in "The Bigs" on PS3. With settings on easy.

Isn't this a monkey typing thing? (1)

PC and Sony Fanboy (1248258) | more than 6 years ago | (#22915208)

Isn't this the same thing as saying that an large number of monkeys typing for a large period of time is more likely to properly re-create the complete works of tolkien ... by accident?

Re:Isn't this a monkey typing thing? (1)

webmaster404 (1148909) | more than 6 years ago | (#22915330)

No, It would be like taking monkeys, giving them RPG style intelligence points, and running them via an RNG and see which one can write the book to stay on the best seller list for the longest time.

Re:Isn't this a monkey typing thing? (0)

Anonymous Coward | more than 6 years ago | (#22915512)

how many times could I reroll my stats? And what race of monkey?

You can't do statistics with a random # generator (1, Informative)

rufusdufus (450462) | more than 6 years ago | (#22915248)

Our simulations did something very much like this, except instead of a coin, we used random numbers generated by a computer.
It is not mathematically sound to do statistics with a random number generator. Computers do not actually generate random numbers, but instead, they can only make pseudo-random numbers that have a certain distribution.
Any 'simulation' done in this way will always have a bias.
In order to get correct statistics, you must actually compute the statistics.

Re:You can't do statistics with a random # generat (5, Informative)

kevinatilusa (620125) | more than 6 years ago | (#22915326)

It is not mathematically sound to do statistics with a random number generator. Computers do not actually generate random numbers, but instead, they can only make pseudo-random numbers that have a certain distribution. Any 'simulation' done in this way will always have a bias. In order to get correct statistics, you must actually compute the statistics.
Sure, the proper way to put it mathematically would have been "we did a Monte-Carlo based simulation of the probability distribution of the longest hitting streak under our model due to the intractability of direct computation", but this is an editorial in the New York Times, not a mathematical journal! As a side note, just because a computation is performed on a set of pseudorandom numbers does not mean it is biased...usually the whole point of pseudorandomness is that the discrepancy between computations involving them and identical computations involving true random numbers will typically be quite small.

Re:You can't do statistics with a random # generat (1, Informative)

Anonymous Coward | more than 6 years ago | (#22915380)

I think it's safe to say that most all statistics uses a random number generator. Computers *do* have the capability to produce true random numbers, as shown with /dev/random [wikipedia.org] , which relies on an entropy pool and is suitable for cryptographic key generation.

Re:You can't do statistics with a random # generat (1, Informative)

Anonymous Coward | more than 6 years ago | (#22915470)

No, it won't always have a "bias". Bias is a technical term here, and infact there is not likely to be bias. Bias is where the long-run average value of simulated variables is not equal to the actual average value of the thing you're simulating. For example, rolling a chipped die to draw numbers uniformly from 1 to 6 will probably cause this.

The problem with pseudorandom number generation tends to be dependence between samples (barring a more serious bug, which has happened... but this is always a problem, and there can also be bugs in the rest of the code anyway). Now this correlation is a problem for cryptography maybe, since there is intense interest in every bit of entropy in a very short signal, and a lot of clever guys hacking at it.

However in statistics, you basically just use the random numbers as "fuel" for a sequence of very stupid computations (more or less, glorified averages and averages of squares, &c.). The functions used in statistics are just too stupid to find out that the numbers have inter-dependence, so that they tend to give the same results for pseudo-random numbers as for real numbers. This is thanks to a lot of hard work from many fields, to improve pseudorandom number generators.

In fact, and as a tangent, theoretical computer scientists tend to believe that any randomness in an algorithm can be replaced by deterministic functions! (although they don't believe this as widely as they believe P!=NP). Since we can consider any statistical procedure an algorithm, the effect (at least philosophically) that this would have on many applied fields is mind-boggling. I would love to a proof and some general techniques for this "derandomization" - if there were one, we could finally absolve ourselves of our state of sin [wikiquote.org] . (It would also imho inform the "free will" debate a bit.)

Re:You can't do statistics with a random # generat (1)

UnknownSoldier (67820) | more than 6 years ago | (#22915858)

There is no such thing as true randomness. You can't measure something without effecting it.

Consciousness can effect randomness, as this Princeton page [princeton.edu] proves.

Re:You can't do statistics with a random # generat (4, Informative)

Vellmont (569020) | more than 6 years ago | (#22915958)


Computers do not actually generate random numbers

That'll be a surprise to the multiple true random number generators build into most operating systems. There's many sources of random data in a computer. Timing between keystrokes, timing of mouse movements, network latency between packets, and of course hardware random number generators that use thermal noise as its source.

So to put it mildly, computers can, and DO generate truly random numbers that are completely unpredictable and free from bias.

(Oh, BTW, to do a Monte-Carlo simulation (which the referenced article is) you actually don't need true random numbers, you only need a pseudo-random source that's free from bias. Those pseudo-random sources do exist, and aren't that even that difficult to code.)

Re:You can't do statistics with a random # generat (1)

ucblockhead (63650) | more than 6 years ago | (#22916364)

None of those things are truly random. Unless you are dealing with quantum effects, you are not dealing with something truly random.

In particular, timing between keystrokes is not at all random. In fact, one can use the timings between keypresses to figure out who the typist is!

Re:You can't do statistics with a random # generat (0)

Anonymous Coward | more than 6 years ago | (#22916250)

Bitches don't know 'bout my Blum Blum Shub.

news? (1)

superwiz (655733) | more than 6 years ago | (#22915260)

maybe. for nerds? i doubt it. "the math guys"... ok, definitely not news for nerds -- too dismissive of the experts

Re:news? (1)

sgbett (739519) | more than 6 years ago | (#22915798)

more importantly (re: your sig) who would ever need a slashdot post number over 2,147,483,647!

So basically... (2, Insightful)

davidbrit2 (775091) | more than 6 years ago | (#22915300)

They took a bunch of measured statistics, ran a simulation with outcomes biased using said statistics, and then acted surprised when the simulation results ended up pretty close to what actually happened?

Re:So basically... (2, Interesting)

kevinatilusa (620125) | more than 6 years ago | (#22915364)

They took a bunch of measured statistics, ran a simulation with outcomes biased using said statistics, and then acted surprised when the simulation results ended up pretty close to what actually happened?
I think their point was that they took a set of numbers that were generally considered unremarkable (the overall statistical distribution of batting totals from the last 100+ years) and tried to show that a number that most people considered very unusual (the 56 game streak) was in fact also typical given this other, "unremarkable" set of data.

Anohter unreported weird fact (5, Funny)

mrfantasy (63690) | more than 6 years ago | (#22915324)

In every simulation, a ground ball went between Bill Buckner's legs in the 1986 World Series.

What about slumps? (2, Insightful)

Squirmy McPhee (856939) | more than 6 years ago | (#22915438)

By assuming the hitter's probability of getting a hit is equal to his season average the researchers don't take into account that most, if not all, batters have a higher batting average at some points in the season than they do in others. As one with experience in Monte Carlo simulations I know that taking that into account would complicate the analysis considerably, but I suspect their results would be a bit different if they even did something as simple as using a 10-game moving average of the batter's average.

Re:What about slumps? (1)

jd (1658) | more than 6 years ago | (#22915566)

Yes, you'd need the variance and not just the mean, and you'd need a suitable distribution, which will probably not be symmetrical and certainly not a single spike.

Re:What about slumps? (1)

Squirmy McPhee (856939) | more than 6 years ago | (#22915788)

Yes, you'd need the variance and not just the mean, and you'd need a suitable distribution

That would be another way to do it, sure. You could also use a Markov chain with states corresponding to slumping/not slumping (or something like that). Bootstrapping might not be a bad way to go, either. It all depends on how much computational effort you want to expend and how well suited these various methods are to the actual statistics (that's a project unto itself). Even then, the decisions a batter and a team's manager make in the real world depend very much on the in-game situation and whether or not a player is slumping. If Joe DiMaggio isn't going into game 45 of his hitting streak, for example, then maybe on his last at-bat of the game the manager has him sacrifice bunt instead of trying to get a hit.

Because of that, I can understand why the study's authors took the approach they did -- from a statistical standpoint, you can argue with the method, but more rigorous methods won't necessarily be reflective of the real world (or a "real" parallel universe). That's why I suggested using something like a 10-game moving average: I think it strikes a balance between rigor and real-world insight, particularly since this isn't exactly an earth-shatteringly important topic. That's just armchair statistics, though.

streaks and slumps (0)

Anonymous Coward | more than 6 years ago | (#22916286)

Your assertion of slump effects sounds similar to "the myth of the hot hand" in analysis of basketball statistics. When you examine streaks, you find that the probability of making a shot following one successful shot is not any higher (or lower!) than the probability of making one after an unsuccessful shot. Same for "cold" hands. The distribution of shooting streaks really is, upon careful analysis, what you would expect based on examining only the mean. You don't gain anything by accounting for streaks, whether short or long or of highly-skilled or mediocre shooters.

I know the above analysis has been done for basketball, and I suspect there exists but have not seen a similar analysis of baseball hitting streaks. I suspect that using, say, a 10-game moving average as you suggest (or similar streak/slump accounting tactic) would be as useless in baseball as it is in basketball. (I'm much more familiar with baseball than basketball, both stats and as a sport).

So intuitively, you'd expect slumps and streaks to influence subsequent attempts. There's even a good body of argument from psychology, climate, environment, physiology (fatigue) etc. that there might. Actually examining the statistics, however, reveals that it just ain't so. and I think that is an earth-shatteringly important point; it's not about something trivial like sports, but about the power of statistical insight we can gain from physical reality. (Stephen J. Gould himself said he wouldn't begrudge anyone for thinking that baseball was as "earth-shatteringly important" a topic as anything else...)

Now that I have waxed poetic, I see that my captcha is "predict".

Does Joe DiMaggio's Streak Deserve an Asterisk? (5, Informative)

harryjohnston (1118069) | more than 6 years ago | (#22915458)

This seems relevant:

http://abcnews.go.com/Technology/WhosCounting/story?id=3694104&page=1 [go.com]

Disclaimer: I'm not an American, so I know next to nothing about baseball - and care less!

Re:Does Joe DiMaggio's Streak Deserve an Asterisk? (1)

Vellmont (569020) | more than 6 years ago | (#22915912)

The thing I find strange is the idea of
The record book.

As if there's some big "official" book that's published as
"The Record Book Of Baseball", and MLB officials all sit around arguing about asterisks. I'm no baseball fan, but I never thought there was "The" record book. Isn't there just a series of "A" record books?

I've never been a big sports fan, but what drives me crazy, especially about a game like this where you win or lose, is how people get all hung up about this record or that record. What does it matter? The players play the game, the people watch the game, they get caught up the in the competition, the game is over. Then people go home and start whipping out their pencil and calculator. That just seems kind of odd to me. The records just seem so created and artificial.

On the point of baseball (1)

earlymon (1116185) | more than 6 years ago | (#22915502)

A lot of people think baseball is boring - today it is, but take it from a geezer, not always so.

I blame television. I can no longer watch a ball game on TV. Might as well be Entertainment tonight. They used to have a camera behind the backstop so you could see the pitch, the swing (from behind) and the infield. Another camera to go to the outfield, and maybe one for the infield. They game has strategy. It has finesse. It even has - to use a term no longer apparently known in the software world - elegance.

Now, it's unshaven bums (I'm an unshaven bum today, so OK) close-up, on the mound, with bad hair cuts curling from under the caps, with their follicles in high def while they spit and scratch. High res, high def and no sense of a team at work - if there still are any!

Baseball is a noble game of strategy, ruined by Madison Avenue's need to sell multi-hundred dollar sneakers to our kids.

Want to check out a good ball game? Here you go - http://www.startrek.com/startrek/view/series/DS9/episode/103565.html [startrek.com]

So, if baseball's good enough for Klingons and Vulcans and Ferengi, it's good enough for /. - and this covers the alternate universe part as well.

Now - get off my lawn!

Re:On the point of baseball (1)

Reziac (43301) | more than 6 years ago | (#22916238)

Bah, Vulcans could never play baseball; they'd be too caught up in the stats! It'd be exactly like Gene Mauch and the Angels -- everything done right per the stats book, but never, ever would they try anything that was out of spec. It's not the stats that win games; it's the quarter of an inch you reach beyond what your stats say you can. (I love little ball, but Mauch made me crazy.)

I haven't been where I could get sports TV reception (or even radio reception) in 11 years. Before that, I worked my business hours around Angels games. :) But the trend was already there -- too much dwelling on details that don't really tell us anything. I want to see the pitcher's presence on the mound, not the zits behind his ear. I want to see the batter's body language, not his chaw-drip. I want to see the runner's balance between the bases, not his crappy haircut. Baseball may be a game of inches and microcosms, but it's first and foremost a game of *balance*, and that means the whole team, seen as a team.

I think what happened was that because we COULD do extreme zoom and the like, we DID, whether or not we SHOULD. Football is going that way too -- sometimes after a play that was zoomed in too tight, I find myself thinking "Wha'happened?" because the whole view was too lacking.

True story: for a while I had such bad TV reception that all I could see was shadows. I knew every pitcher by his motion, every batter by his stance -- and none of their faces, cuz I'd never seen 'em. My sister comes in and wants to know why I'm intently watching static. "See here," I tell her with great enthusaism, "this is X on the mound, and Y at bat, and watch how that runner is leaning toward second, etc, etc." She peers closely at the TV, and finally says "I don't see *anything*!!"

In an alternative baseball universe... (1)

creimer (824291) | more than 6 years ago | (#22915530)

Tickets, hot dogs and beer would be a lot more affordable.

Lies, damned lies, and statistics (-1, Flamebait)

rockhome (97505) | more than 6 years ago | (#22915554)

"In other words, streaks of 56 games or longer are not at all an unusual occurrence. "

um....um.....um.....

In the odd little universe where statistics really have meaning, the result of the SIMULATIONS showed a likelihood of these streaks.
Unfortunately, this research neglects the fact that real life is drastically different from purely statistical "universes". the real fact is that it happened once and never again.

Joe DiMagio, or any batter, isn't just the result of statistics, he is the result of skill, practice, and circumstance. I always like it when mathy people try and put some statistical "context" around what happens in sport, they always forget that no matter how valid their statistics, real life often tells a different story. Remember, any event ultimately has a 50-50 chance. It either happens or it doesn't. The particular outcome is rarely significant, it is a binary proposition.

Re:Lies, damned lies, and statistics (0)

Anonymous Coward | more than 6 years ago | (#22915762)

Remember, any event ultimately has a 50-50 chance. It either happens or it doesn't. The particular outcome is rarely significant, it is a binary proposition.

Tell that to the casinos.

Re:Lies, damned lies, and statistics (1)

demeteloaf (865003) | more than 6 years ago | (#22915776)

See, I think the quote you quoted really doesn't sum up the article at all, or at least, it's very misleading. The article doesn't say that streaks of 56 games are a common occurrence. What it says is that the fact that baseball history has a record of a 56 game hitting streak isn't that rare, given players historical batting averages.

Think of it this way. I flip a coin 10 million times and find the longest streak of heads. Say it's 50 (I have no idea how likely that is and i'm too lazy to actually simulate it, so i'm just going to make up numbers). I flip another 10 million coins, and find that the longest streak is 53. I flip another 10 million coins and find the longest streak is 40. I do this another 10,000 times and record the record in each case, and get a distribution for the longest streak in any given 10 million flips. Now, i'm not going to claim that 50 heads in a row isn't an extremely rare streak. But i can say what the odds are of getting a streak of at least 50 heads when i flip 10 million coins are.

That's what these researchers did. Simulated 10,000 seasons from 1871-2005 (using players historical batting averages as their likelihood to get a hit) and found what the longest hitting streak in each 'alternate history' is. Their claim is that 56 games is not an unusually high number (in fact It looks to be right about the median.) [nytimes.com] So sure, DiMaggio's streak was great, and incredibly rare. But it's interesting to see what the odds are of having a hitting streak that long.

it's basically a sure thing. (2, Funny)

iminplaya (723125) | more than 6 years ago | (#22915730)

Yeah, and so is the Cubs winning the World Series more than once in a hundred years

I've switched domains... (2, Funny)

xactuary (746078) | more than 6 years ago | (#22915784)

I've switched domains to operating systems and can now say that it took 42 googleplex simulations before I found a parallel universe where Vista doesn't suck. As you would expect, that's also the only parallel universe that had Steve Jobs throwing chairs.

Re:I've switched domains... (1)

Rod Beauvex (832040) | more than 6 years ago | (#22915854)

As you would expect, that's also the only parallel universe that had Steve Jobs throwing chairs.

I had to read this four time for my brain to process it correctly.

Feedback... (1)

argent (18001) | more than 6 years ago | (#22915890)

There's no feedback here.

Don't forget that the makeup of teams, the behavior of other players, and even the rules of baseball all depend on what happens in the game. If someone was setting a 109-game hitting streak in the 1890s, then they would be facing more determined pitchers and probably better pitchers by the time they were more than 20 or 30 games into the run. It seems pretty good odds that would have changed their batting average for that year. :)

How are real hitting streaks distributed in time? Do they bunch up in the 1800s and early 1900s the way their simulations did?

Were there changes in the rules between the 1890s and the 1940s that might have reduced the effect of this kind of feedback mechanism?
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...