And what does any of this have to do with space exploration?
2004, a large, musty conference room at Kennedy Space Center, today holding only the six or so of us to mull over a question trickled down from on high. What is the *probable* year the Shuttle will complete another 22 launches? I’d been invited, having taken an interest in the topic of these probabilities and occasionally contributing some value. From my corner, the question is pretty cut and dry. The calculations leave little room for debate. But that’s not to say today there wouldn’t be a lively discussion. The launches in question were those to come once we resumed operations after the loss of Columbia. The mission called for completing the construction of the International Space Station, with the Shuttle program told to call it a day when done.
Launching at a certain pace is only one of many probabilities calculated by NASA’s brain trusts. Much more analyzed, parsed, and debated was the probability of rockets failing. Even when failure is not an option, we know the possibility hangs in the air. Soon we will see this in play, the dice tossed for the NASA SLS, the SpaceX Starship, the ULA Vulcan, and the Blue Origin New Glenn rockets, with more to follow. So, what do probabilities tell us to expect in the first launches of these rockets – more or less, and with what certainty? More importantly, what do the numbers mean?
-o-
Star Trek’s Spock dead-panned the probabilities for any plan; the numbers were never good. Kirk either found ways around the odds or ignored them. Everything I needed to know about probabilities and what to do with them I did *not* learn from Star Trek. Nonetheless, I probably picked up a fascination with forecasting the odds from Spock. As our meeting about the Shuttle droned on, it too devolved into a tense Spock/Kirk dialog. I could tune out the unkempt room and poor lighting in a Walter Mitty moment, imagining the participants in Starfleet uniforms. Spock – “the numbers are clear.” The Shuttle would likely do what it always did, launching about 5 times a year. Without flinching, pull that band-aid. Here is your answer. Always added was “give or take,” all the jazz about if this, then that. Kirk responded impatiently. We must encourage the crew, err, workforce, to do better. We must “drive” the schedule, but the word “push” was implied. How to do this went unsaid.
Before entering the room, I figured the hour would be a curious break from my other duties in my day. Fascinating. Numbers can be stimulating, and I knew most people in the room for many years. Here came a chance to catch up and refresh on just where the numbers stood of late. Take it in. “Let the numbers wash over my mind,” I thought. Sit back. My pre-disposition said listen to Spock and take his numbers as gospel. Even his guesses were better than most people’s analyses. Not everyone jumped on board. From an expected Zen, the meeting went and zagged instead.
There is a phenomenon – Gambler’s Fallacy – whereby a person thinks a prior event affects the probability of a future outcome. If the little ball on the roulette wheel just landed on 30 Red, there is a flawed belief that 30 Red is a tad less likely to land the ball again on the next spin. Naturally, we believe the past is stored in some ledger, ready to affect the future. Watch out, Kevin Durant has a hot hand! As our brains detect patterns helpful to our ancestors to survive the day, we store what happened and try to connect it to what will. We are our ancestors’ children with practical pattern-matching skills, or we would not be here. It’s all good for staying alive another day. But give someone a hammer and, well, don’t look for patterns at the roulette wheel. Working the Shuttle’s probabilities, we may not have focused on people’s biases or misconceptions, but we did know how the wheel worked.
The car! Go for the car!
Besides Star Trek, where Spock says, “The odds against you and I both being killed are 2,228.7 to 1,” a child growing up on early 1970s re-runs might catch Let’s Make a Deal. You could do worse. Audience members chose between a sure thing like $500, ending the game and everyone’s fun, or possibly something better. The car! Go for the car! Famously, a contestant having chosen Door #3 saw the host, Monty Hall, unveil Door #2. Then we saw a stack of tuna cans or a goat chewing hay. (As a child, I wondered if you actually got to take home the goat, not a bad prize, because who wouldn’t want their own goat!) Then came the moment orchestrated to shift everyone forward in their seat. Monty Hall, the consummate gameshow host, easily kept the fun going. He asks the audience member who initially chose Door #3, now seeing a goat behind (initially unchosen) Door #2 – “would you like to switch to the unopened Door #1?”
Counterintuitively, switching doors is always the best move. The past event, like the ball landing on 30 Red, here showing a goat behind an unchosen door, has no effect on the future event. And the future event is a 2/3rds probability of a car behind either unchosen door. In this case, that’s the remaining door you did not choose initially. Your initial door still has only a 1/3rd chance of being the car. Switch doors! Switch doors!
Because probabilities are not patterns, but our brains insist on connecting the dots, events playing out tax our brain’s operating system. Monty unveils a door. The ball lands on Red 30. We think, erroneously, the next event must now have a different probability– because what are the chances of Red 30 twice? If a plane crashes into a home for sale, buy it. It’s been pre-disastered!
-o-
Who gets the goat?
More likely than not, NASA will soon launch its new Space Launch System (SLS). Around the same time, it looks like SpaceX will launch its new Starship in its complete configuration – unlike earlier tests with just the upper half, minus the booster. Blue Origin will come along with its large, new New Glenn rocket, and United Launch Alliance will have ready its Vulcan rocket (Spock would think the name a logical choice.) It’s easy to imagine the rooms of people talking about probabilities of success vs. failure on that first launch. There are many asides about partial success, the first vs. the second stage, the launch vs. landing, the software or the engines, and so on. Yet eventually, we will know what is behind each door, one rocket at a time.
Who gets the goat? FAA guidance says a new rocket from an experienced organization will succeed more likely than not on a first launch. A new rocket from a new organization will more than likely fail. For all four new rockets, all together, experienced teams and new teams, the chance at least someone has a bad day becomes a pretty good bet.
All this comes with asterisks about as long as the one’s on your cell plan bill. This is *only* true over many samples, much more than just 4 rockets. Also, we don’t live in all the universes where the samples can be run again and again. In my time running simulations at NASA, I was fond of running out 30 years, and running those 30 years 100 times, then averaging it all. In the real world, we don’t get to see the other 29 universes like the Avenger’s Dr. Strange. Car, goat, goat, car, goat, goat, goat, car and so on. Worse, we may not even see our own universe entirely play out. A two-handed economist has nothing on the likes of aerospace folk generating possibilities. NASA, SpaceX, and ULA have new rockets, except in the FAA asterisks, “A formal definition of “new launch vehicles” does not exist.” The same might be said of new operators and experienced operators, legacy know-how, and when it’s present or not.
This can all sound less than useless, more likely than not. The Shuttle program put astronauts on its first Shuttle launch. We knew long afterward the Shuttle program woefully overestimated its chances of launch success. Still, the optimistic probability supported going ahead with that first launch with a crew. Inversely, the Apollo program’s chance of success for a first Moon landing was 5%. The under-estimated number was kept hidden from the public, and the program proceeded with the Moon shot anyway. Spock spits out the numbers, but Kirk just goes ahead with the plan.
Who knows, a launch today, a launch tomorrow, before you know it, you might have regular launches once a day.
With time, there is an awareness the use of probabilities lies in recognizing the limits of small numbers. SpaceX has recently been launching once a week. Who knows, a launch today, a launch tomorrow, before you know it, you might have regular launches once a day. We must know where to change the game, between spins of the wheel. Knowing your probabilities tells you where to invest, but also the goal – to get the odds forever in your favor. Learning loves big numbers, not tens and hundreds of spins of the wheel, but thousands, and millions. Admittedly, NASA is not in the business of big numbers, but the recognition is there that if one day we have space travel as vast as air travel, NASA’s research and development investments will have played an important role. Of this, us number crunchers are quite certain.
Zapata Talks NASA 