There is the movie “The Sixth Sense,” and there are NASA cost estimates. I am not sure which is a better example of the unreliable narrator. “Call me Ishmael.” Why should I call you Ishmael? Most people would say, “My name is Ishmael.” What are you hiding? Of course, the first version sounds more intriguing. I want to know more. It turns out it’s a whale of a tale, and the whale gets bigger with every telling.
Above: The schedule from a few days after I first reported for work at NASA Kennedy Space Center. It was called the “KICS” for Kennedy Integrated Control Schedule. Credit: Edgar Zapata, zapatatalksnasa.com
Having been on one too many NASA cost estimating teams, I came to realize the critical question was not “what will it cost?” The real intriguing question was, “does anyone really want to know what it will cost?” I was told, many times frankly, if we said what it might cost, we’d never get any projects approved in NASA ever again. One time the team leader pulled their chair toward mine, knee to knee, to convey this, after perusing my analysis. Awkward, but ineffective. Initially, and erroneously, I thought I wanted to nail the number so years later, I could whip out my estimate when the project finished. I would go to the closing paragraphs and the final number, always with a give or take. It would be close to the mark, close to how it really turned out. Not the case.
I actually did this once. I learned rule #1 in cost estimating – no one ever, at all, under any circumstances, likes to hear “I told you so.” This much is constant, even in alternate universes. I had been told cost estimating was a learning process, and this was true. Though to be truthful, I pulled off the correct cost estimate by cheating. Lacking a time travel machine then, I had discovered that a great starting point to the question “how much will it cost?” was “how much do you have?”. This cheat does not always work, and I discovered a better method was to couple “how much do you have? to “how long do you have?” Time is money after all.
At the time, “save a tree” was not yet a thing.
All humor aside, it’s worth going back to the beginning. I had just arrived at Kennedy, noticing on the third day at the office that a large half-inch thick document was left in front of the locked doors every morning. Stacks of these were here and there as well for the taking (if you were first, you fought with the string tying them all together). Like an early morning newspaper on everyone’s stoop, these were everywhere around the space center, from Headquarters to our office complexes to the gargantuan VAB. I asked, “can I keep one?” Yes, of course. There were many more to come. Even a smaller one with just the highlights most days. At the time, “save a tree” was not yet a thing.
Above: Left, the major schedule events sheets at the start of a daily schedule package. Right, an example schedule from the 1988 Kennedy Integrated Control Schedule (KICS), for Pad B and Firing Room 3, Tuesday October 25 to Friday October 28. Credit: Edgar Zapata, zapatatalksnasa.com
These were schedules. Among the things I learned that first week, this place is big on schedules. Months later, I was told to cover the daily scheduling meeting over in the Launch Control Center. “Daily,” as in there were two a day. Why have a single meeting when you can also have a pre-meeting. I looked forward to covering the meeting, which seemed to me, an easily impressed grunt at the time, an important meeting in an important room with important people. The meeting room was once wall-to-wall schedules. There were huge uninterrupted walls, front and sides just for this. The black and white lines and grids reached where they would have needed a ladder and a thousand magnetic labels back in the day. The schedules were not kept on the walls anymore here, though, everyone walking in with their trusty “KICS” hardcopy. Going to church? Bring your bible. It’s part of the look, I learned.
After the first time I covered the meeting came many more. Some years in, everyone getting caught in a rainstorm, I rushed drenched to some random seat in the back. Usually, there were assigned seats. Today everyone ended up scattershot. In a very hierarchical arrangement, seats at the table were labeled. Seats around the table seemed to go from more to less important, with the nose bleed section three rows back. This place was big on schedules and structure. By chance, this time we were days away from a launch, and there was a major mechanical problem on the Shuttle. This is not to be confused with the usual brokes. This was a big one.
Next to me, by chance, ended up sitting our public affairs representative. It was pretty clear we would not launch on Friday, and telling the world would save a lot of tourists a lot of trouble and traffic. I leaned over to the public affairs rep with a whisper about how “I guess it’s time to tell the tourists it’s no go.” The reply – umm, not quite.
The team was told to troubleshoot the problem, come back later (another meeting, same day), and re-assess tomorrow (the next meeting). In the meantime, we are still working toward a launch on Friday. So, another day came and went. Then another. As said earlier, the repair was very likely and would take many days. The tourists never got word until late the day before the canceled launch.
Schedules were made to be changed. In government, given even more years, episodes like these would happen again, eventually, a supervisor telling me, “We say what we do, and that’s all.” To add in the meaning of what we do, how what we do adds up or not, was above our pay grade.
More so, I had joined a society of unreliable narrators.
Above our pay grade? Tell me more. Intriguing, but incomplete. Of course, I went like Alice down that rabbit hole. Seeming a lifetime later, really just 10 years in, I had wandered into cost estimates in NASA programs. In 1999 I watched Bruce Willis as the psychologist on the big screen, who thinks he is helping a boy, only to find out – true, intriguing, but not the whole story. More so, I had joined a society of unreliable narrators. It was vast, pretty exclusive, and did a lot of math. The math part is why I fit right in, I suspect (I’m pretty decent at math). But of course, there was much more underneath the math.
“Optimism” is perhaps one of the most common words in NASA next to “cost estimates.” (It appears 26 times in an Inspector General report on the subject). We didn’t get paid to discover problems. We got paid to find solutions. Projects reported what they were doing. Report on plans? The plan is to launch on Friday. There is even a school of thought where estimating costs for NASA projects is not only thankless but worse, it’s impossible (This is part of the “you can’t nail jello to a wall” school, “endogenous” effects, and all that.) “It’s a process” was a first cousin. Being too-close followed as another theory, but projects had a way of making sure independent assessment, from afar, would not happen. “It’s a team process” – came the clarification.
As I write, I must remember all these experiences. The whale was huge, and malign, and NASA cost estimators are here to help that project, and we plan to launch on Friday. More so, it’s worth remembering this in a world of fresh ideas, start-ups, technology and innovation. Since wild ideas are wild, until they are the baseline, perhaps we need to ask “who helps who?” among cost estimating and new space projects. A joke once upon a time in cost circles was “we need to build a reusable launch vehicle, so we can make better cost models.” Cost estimating for space systems may want to help a project, like Bruce Willis wants to help the boy, but it should worry about its own state too.
Also see:
- The KSC Next-gen page on our operations cost modeling at the Wayback Machine
- What is wrong with space system cost models? A survey and assessment of cost estimating approaches, by Kellern, Collopy, and Componation
- Underestimation of Project Costs, by Jones
- NASA’S Challenges in Meeting Cost, Schedule and Performance Goals, Inspector General Report
Zapata Talks NASA 