production outages are like sexism
September 16, 2018
I write this as a privileged white cis woman, or at least that’s what I look like, with my blue hair, ivy league education and house in San Francisco. I have compound privilege, so what I describe below represents an easy life.
I work in the tech industry.
I started programming when I was twelve. My first commercial software was a Mac application. People bought our software and we sent them a box with a paper manual and some floppy disks. I thought sexism was something I would only experience from old men and young jerks. There were a lot of messed up things in the world, but on this point I thought my generation was cool.
Now I work on distributed systems. People fill out a web form and enter their credit card and then just start using our software. It runs 24×7 on computers all over the world.
We write good code. We have tests that run automatically when we change anything, and extra checklists when we make something new. We still write bugs, and even if our code were perfect, it runs on hardware that can fail. So we create data centers with redundant systems, and just in case there’s a storm or earthquake, we run some more copies of the software on the other side of the planet. We do all that work, and still, we prepare for a production outages — that’s what we call it when things are so bad that people will start noticing. It’s inevitable. It happens. In fact, it happens so much it’s routine. We prepare. We create systems to alert us as early warning signals, so we can do something to avert catastrophic failure.
No matter how much we prepare. No matter how hard we work. No matter how brilliant we are at writing code. It happens.
By the time there’s a production outage, We’ve already worked through all the solutions in the standard playbook and nothing worked. Maybe we’ve already looped in a few colleagues who happen to be awake or nearby. We start improvising. We try anything that is both fast and safe. As we pull in some experts, we hope we’ve written all the important things down.
Afterwards, we’ll do a root cause analysis. It could be a mistake we made 6 months ago that’s been quietly corrupting data. Maybe we made an incorrect assumption, or some other system changed and we weren’t notified. Sometimes we could have prevented the problem. Sometimes the root cause is entirely outside of our control.
Sexism is like a production outage.
We learn to expect it. We create systems that mitigate risk. We learn the rules. We study the written laws and policy, because we find that the unspoken rules we learned as kids don’t make any sense. Guidelines about appropriate behavior box us in so tightly we can’t actually get anything done. Sometimes you can ignore the rules.
If you have the privilege of working with men who will respect you, it can work pretty well to pretend these unwritten rules don’t exist. If you want to move to a new team or a new position, you make a judgement call: is the opportunity worth the risk?
There are places I won’t work. Being part of the next unicorn startup isn’t worth the kinds of production outages I would expect there — it’s too disruptive to get any work done.
To be successful in the software industry, we need to take risks — make new software, work with new teams. Men take risks too and can experience their own production outages. And I’m sure I can’t imagine your situation, but from what I’ve observed in the United States, there’s a class of privileged white cis men where failure is a sign of positive risk taking. For blacks and latinos, it seems that failure is more often perceived as failure, or condescendingly, as a lack of preparation that is not their fault. Asian men seem to have fewer production outages in their early careers, then at some point they stop advancing and hit the bamboo ceiling.
I’m not sure the tech industry is really worse than any other place to work, but it does seem different. I wonder if its because many of us were like me. I expected it to be different. When I encountered obstacles, I just worked harder. I thought everyone was held to the same high standards, but when I look back, I’m not sure that’s true. If you have to defend your ideas and plans more than your colleague, that slows you down. If you don’t ask permission, it’s more likely you will be held accountable for your failure, so you have to make sure you don’t fail. We need to take risks to do great things, but for some of us, there are more risks and the risks are bigger.
The “leaky pipeline” is well-documented. Through my own lived experience and hundreds of women I’ve known as fellow travelers in this strange world of the tech industry, I see patterns that seem to be invisible to most of my male colleagues. This metaphor of the effects of sexism as a production outage helps me develop strategies and advise other women when they face obstacles. If there is only one woman on a team, it’s hard to tell the difference between sexism and incompetence. Is she being treated differently? Did he just make a mistake? The research is interesting, but when it happens to you, it doesn’t matter. You can do root cause analysis later. Right now, you need to put your life on hold and deal with a production outage.