Design and Agency

Shoot for the moon. Even if you miss, you’ll land among the stars.

Norman Vincent Peele

… but I keep hitting London.

Mort Sahl, in regards to I Aim at the Stars

I’ve been sitting on a “Design and Culpability” piece since starting to write two years ago. There’s a fire here and brimstone there, and I’m unfortunately only motivated to flesh it out when something’s on fire. Fortunately for you, dear reader, the world’s aflame.

When I started at Zello over ten years ago, I remember some of the first “trending” channels I found were named “LOUCW,” who I later learned were the “League of United Christian Warriors.” This reoccured to me in April 2020, when I decided to delve into moderation on our Friends & Family platform. What I found was grim, it prompted a lengthy internal discussion regarding freedom of speech^[1] and a private company’s role in censorship.

I carry my opinions like a mallet^[2] on the best of days, and Summer 2020 was a litany of bad days. I argued that by enabling harm, we were responsible for its mitigation. By creating a tool that could be used maliciously, we were culpable for the harm its users wrought.

I’ve softened since, and realized that while I apply that standard to myself, I don’t to others. However, I will hold myself and others equally to a standard of goodness. As opposed to “I’m responsible for everything done with the tools I build,” I ask “What uses of my tool lead to the world I want to see?”

With that in mind, if I make a hammer, I’m not responsible for what’s done with it. However, I can encourage that it’s used as I envision. If a loud, angry man comes into my hammer store, pauses, and kindly asks to buy a hammer, I have reason to suspect he’ll misuse it. I sure as shit shouldn’t sell it to him.

At what point does speech predicate harm? At the point that it’s considered hate speech, sure. A group of III% Militia^[3] members discussing their next act of domestic terror is harm, and a group of Threepers discussing their personal racial hierarchies is close enough if not already hate speech. A few Proud Boys in an empty room, or talking about their dinner plans? Should I care about smoke, the fire, or both?

I wrote an internal proposal for how we should handle moderation processes and tools at the company, and hounded it at the followup discussion at our monthly all-hands meetings until we did something.^[4] This didn’t make me many friends at the time, and while my colleagues have softened a hair, I still tread carefully.

These days, I hesitate before speaking up about an impactful issue. As useful as has been my woeful lack of tact, I’m afraid of poisoning the well for others who could achieve the same outcomes without alienating their peers. However, I still worry that without my blunt pigheadedness, the conversation may not start.

While ICE is kidnapping children and murdering and disappearing innocents, not just my company but the broader tech industry is being asked: “what will you allow?”

I’ve watched this question be mired down by policy and precedent. “How do we consistently enforce these changes on such a massive platform?” and “Hate symbols in one language and locale don’t necessarily map to another; how can we responsibly apply these standards consistently?” In ICE’s case, “how can we prevent use by one law enforcement agency but not others?”

I propose simply that we don’t — those being harmed don’t give a rat’s ass about our policy and precedent; they care about the boot on their fucking neck. Rather than litigating a proposed policy, we must back up and ask ourselves: “what’s acceptable use for our lives’ work?” and “do I want Dilley on my conscience?”

We’re not necessarily culpable, but we have an opportunity to reduce^[5] harm, and what matters is the lived effect. This leads to another point — October 2019 saw Github employees quit over their acquirer, Microsoft’s, DHS contracts. Yes, object to your employer’s^[6] blood money. However, were the same users enjoying free accounts, I wonder what the outcry would’ve resembled. Whether on a free or paid tier, Discord’s 2000 banned communities made — and continue to make — heavy use of the platform. Had IBM freely given away its punchcards, outcomes would’ve been equally if not more heinous.

With that rancid comparison, we return to my point. Responsible or not, we can each make our small corner of the world a little better. Let’s not overthink it.