← All field notes

Just Because We Can

By Caleb Crane

I read Paul Kingsnorth's Against the Machine last year and it has not left me alone since.

I want to be careful about how I recommend it, because it's easy to file as a fear book and dismiss. I don't think that's what it is. I think it's a warning that the road we are calling progress may end somewhere very different from where we assume, and that almost nobody is checking.

His argument, reduced to its simplest form: we have gotten extremely good at asking whether something is possible and extremely bad at asking whether it is good.

Progress as its own justification

Notice how rarely anyone defends a new technology on the merits.

The defense is usually that it's coming anyway. That someone else will build it if we don't. That you can't stop progress. All of which may be true and none of which is an argument that the thing is good.

This is the pattern of the moment, and it is worth naming because it operates below the level of argument. If it can be done, it will be done, and asking whether it should be is treated as naive, or nostalgic, or an attempt to hold back something inevitable.

But inevitability isn't a moral category. Plenty of things were possible and widely adopted and remain bad for us. We usually work that out twenty years late, after the harm is distributed and the profits are booked.

Where I actually sit

I want to be honest about my position, because I'm not an outside critic here.

I build automation for a living. I use AI every day. I teach students to build things with it. I'm not standing outside this industry throwing stones; I'm in it, and my income depends on it.

Which is exactly why the question matters to me rather than being an abstraction. I have to answer it about my own work, repeatedly, and sometimes the answer costs me something.

I think AI can be used in genuinely beneficial ways. I also think it can be used for serious harm. Both are obviously true and neither is interesting on its own.

The interesting question is who decides which, and when, and on what basis.

The two questions aren't the same

Here's the distinction I keep returning to.

Can we do this? A technical question. Answerable. Increasingly the answer is yes.

Should we do this? Not a technical question at all. It requires a view about what people are for, what a good life looks like, and what we owe each other. Engineering can't produce that view and was never meant to.

The failure of our moment isn't that we answer the second question badly. It is that we treat the first as though it settles the second. Capability arrives, adoption follows, and nobody ever convened to decide anything.

What this looks like in practice

I've turned down work. Not often, and not dramatically, but it has happened.

There are automations I could build that would work exactly as specified and make the humans on the other end of them worse off. Systems designed to make it harder for a person to reach a person. Tools whose actual purpose is to obscure who is accountable for a decision.

Those are all buildable. Some of them are lucrative. "It would work" was never the question.

The rule I keep is small: a tool is worth what it gives back to the people who use it, and to the people it is used on. If the second group is worse off so the first group can be more efficient, that isn't automation. That is just moving a cost onto someone with less power to refuse it.

Not a fear book

I said Kingsnorth is easy to misread and I want to close on that.

Reading him as anti-technology gives you an excuse to dismiss him. The more uncomfortable reading, and I think the right one, is that he is describing a culture that has stopped asking the question at all, and that we are all inside that culture, including the people who think they're being careful.

I don't agree with everything he concludes. I still work in this industry. But he made me notice how much of what I do is downstream of assumptions I never actually examined, and how easy it is to mistake momentum for direction.

Just because we can doesn't mean we should. It is nearly a cliché at this point, worn smooth from use.

It is also, as far as I can tell, still true, and still almost never applied.

Where in your work have you asked whether you can, and not yet asked whether you should?