Content Providers

I’ve mostly avoided the dismal topic of Large Language Models and what they will do to the future of writing, both in college and outside it. But like the narrator of H. P. Lovecraft’s At the Mountains of Madness, “I am forced into speech because men of science have refused to follow my advice without knowing why.” The men of science are in charge of pedagogical rules and resources at MIT, and my advice is to restrict the use of LLMs by undergraduates—a path that calls for ingenuity and material support.

I don’t want to dwell on academic infighting. Instead, I want to write about why writing matters, inspired by the social historian Joe Moran’s ardent letter to his students, “You are not an angel.”

The fantasy of angelic communication—mind to mind, without the messy intervention of the body—is not new. It goes back to St. Augustine and returned at the birth of the internet. Moran cites Mark Dery, in Escape Velocity, on the “activist-entrepreneur Stewart Brand [who] argued that ‘when you communicate through a computer, you communicate like an angel’”: far from being estranged by the technological interface, we become “disembodied intelligences of great intimacy.”

Against this dream, Moran embraces imperfection:

A letter concedes the inevitable distance between writer and reader, and the failure of writing to entirely bridge it.

I could sum up what I want to teach you in five words: you are not an angel. Words are not an empty vessel through which meaning pours, clear and cold as water.

We see [writing] as a near-instantaneous, computational process, instead of a creative, imperfect, time-consuming human act.

The hardest thing to teach you, I have found, is that writing is meant to be hard.

I don’t agree with everything in the essay—or I don’t think I do. Aptly enough, the written word leaves me uncertain just how far our thoughts converge. We are not angels!

Some of what Moran protests is not specific to writing, as opposed to speech, which also deals with words composed and recomposed. The lines are blurred by SMS and online “chat.” And in the blur are social media, on which comedian Emo Philips has perhaps the most incisive take:

I like Twitter because it combines my two favorite forms of communication: texting, and throwing a note in a bottle out into the sea.

But the point remains that serious writing is revision, its product an asynchronous well of thought and affect, poured and emptied, mixed and remixed over time.

We speak about “distilling” one’s ideas into words because thinking and writing are rarely separable. What one thinks becomes more, or less, determinate in composition. One comes to see how murky it has been, or through what channels it must run; and seeing it, one comes to know one’s mind or redirect its course. To outsource this is to disown one’s own cognition.

It’s sometimes argued that the likes of ChatGPT are helpful tools, like the pocket calculator, which can be used judiciously as time-savers, improving the efficiency of “knowledge work.” ChatGPT will manage the mundane tasks, like turning notes into full drafts, leaving us more free to cogitate.

This is to bet on a conjecture in cognitive psychology for which we have no evidence and serious grounds for doubt: that the capacities involved in discursive reasoning are separable from those involved in writing and revising shitty first drafts. Perhaps it’s true that the skills outsourced to pocket calculators are “modular”: independent of growth in mathematical understanding. But if so, this need not generalize. The evidence that writing is thinking and that writing is revision—the empirical wisdom of writers everywhere—counts against the corresponding modularity for the skills of composition and creative thought.

Even if it could be done, outsourcing composition to LLMs would mean a terrible loss. To a first approximation, we could say that this proposal rests on an instrumental picture of writing. But we have to be more precise, since one could value the products of LLM-assisted composition for their own sakes, not as means to further ends. The problem is that outsourcing treats the process of writing as having merely instrumental value—as if we were content providers, and what mattered was only the “content” provided, and not the provision.

Some composition may be like that. There could be a place for LLMs in administration and book-keeping, for instance. But the writing I want to teach my students to do, the writing we do as humanists—including humanists in scientific fields—is not like that. It is continuous with creative thought and shares its value with the process of thinking.

Moran cites a recording of Virginia Woolf—the only record of her voice—from April 1937, when she spoke about “Craftsmanship” on BBC radio. What he does not say is that Woolf’s treatment of the difficulty of writing anticipates the problems posed by LLMs.

“Words belong to each other,” she intones, “although, of course, only a great writer knows that the word ‘incarnadine’ belongs to ‘multitudinous seas’.” The history of these “belongings” is the database of LLMs: they know it better than we ever could.

How can we combine the old words in new orders so that they survive, so that they create beauty, so that they tell the truth? That is the question.

But the sheer accumulation of connections won’t suffice. As Woolf complains, “there is, it would appear, some obstacle in the way, some hindrance to the teaching of words.”

For though at this moment at least 100 professors are lecturing upon the literature of the past, at least a thousand critics are reviewing the literature of the present, and hundreds upon hundreds of young men and women are passing examinations in English literature with the utmost credit, still—do we write better, do we read better than we read and wrote 400 years ago when we were unlectured, uncriticised, untaught?

Quantity does not make for quality, either with professors, or critics, or LLMs, and “to lay down any laws for such irreclaimable vagabonds [as words] is worse than useless.”

All we can say about them, as we peer at them over the edge of that deep, dark and only fitfully illuminated cavern in which they live—the mind—all we can say about them is that they seem to like people to think and to feel before they use them, but to think and to feel not about them, but about something different.

If LLMs think of anything, it is words, and if we only think of words, we do not think.

Finally, and most emphatically, words, like ourselves, in order to live at their ease, need privacy. Undoubtedly, they like us to think, and they like us to feel, before we use them; but they also like us to pause; to become unconscious. Our unconsciousness is their privacy; our darkness is their light…

It is in this darkness that our students, as writers and as thinkers, must learn to live—to move with patience, stumbling in the gloom; to bruise the bodies of their minds; to swim in the deeps of language and be lost for words.