I’ve mostly avoided the dismal topic of Large Language Models and what they will do to the future of writing, both in college and outside it. But like the narrator of H. P. Lovecraft’s At the Mountains of Madness, “I am forced into speech because men of science have refused to follow my advice without knowing why.” The men of science are in charge of pedagogical rules and resources at MIT, and my advice is to restrict the use of LLMs by undergraduates—a path that calls for ingenuity and material support.
I don’t want to dwell on academic infighting. Instead, I want to write about why writing matters, inspired by the social historian Joe Moran’s ardent letter to his students, “You are not an angel.”
The fantasy of angelic communication—mind to mind, without the messy intervention of the body—is not new. It goes back to St. Augustine and returned at the birth of the internet. Moran cites Mark Dery, in Escape Velocity, on the “activist-entrepreneur Stewart Brand [who] argued that ‘when you communicate through a computer, you communicate like an angel’”: far from being estranged by the technological interface, we become “disembodied intelligences of great intimacy.”
Against this dream, Moran embraces imperfection:
A letter concedes the inevitable distance between writer and reader, and the failure of writing to entirely bridge it.
I could sum up what I want to teach you in five words: you are not an angel. Words are not an empty vessel through which meaning pours, clear and cold as water.
We see [writing] as a near-instantaneous, computational process, instead of a creative, imperfect, time-consuming human act.
The hardest thing to teach you, I have found, is that writing is meant to be hard.
I don’t agree with everything in the essay—or I don’t think I do. Aptly enough, the written word leaves me uncertain just how far our thoughts converge. We are not angels!
Some of what Moran protests is not specific to writing, as opposed to speech, which also deals with words composed and recomposed. The lines are blurred by SMS and online “chat.” And in the blur are social media, on which comedian Emo Philips has perhaps the most incisive take:
I like Twitter because it combines my two favorite forms of communication: texting, and throwing a note in a bottle out into the sea.
But the point remains that serious writing is revision, its product an asynchronous well of thought and affect, poured and emptied, mixed and remixed over time.
We speak about “distilling” one’s ideas into words because thinking and writing are rarely separable. What one thinks becomes more, or less, determinate in composition. One comes to see how murky it has been, or through what channels it must run; and seeing it, one comes to know one’s mind or redirect its course. To outsource this is to disown one’s own cognition.
It’s sometimes argued that the likes of ChatGPT are helpful tools, like the pocket calculator, which can be used judiciously as time-savers, improving the efficiency of “knowledge work.” ChatGPT will manage the mundane tasks, like turning notes into full drafts, leaving us more free to cogitate.
This is to bet on a conjecture in cognitive psychology for which we have no evidence and serious grounds for doubt: that the capacities involved in discursive reasoning are separable from those involved in writing and revising shitty first drafts. Perhaps it’s true that the skills outsourced to pocket calculators are “modular”: independent of growth in mathematical understanding. But if so, this need not generalize. The evidence that writing is thinking and that writing is revision—the empirical wisdom of writers everywhere—counts against the corresponding modularity for the skills of composition and creative thought.
Even if it could be done, outsourcing composition to LLMs would mean a terrible loss. To a first approximation, we could say that this proposal rests on an instrumental picture of writing. But we have to be more precise, since one could value the products of LLM-assisted composition for their own sakes, not as means to further ends. The problem is that outsourcing treats the process of writing as having merely instrumental value—as if we were content providers, and what mattered was only the “content” provided, and not the provision.
Some composition may be like that. There could be a place for LLMs in administration and book-keeping, for instance. But the writing I want to teach my students to do, the writing we do as humanists—including humanists in scientific fields—is not like that. It is continuous with creative thought and shares its value with the process of thinking.
Moran cites a recording of Virginia Woolf—the only record of her voice—from April 1937, when she spoke about “Craftsmanship” on BBC radio. What he does not say is that Woolf’s treatment of the difficulty of writing anticipates the problems posed by LLMs.
“Words belong to each other,” she intones, “although, of course, only a great writer knows that the word ‘incarnadine’ belongs to ‘multitudinous seas’.” The history of these “belongings” is the database of LLMs: they know it better than we ever could.
How can we combine the old words in new orders so that they survive, so that they create beauty, so that they tell the truth? That is the question.
But the sheer accumulation of connections won’t suffice. As Woolf complains, “there is, it would appear, some obstacle in the way, some hindrance to the teaching of words.”
For though at this moment at least 100 professors are lecturing upon the literature of the past, at least a thousand critics are reviewing the literature of the present, and hundreds upon hundreds of young men and women are passing examinations in English literature with the utmost credit, still—do we write better, do we read better than we read and wrote 400 years ago when we were unlectured, uncriticised, untaught?
Quantity does not make for quality, either with professors, or critics, or LLMs, and “to lay down any laws for such irreclaimable vagabonds [as words] is worse than useless.”
All we can say about them, as we peer at them over the edge of that deep, dark and only fitfully illuminated cavern in which they live—the mind—all we can say about them is that they seem to like people to think and to feel before they use them, but to think and to feel not about them, but about something different.
If LLMs think of anything, it is words, and if we only think of words, we do not think.
Finally, and most emphatically, words, like ourselves, in order to live at their ease, need privacy. Undoubtedly, they like us to think, and they like us to feel, before we use them; but they also like us to pause; to become unconscious. Our unconsciousness is their privacy; our darkness is their light…
It is in this darkness that our students, as writers and as thinkers, must learn to live—to move with patience, stumbling in the gloom; to bruise the bodies of their minds; to swim in the deeps of language and be lost for words.
"incarnadine" belongs to "multitudinous seas" because they're on the same line in Macbeth... so that's exactly the kind of connection an LLM will now pick up on. What the great writer did was to be the first to put them together, but maybe some random text generator would do that if we left it long enough. We just need to automate the 500 years of historical selection that gets that set of strings canonised as the great literature that the Woofs will be taught to admire.
But now we want to protest that it isn't a "learned response"... we really do admire the great line for its inherent greatness. How did we learn about that? "Incarnadine" is odd because a different modernist also used it to make a point, Ezra Pound in "ABC Of Reading". pgs 36-7
"The charging of language is done in three principal ways:
You receive the language as your race has left it, the words
have meanings which have 'grown into the race's skin';
the Germans say 'wie in den Schnabel gewachsen ', as it
grows in his beak. And the good writer chooses his words
for their 'meaning ', but that meaning is not a set, cut-off
thing like the move of knight or pawn on a chess-board.
It comes up with roots, with associations, with how and
where the word is familiarly used, or where it has been
used brilliantly or memorably.
You can hardly say 'incarnadine' without one or more of
your auditors thinking of a particular line of verse. "
https://monoskop.org/images/a/a4/Pound_Ezra_ABC_of_Reading.pdf
Dear old Edward Craig, as well as his lectures on current and old philosophy, wrote the wonderful book "The Mind Of God And The Works Of Man". His overall thesis was that there has been a shift in western philosophy since the Renaissance. The early moderns such as Descartes and Spinoza are concerned with pure Thought, which is poorly expressed in worldly language, because the central image guiding them is the idea of God who just sees how things are. Whereas the world after Kant put Man The Doer/Maker at the centre of the universe, and so we end up with various "linguistic turns" that treat natural language as primary and are distrustful of immaterial Thoughts. Ideas, Beliefs. As you'd expect from Craig, there is some amused scepticism about what Kant and Hegel thought they had achieved, and their influence was explained as their utility to anyone who wants to cite an influence.
As usual with this sort of broad-brush history, it gets a bit stretched the closer it gets to present time, when there are lots of different people doing different things rather than a small elite working the same problems (see also: Stephen Toulmin's "Cosmopolis"). There is the obvious point that the notion of ideal logical languages was still active in the 20th century, and some of the attraction of psychoanalytic theory is that it is getting at the real machinery behind the appearances.
But I think he was right in sensing a tension between 2 competing images: humans as observers, trying imperfectly to capture the world as they found it, and humans as the creators of their world, who are imperfectly theorising what they are doing along the way. Debates about authenticity, ownership, originality and awareness are run through with background assumptions which are drawing on different traditions and projects that have left their traces.
If we're going to talk about "the rise of science", remember there are at least 2 kinds of science: the physical and biological ones - the so-called "hard sciences" - but also the rise of the social sciences and anthropology, which very much put Man at the centre of the universe even when the former were supposedly moving him away from there.
Beautifully put, and hard to find much to disagree with in either your piece or Moran's. However, I am increasingly convinced that those who want to restrict or limit this sort of technology in education aren't fighting their students or even their administrators; they're fighting a kind of ideology. One so deeply baked into contemporary social life that it floats free of any particular political orientation or economic system. It's all basically there in Jacques Ellul, and it's what Neil Postman was fumbling towards when he spoke of a "Technopoly".
Under this ideology, if technology *can* be deployed in the service of some short-term convenience, it generally will be. There is simply no stopping this by conventional political or ideological means; arguments given to students or academic staff about long-term harms or violations of deeply held values are ineffectual. And each new incursion is sorites-sized, such that at each stage we don't notice that we are acquiescing to an entirely new form of life, one in which embodiment simply means much less, and where the connection between embodiment and words (and sounds, and images) is completely severed. "Let's do our department meetings on Zoom!" "Let's have course discussions online via a convenient message board!" "Let's save money and increase accessibility by moving that course online!" Each a grain of sand added to the pile.
Most of my philosophy of tech. students enthusiastically support the idea of a humanities education that returns to more traditional modes of teaching and learning, rigorously enforcing all the difficult measures this would involve. But it simply will not appear as an option, despite the fact that we allegedly inhabit a "supply and demand" economy and a democratic polity.
In short, I fear it will take much more than words to halt the LLM's assault on words.