I wonder if the "advance" of AI "poetry" is dependent on its users being trained to prefer its "writing" to the writings of human writers, especially writers whose work is older and/or more complex. If, say, students use chatgpt regularly - to answer questions, write papers, etc - and working adults use it regularly in their jobs and in private life - dumb little things like composing facebook market place listings - and are not wrestling with language themselves even in little things, could this be the necessary conditioning a general public needs to enjoy machine-made "poetry"? It seems doubtful to me that a culture that required students to memorize and recite poems like Horatius at the Bridge would be taken in by this kind of verbal puffery AI produced for you. It isn't popular now, and in articles I've read about 19th century British education "learning by rote" is usually cast as a negative, but considering where self-expression has got us, one could say all that memorization of poetry and well-turned, meaning-full phrases *could* served one (and could serve us) as a kind of armor against empty, meaningless language and empty, meaningless thinking - or even just sloppy language and sloppy thinking. It could. All that would take work on our part, and it isn't fool proof.
I use llms at work to write code and structure projects. I find myself doing this in the way the llm likes. So my standards are adapting to the llms, which, to be fair, are the distilled standards of the industry. I don't mind this process at work...but for poetry, yes, I could imagine llm poetry getting better, then becoming the standard. Which would be sad.
I think our age has largely lost a taste for *urgency* -- what Sayers would have seen as, yes, akin to the Holy Ghost -- in poetry -- and that happened before ai. Annie Dillard talks about highly polished surfaces with nothing underneath. Wherever prestige, or any other artistically compromised -- perhaps clique-driven -- dynamic hijacks a genuine response, it erodes the irreplaceable and irreducibly individual burden of mystery for some brand of collective meaning/polish. An environment that makes it possible for soulless creativity to pass and flourish. (I don't mind Wonder being personified? And Lewis talks about how good poetry has a fluidity in metaphor where one metaphor is constantly vanishing/morphing into another -- but it feels in the ai poem like "canned elegance".)
Canned is right. And I guess I wouldnt mind the use of wonder if there weren't already so many generically personified things as there are. And isn't that part of the problem? It doesnt know that that's not something you do, because too many makes it trite. It has no sense of taste. It has no sense, really. Just a seeming sense.
By the way I've been reading that Alter translation of the Hebrew Bible you mentioned. The commentary is fantastic. Thanks for the recommend.
And yes, I agree about the whole genericness of its personification. Ai prose has the same "canned" feel to me, though in a lot of technical aspects it's better. The expression is always a bit too glassy (to personify -- and, rapidly change metaphors :-P).
I should have added above that I quite enjoy your essays I've been able to read (and I laughed about the number of wonder's fingers). Thank you.
Not only is AI poetry as boorish and brainless as Gaston, even worse, it is soulless and placeless. I have been thinking a lot about this topic and why it bothers me so much. I wrote a note and then deleted it without posting it in which I asked people to stop restacking AI generated content. I realize that people are restacking AI slop only because they can't tell it was AI generated, and my note will accomplish nothing. Frighteningly, AI poetry using form is significantly better than free verse. I read an article this week about how someone was unjustly accused of using AI, and all the comments were complaining that the AI checkers are inaccurate and inconclusive. I don't understand this claim. The person writing the article, who was accused of using AI, said they put Jane Austen's writing into an AI checker, and her writing was flagged as AI as well. I have no idea which checker they were using, and they are likely not created equal, but GPT Zero says Austen's writing is 100% human, unsurprisingly. As a teacher, I have used the checkers as a baseline, to confirm what I already knew. Every time a checker says there is an 80-100% chance that writing was written by AI and I have asked the student if they used AI, the student confessed to using it. I never bullied or accused. I just asked. They almost seemed relieved to be found out. Relieved because it means they are inherently superior to a computer. Error-laden, authentic writing with genuine mistakes means real learning is happening and a real grade is given. Deep down, students want to make their own mistakes. I always tell my students they are unequivocally better than a computer at reflecting on their own lives and education and processing this in written form, and if they are willing to work hard, they have what it takes. I have muted a couple writers I am certain are using AI and yet I see others restacking and praising their work. I will never ask AI to make me a meal plan or a workout or do any amount of thinking for me. Asking it to write a poem is as sacrilegious as asking it to pray for me. Poems and prayers are not commodities that can be outsourced. My friends accuse me of being bitter about AI. I know it sounds like a slippery slope fallacy, but I don't think there's an iota of our thinking we want to give over. I still remember what it was like for human beings to have fifty phone numbers memorized effortlessly, to be able to read maps correctly, to drive places without GPS, to cook recipes out of their head. It strikes me that I am very much preaching to the choir, and this note is embarrassingly long. Your essay struck a chord, and my brain is still vibrating.
As a teacher, it's been hard. Students use it for the sake of ease, not knowing what sort of contract they're making. They'll rely on it the rest of their life. When they have the inkling of a thought, their first instinct will be to go to a bot. When they read something they don't understand for the first time, when they have an idea, when they write anything, it will always have to be with that crutch. I've moved everything I can to paper. We write all our essays in class now. It's the only way you can get a sense for what they're capable of on their own.
The good news is, there's a deep sense of satisfaction to be gained from thinking one's own thoughts, and writing one's own words, and reading someone's genuine voice. That's baked into our nature. All it takes is someone realizing this to make the change and forgo what's easiest.
Yes! They can learn to value delight over ease. We have to believe this as educators for them to have a chance of catching it. I switched to in-class writing as much as possible, but the longer research papers are tough. I had outlines and elements of first drafts and workshops done by hand, but later realized this hadn't been foolproof. Some educators seem resigned that students will use it and caution against trying to find them out, lest it turn into a witch hunt. A simple bluebook essay is proof no hunt is needed. Students' original ideas will always be their best defense.
Right! What is safe from harm is what is not valued. Although there seems to be value in AI writing poetry as a sort of benchmark, which is how Altman and Cowen talked about it in the podcast. It's valuable because it seems out of reach.
The relative failure of LLM poetry versus prose is fascinating. It gets to what poetry is, and what LLMs cannot do. LLMs do facts and (surprisingly) simulate meaning pretty well. They also do word choice and word context pretty well. Poetry has a lot of meaning, and a lot of very carefully chosen words, so what's the problem? Well obviously a poem is more than it's meaning. It's sound, for instance. And feeling. LLM poetry has the meaning, but not the sound or the feeling. So is poetry defined by the inadequacy of mere meaning? "Time is passing and it makes me sad" is not a poem. Nor is "I feel empty and am waiting to be full." Those are not poems. So what is a poem? At the very least it the relationship between the meaning and all the extra-meaning elements--which might be supportive, or in tension or whatever.
It's a very interesting problem. It's at least a problem of more variables, like sound, as you mentioned. It would have to be trained on audio, because it would need to develop a sense for accent and time. But we understand so little about poetry, and have so little expertise, that the kind of conditioning and tunabilty would probably not produce the effect we desired. I remember Jordan Peterson once saying that he had assistants who created him an LLM using just biblical and philosophical texts, and he seemed to be impressed with the result. But you'd think, If you asked ChatGPT to write a poem in the style of Shakespeare, it would attend just to Shakespeare. The result is just as bad though, in a different way. It has no sense for what makes Shakespeare good. Can it learn these things? Again, I remain highly skeptical. I think there are limits to what it can simulate.
Why do we know so little about poetry? It really is a mystery. A lot of the formal stuff can be taught in a week. The ideas and content and meaning are just boilerplate from the times, mostly. Why so hard then? I dunno. Poetry is shaping up to be the little chicken bone that is stuck in the throat of AI.
Perhaps part of the problem is that the term is quite nebulous. If you go back far enough, poetry was synonymous with invention. In Sidney's defence of poetry, he conceives of it as something opposed to non-fiction writing. It's the mind's ability to imagine what is not there, to invent. You see this in the origin of the word, poeisis, to make. And poetry is a different thing than verse. Barfield talks about a poetic spirit, which is something like a mode of creative engagement with language. It's not just the techniques that we've discovered along the way and added to our repertoire of figurative devices, but a certain way of being in relation to the world. I think AI could only do poetry if it were actually conscious, and could take such a stance in regards to language.
An Ai project to help write poetry might go like this:
* Don't have AI write the poetry, just make it respond to more restricted prompts like this: Give me a figure for marriage drawn from anything to do with the kitchen or cooking. The marriage is coming after a volatile relationship and both husband and wife think that the formality of marriage will calm them down.
* It would use diffusion models instead of LLMs (like the much more successful image models). Diffusion models are diabolically clever. What they do is gradually add noise to images, then train a model on the opposite movement, form noisy to less noisy, with a prompt (some embeddings) to guide you on which direction to go as you move to less noisy versions. I'll bet this would work better than LLMs for poetry.
* Train on anthologies of poetry across languages. Make this elitist, like the Go models that use the plays of the masters, not any old game.
* What you are missing with poetry on the web is "work product." That is, information about the poetic process. There are plenty of advice forums for novelists, and coders, and gardeners. But almost nothing for poetry. I have these lines, I want to get to X, how do I get there? Force poets to get together :) and pair-program, to make their process explicit.
* Include the audio. Especially in a language as unpredictable in pronunciation and spelling as English, the training needs to be on audio as well. New multimedia models will be coming out soon, that can handle text and audio together.
* Lastly, get reader-reaction data into the mix. Elitist, of course, not any old reader. Maybe something as simple as galvanic skin response? Some other physiological measure of pleasure or surprise? I'm reading Dante now, and I literally get the chill on the regular.
As I said in the comment to Robert's note, this would take money and a massive amount of cooperation, something poets are not known for.
I think the fact that the average person prefers AI poetry to the real thing shows that, in the only way that matters to it, AI has already succeeded at writing better than humans. AI can and probably will get better and better at pleasing the average Bildungsphilister, but this by definition shuts it off from greatness. I think the real danger is that our brains could rot so thoroughly in the coming decades that no one will be left to tell the difference between great poetry and AI. I think that will happen long before Altman and co. succeed in building a Shakespeare simulator.
If you look at it from the quantitative angle: GPT-5 was trained on something like 45TB of data. The whole corpus of truly great English poetry probably adds up to a few megabytes. For every good line of poetry, it consumed thousands of lines of trash.
I think the right way to look at it is that, when you ask an AI to write a poem, it does a near-perfect impression of a mediocre poet rather than a bad impression of a great poet.
You can ask it to write in the style of Shakespeare, which I did for the philosophy poem it wrote. It was a pretty bad impression of Shakespeare. It has no instinct for the thought and imagination of Shakespeare, but then who does? How could you train it to? We dont even know why Shakespeare is so great, but anyone who's spent enough time reading him is aware of the difference. Are we assuming it could just figure it out given enough data points? I'm skeptical. It writes a Maya Popa poem well enough, as Popa herself has written about, and that seems to be where it stops, which is fine. It's still a great tool for other things.
Thank you Esther. I think it's definitely the case that when you study literature you understand what's missing from LLMs. As long as writers like Shakespeare are still around to read then people will be able see the difference.
I wonder if the "advance" of AI "poetry" is dependent on its users being trained to prefer its "writing" to the writings of human writers, especially writers whose work is older and/or more complex. If, say, students use chatgpt regularly - to answer questions, write papers, etc - and working adults use it regularly in their jobs and in private life - dumb little things like composing facebook market place listings - and are not wrestling with language themselves even in little things, could this be the necessary conditioning a general public needs to enjoy machine-made "poetry"? It seems doubtful to me that a culture that required students to memorize and recite poems like Horatius at the Bridge would be taken in by this kind of verbal puffery AI produced for you. It isn't popular now, and in articles I've read about 19th century British education "learning by rote" is usually cast as a negative, but considering where self-expression has got us, one could say all that memorization of poetry and well-turned, meaning-full phrases *could* served one (and could serve us) as a kind of armor against empty, meaningless language and empty, meaningless thinking - or even just sloppy language and sloppy thinking. It could. All that would take work on our part, and it isn't fool proof.
Anyway, really great article.
I use llms at work to write code and structure projects. I find myself doing this in the way the llm likes. So my standards are adapting to the llms, which, to be fair, are the distilled standards of the industry. I don't mind this process at work...but for poetry, yes, I could imagine llm poetry getting better, then becoming the standard. Which would be sad.
I think our age has largely lost a taste for *urgency* -- what Sayers would have seen as, yes, akin to the Holy Ghost -- in poetry -- and that happened before ai. Annie Dillard talks about highly polished surfaces with nothing underneath. Wherever prestige, or any other artistically compromised -- perhaps clique-driven -- dynamic hijacks a genuine response, it erodes the irreplaceable and irreducibly individual burden of mystery for some brand of collective meaning/polish. An environment that makes it possible for soulless creativity to pass and flourish. (I don't mind Wonder being personified? And Lewis talks about how good poetry has a fluidity in metaphor where one metaphor is constantly vanishing/morphing into another -- but it feels in the ai poem like "canned elegance".)
Canned is right. And I guess I wouldnt mind the use of wonder if there weren't already so many generically personified things as there are. And isn't that part of the problem? It doesnt know that that's not something you do, because too many makes it trite. It has no sense of taste. It has no sense, really. Just a seeming sense.
By the way I've been reading that Alter translation of the Hebrew Bible you mentioned. The commentary is fantastic. Thanks for the recommend.
He's truly brilliant.
And yes, I agree about the whole genericness of its personification. Ai prose has the same "canned" feel to me, though in a lot of technical aspects it's better. The expression is always a bit too glassy (to personify -- and, rapidly change metaphors :-P).
I should have added above that I quite enjoy your essays I've been able to read (and I laughed about the number of wonder's fingers). Thank you.
Not only is AI poetry as boorish and brainless as Gaston, even worse, it is soulless and placeless. I have been thinking a lot about this topic and why it bothers me so much. I wrote a note and then deleted it without posting it in which I asked people to stop restacking AI generated content. I realize that people are restacking AI slop only because they can't tell it was AI generated, and my note will accomplish nothing. Frighteningly, AI poetry using form is significantly better than free verse. I read an article this week about how someone was unjustly accused of using AI, and all the comments were complaining that the AI checkers are inaccurate and inconclusive. I don't understand this claim. The person writing the article, who was accused of using AI, said they put Jane Austen's writing into an AI checker, and her writing was flagged as AI as well. I have no idea which checker they were using, and they are likely not created equal, but GPT Zero says Austen's writing is 100% human, unsurprisingly. As a teacher, I have used the checkers as a baseline, to confirm what I already knew. Every time a checker says there is an 80-100% chance that writing was written by AI and I have asked the student if they used AI, the student confessed to using it. I never bullied or accused. I just asked. They almost seemed relieved to be found out. Relieved because it means they are inherently superior to a computer. Error-laden, authentic writing with genuine mistakes means real learning is happening and a real grade is given. Deep down, students want to make their own mistakes. I always tell my students they are unequivocally better than a computer at reflecting on their own lives and education and processing this in written form, and if they are willing to work hard, they have what it takes. I have muted a couple writers I am certain are using AI and yet I see others restacking and praising their work. I will never ask AI to make me a meal plan or a workout or do any amount of thinking for me. Asking it to write a poem is as sacrilegious as asking it to pray for me. Poems and prayers are not commodities that can be outsourced. My friends accuse me of being bitter about AI. I know it sounds like a slippery slope fallacy, but I don't think there's an iota of our thinking we want to give over. I still remember what it was like for human beings to have fifty phone numbers memorized effortlessly, to be able to read maps correctly, to drive places without GPS, to cook recipes out of their head. It strikes me that I am very much preaching to the choir, and this note is embarrassingly long. Your essay struck a chord, and my brain is still vibrating.
As a teacher, it's been hard. Students use it for the sake of ease, not knowing what sort of contract they're making. They'll rely on it the rest of their life. When they have the inkling of a thought, their first instinct will be to go to a bot. When they read something they don't understand for the first time, when they have an idea, when they write anything, it will always have to be with that crutch. I've moved everything I can to paper. We write all our essays in class now. It's the only way you can get a sense for what they're capable of on their own.
The good news is, there's a deep sense of satisfaction to be gained from thinking one's own thoughts, and writing one's own words, and reading someone's genuine voice. That's baked into our nature. All it takes is someone realizing this to make the change and forgo what's easiest.
Yes! They can learn to value delight over ease. We have to believe this as educators for them to have a chance of catching it. I switched to in-class writing as much as possible, but the longer research papers are tough. I had outlines and elements of first drafts and workshops done by hand, but later realized this hadn't been foolproof. Some educators seem resigned that students will use it and caution against trying to find them out, lest it turn into a witch hunt. A simple bluebook essay is proof no hunt is needed. Students' original ideas will always be their best defense.
Sartre's obsession with holes is such that I'd not want to mention his finger.
This was fun. I might noodle at this.
He had a hole so big no size of spiritual index finger could fill it.
Looks like there's an upside to the fact that poetry doesn't pay. And level 10 for the Indiana Jones reference!
Right! What is safe from harm is what is not valued. Although there seems to be value in AI writing poetry as a sort of benchmark, which is how Altman and Cowen talked about it in the podcast. It's valuable because it seems out of reach.
The relative failure of LLM poetry versus prose is fascinating. It gets to what poetry is, and what LLMs cannot do. LLMs do facts and (surprisingly) simulate meaning pretty well. They also do word choice and word context pretty well. Poetry has a lot of meaning, and a lot of very carefully chosen words, so what's the problem? Well obviously a poem is more than it's meaning. It's sound, for instance. And feeling. LLM poetry has the meaning, but not the sound or the feeling. So is poetry defined by the inadequacy of mere meaning? "Time is passing and it makes me sad" is not a poem. Nor is "I feel empty and am waiting to be full." Those are not poems. So what is a poem? At the very least it the relationship between the meaning and all the extra-meaning elements--which might be supportive, or in tension or whatever.
It's a very interesting problem. It's at least a problem of more variables, like sound, as you mentioned. It would have to be trained on audio, because it would need to develop a sense for accent and time. But we understand so little about poetry, and have so little expertise, that the kind of conditioning and tunabilty would probably not produce the effect we desired. I remember Jordan Peterson once saying that he had assistants who created him an LLM using just biblical and philosophical texts, and he seemed to be impressed with the result. But you'd think, If you asked ChatGPT to write a poem in the style of Shakespeare, it would attend just to Shakespeare. The result is just as bad though, in a different way. It has no sense for what makes Shakespeare good. Can it learn these things? Again, I remain highly skeptical. I think there are limits to what it can simulate.
Why do we know so little about poetry? It really is a mystery. A lot of the formal stuff can be taught in a week. The ideas and content and meaning are just boilerplate from the times, mostly. Why so hard then? I dunno. Poetry is shaping up to be the little chicken bone that is stuck in the throat of AI.
Perhaps part of the problem is that the term is quite nebulous. If you go back far enough, poetry was synonymous with invention. In Sidney's defence of poetry, he conceives of it as something opposed to non-fiction writing. It's the mind's ability to imagine what is not there, to invent. You see this in the origin of the word, poeisis, to make. And poetry is a different thing than verse. Barfield talks about a poetic spirit, which is something like a mode of creative engagement with language. It's not just the techniques that we've discovered along the way and added to our repertoire of figurative devices, but a certain way of being in relation to the world. I think AI could only do poetry if it were actually conscious, and could take such a stance in regards to language.
An Ai project to help write poetry might go like this:
* Don't have AI write the poetry, just make it respond to more restricted prompts like this: Give me a figure for marriage drawn from anything to do with the kitchen or cooking. The marriage is coming after a volatile relationship and both husband and wife think that the formality of marriage will calm them down.
* It would use diffusion models instead of LLMs (like the much more successful image models). Diffusion models are diabolically clever. What they do is gradually add noise to images, then train a model on the opposite movement, form noisy to less noisy, with a prompt (some embeddings) to guide you on which direction to go as you move to less noisy versions. I'll bet this would work better than LLMs for poetry.
* Train on anthologies of poetry across languages. Make this elitist, like the Go models that use the plays of the masters, not any old game.
* What you are missing with poetry on the web is "work product." That is, information about the poetic process. There are plenty of advice forums for novelists, and coders, and gardeners. But almost nothing for poetry. I have these lines, I want to get to X, how do I get there? Force poets to get together :) and pair-program, to make their process explicit.
* Include the audio. Especially in a language as unpredictable in pronunciation and spelling as English, the training needs to be on audio as well. New multimedia models will be coming out soon, that can handle text and audio together.
* Lastly, get reader-reaction data into the mix. Elitist, of course, not any old reader. Maybe something as simple as galvanic skin response? Some other physiological measure of pleasure or surprise? I'm reading Dante now, and I literally get the chill on the regular.
As I said in the comment to Robert's note, this would take money and a massive amount of cooperation, something poets are not known for.
Very nice analysis.
I think the fact that the average person prefers AI poetry to the real thing shows that, in the only way that matters to it, AI has already succeeded at writing better than humans. AI can and probably will get better and better at pleasing the average Bildungsphilister, but this by definition shuts it off from greatness. I think the real danger is that our brains could rot so thoroughly in the coming decades that no one will be left to tell the difference between great poetry and AI. I think that will happen long before Altman and co. succeed in building a Shakespeare simulator.
If you look at it from the quantitative angle: GPT-5 was trained on something like 45TB of data. The whole corpus of truly great English poetry probably adds up to a few megabytes. For every good line of poetry, it consumed thousands of lines of trash.
I think the right way to look at it is that, when you ask an AI to write a poem, it does a near-perfect impression of a mediocre poet rather than a bad impression of a great poet.
You can ask it to write in the style of Shakespeare, which I did for the philosophy poem it wrote. It was a pretty bad impression of Shakespeare. It has no instinct for the thought and imagination of Shakespeare, but then who does? How could you train it to? We dont even know why Shakespeare is so great, but anyone who's spent enough time reading him is aware of the difference. Are we assuming it could just figure it out given enough data points? I'm skeptical. It writes a Maya Popa poem well enough, as Popa herself has written about, and that seems to be where it stops, which is fine. It's still a great tool for other things.
These poems sort of remind me of my work in skill...I find that unfortunate.
The good news is you can get better, whereas I'm skeptical that AI can
Been writing poetry for half my life now….
Me too. It's one of the hardest things to get good at. Easy to pick up, difficult to master. A lifetime studying it is a life well spent, though.
Fair point actually.
Thank you Esther. I think it's definitely the case that when you study literature you understand what's missing from LLMs. As long as writers like Shakespeare are still around to read then people will be able see the difference.