Writing as synthesis
A short loop around the latest in generative AI and writing
News and views
This week you can catch me in full flow on the joys of writing, thanks to a lovely interview with student writing experts Carina Buckley and Alicja Syska. It’s part of a series on learning development and learning futures. Somewhere in the mix I confess that my substack writing is nothing more than a long digression from the writing I’m supposed to be doing, a kind of loop through the foothills that ends up covering more ground and (sometimes) gaining more height than the direct route to the summit.
The loop takes us first to Hollywood, where the Writers Guild of America has negotiated a Memorandum of Agreement on the use of generative AI in the script writing business. It’s not the outright ban that some were asking for, but it is a clear win for the striking writers. A new clause, Article 72, will be added to contracts (scroll to p.68 to find it). Just as Nature journals decided back in January that generative AI (GAI) could not in principle be the author or co-author of a published paper, Article 72 declares that:
because neither traditional AI nor GAI is a person, neither is a ‘writer’ or ‘professional writer’ … and, therefore, written material produced by traditional AI or GAI shall not be considered literary material.
As a consequence of this:
GAI-produced written material shall not be considered source material for purposes of determining writing credit [and] GAI-produced written material shall not be the basis for disqualifying a writer from eligibility for separated rights.
In other words, human screen writers get full credit even if the company insists on using GenAI material as part of the development process. Unless companies plan to shoot scripts that are AI generated in their entirety - no judgement calls, no good lines chosen over bad, no character edits, no interventions that might be deemed ‘writerly’ at any point - they have to pay writers the same. So there are no economic benefits. But GAI remains risky to those same companies, because as the memorandum slyly points out, writers and the WGA can still use existing laws to demand compensation for the ‘exploitation of their literary material to train, inform, or in any other way develop GAI software or systems.’
Or as Ted Goia wrote in his Honest Broker substack:
AI can exist as a tool for humans, but not as their cheap and inferior terminator and replacement.
AI is relegated to a warm-up exercise for real writers. An expensive and legally risky one. Writers two, terminators nil.
In an adjacent courtroom drama, the Authors Guild is well on the way to securing a jury trial for its class action against OpenAI, Meta and Google, where the class is professional writers ‘whose works spring from their own minds and and their creative literary expression’. Elsewhere in the terminator franchise, however, Goia sees musicians losing ground to the machines. He lists the signs that streaming services are flooding playlists with AI-generated content and ‘fake’ artists, perhaps because musicians have failed to organise as effectively. Screen writers and actors may have won concessions, for now, but ‘the sound track sucks’.
Will academics negotiate their own Article 72 for academic content? Or will students find their sources flooded with auto-generated study notes, the learning equivalent of hold music? Right now, the film has several endings in development.
And that reminds me.
This week’s top download…
… is a real blockbuster from the EU’s Joint Research Centre: On the Futures of Technology in Education (hereafter the ‘JRC report’)
Section 4: AI and learning analytics is detailed and perceptive on the state of the generative AI economy. ‘Training foundation models requires compute power that only a few organizations can afford’, the authors remind us, so development is likely to be ‘consolidated in a small number of dominant designs’. They don’t name the organizations with enough chips to buy in to the game, but we know who they are. OpenAI (in lockstep with Microsoft and its AI-enhanced Office suite); Google, rapidly embedding Bard and its multimodal model Palm into all of its products; and Amazon, now working with Huggingface (provider of open source transformer libraries and developer tools) to embed AI into its web services. Meta is making up for its late entry to the game by releasing a large language model based on Llama2 that can ‘emulate human expressions’ and comes with 28 distinct chat personas to fill your metaverse/social media feed with personalised joy. Only Apple is hiding its hand.
Funny how the ‘next big thing’ looks a lot like the last one, dominated by the same platforms and corporations.
But what if language models could escape the grip of the big four/five? There are now thousands of lightweight models, like Alpaca, developed at Stanford University’s Human-Centred AI lab. This LLaMA version was retrained in just 3 hours ($100) on prompt-output pairs generated from OpenAI use data ($500). Or the even neater Vicuna, trained only on user -shared conversations (so not scraped from anyone’s data warehouse: total $440). There are problems with small scale. The models are even more liable to misinformation, and the costs of generation are high compared with the costs of training. But these projects show it is possible to get a working language model up and running on consumer hardware for less than the cost of many e-textbooks.
As an alternative to ‘letting many flowers bloom’ in a landscape of open models and developer tools, what if large language models could be (re)built as a kind of public commons? Sweden is developing public sector language models for reasons of national sovereignty, for example, while EleutherAI offers a suite of models (Pythia) for open research, including research into explainability and transparency in generative AI. BLOOM is an open access model built on GPT2 by a 1000-strong research consortium, BigScience. The consortium took almost a year to build a multilingual text data set - 46 natural languages, 13 coding languages, 1.6 terabytes of text - and trained the model with compute donated by the French government (via CNRS). Could something like BLOOM, the JRC report authors wonder, become ‘a shared platform and a technological artefact where the interests of many developers and users meet’? One where:
part of the costs could be shifted to governments if educational benefits are identified from developing foundation models (or, for example, training such models for small regional European languages).
For educational organisations, and national education sectors, these are game-changing possibilities. But they demand collaboration on the scale perhaps only the EU can dream of. I have a long post about the model-as-commons idea, what is stacked against it, and why I think the education sector should invest in it anyway. Meanwhile, I have only one issue with the excellent JRC report: it doesn’t always join the dots between its hard-headed analysis of the big picture and its report of educational benefits to learners.
For example - looping back to the musical theme - the report compares AI writing tools to the synthesisers that revolutionised music production in the 1980s. It’s a neat analogy, but I’m not sure it’s the only one. Synth players use a vocabulary of recorded sounds, but they use it consciously and inventively (I’ve added a playlist at the end). Like the electric guitar for an earlier generation, the synth made it easier to sound quite good, but it was still the player who made the sounds into music. And a sample or loop from another artist is a homage to the original, adding another layer to the music’s meaning.
Generated text is not a homage to previous writing but a badly pirated copy. In the wider economies of creative production, generative AI might be more analogous to music streaming than music synthesis, delivering content in ways that tend to concentrate rewards with a small number of big players, and can make it harder for new talents to rise.
Once more with Vygotsky on bass
All this is to say that the benefits of AI in learning should be seen in the context of the wider economies and meanings of text production. And this is a context that universities can help to shape. A university that is committed to open access models, refined with open source tools to meet the needs of different knowledge communities, with the needs of research and learning to the fore, can offer different ways of relating to generative AI than a university that has signed up to a package of services from Google or Amazon or Microsoft. The risks and opportunities are different. The roles of learners, researchers and teachers within the two techno-social systems are different. Different capabilities will emerge.
The JRC report provides an invaluable overview of the economies of texts and models, the corporate AI industry and its alternatives. And this could be followed through in the analysis of educational opportunities and learning tasks. As the report says, generative AI can be used to summarise textbooks and academic articles, or to provide feedback on learners’ written work. But how do these new syntheses support different learners to become more capable writers, actors and thinkers? And shouldn’t these uses be seen as interconnected with the wider systems of copyright, copytheft, data privacy, surveillance and bias?
I was interested to find Vygotsky summoned as a witness for some of the claims of educational value:
From … a Vygotskian point of view, generative AI systems are not interesting because they produce text; instead, their relevance for education is in their capability to engage humans in advanced forms of thought where concepts, conceptual systems, and language are the tools for thought.
Vygotsky certainly regarded writing as a technology that mediates thought, like Ong and Macluhan later in the 20th century. For all these theorists, ‘forms of thought’ are shaped by tools for expressing and communicating them, such as the pen, printing press and camera, and (if they had lived to see them) the synthesiser and the computer. But for Vygotsky, thought is also inherently interactional, emerging through activities that involve other people, and cultural resources in the social environment (resources that Engestrom later codified as ‘rules, norms, roles, and divisions of labour’). Vygotsky did not locate capability in the minds of learners, or in their tools, but on the plane of inter-personal, tool-mediated activity.
I think Vygotsky would have been intensely interested in how generative AI is reshaping the rules, roles and divisions of labour involved in the production of text. But I suspect he would not have regarded the ‘forms of thought’ mediated by LLMs as ‘advanced’ just because the technology is complex. He would have asked, I imagine, what activities learners are engaged in, mediated by these new tools. What are they asked to do, and why? What relationships with other people are they entering into, including with people they will never meet, and what collective forms of knowledge production are they joining in with? And this opens naturally into questions - that certainly animated Vygotsky and his school of educational pscyhology - about difference, conflict, and power.
Outro/Coda
This has been a multimodal digression, and appropriately enough it comes to a close just as OpenAI announces new voice and image capabilities for ChatGPT. ‘Tried and tested’ prompts are already being offered to students and teachers for these new functions. They can’t have been tried in any real situation of learning, as yet, and some of them don’t even seem to have been tested by the people promoting them. ‘Two truths and a lie’, for example, is a terrible ChatGPT prompt, as anyone who has played with the bot for five minutes could tell you. ‘Getting ChatGPT to play the part of a patient’ is the #1 way to use it with medical students, apparently. Good luck with that one too.
As the JRC report says:
the importance of effective domain-specific prompt design has quickly been noted, and there is now a rapidly increasing group of people claiming to be specialists in this area. [However]… as the underlying language models are continuously changing, it is not clear that deep expertise can emerge in prompt engineering on these platforms.
In education, when the point is not to produce text but to learn from the process, the value of prescribed prompts seems particularly doubtful. What do learners have to gain from cutting and pasting text into a chatbot window, just to read the text that comes out? They might as well engage with… text, perhaps? Quality educational content? A search of wikipedia? Or they could sit back and wait for the direct-to-brain capability that Elon Musk is working on in his Neuralink labs, and give up on the tedious business of reading and writing altogether.
(No links to Neuralink from me - and if you plan on searching, a content warning about images of animal testing that you may find upsetting).
Thanks for reading, and here’s a synth-tastic generative AI playlist to take you through the weekend.
Tubeway Army: Are friends electric?
Kraftwork: The Model
Missing Persons: Words
Yazoo: I before E except after C [enjoy Alison Moyet reading part of a synthesiser manual]
New Order: Round and Round
Electronic: Disappointed