What do you think?
Rate this book
ChatGPT-Based Learning And Reading Assistant (C-LARA): Second Report
ChatGPT-based Learning And Reading Assistant (C-LARA - pronounced "Clara") is an AI-based platform which allows users to create multimodal texts designed to improve reading skills in second languages. GPT-4/ChatGPT-4 is central to the project: as well as being the core language processing component, it has in collaboration with a human partner developed the greater part of the codebase.
Following on from the initial progress report, released in July 2023, we focus on new work carried out during the period August 2023 -- March 2024. The platform is far more usable. C-LARA is now packaged with a wizard-style interface ("Simple C-LARA") that allows the non-expert user to create a complete illustrated multimodal text by entering a prompt and approving default choices a few times, and the software is deployed on a fast dedicated server maintained by the University of South Australia. Other substantial new pieces of functionality are support for ``phonetic texts'', where words are automatically divided up into units associated with phonetic values; ``reading histories'', which support the combination of several texts into a single virtual document; and the social network, rudimentary in the first version, which now includes support for friending, an update feed, and email alerts.
To investigate the AI's abilities as a language processor, we present an experiment where we created six texts for each of five languages, using the same prompts for each language, and evaluated the accuracy of the language processing. We also give the results when some of the experiments were repeated five months later with a newer version of GPT-4, in the case of English revealing a dramatic reduction in error rates. A small questionnaire-based study probes users' subjective views of C-LARA projects they have created: in general, people are pleased with the results, to the extent that they are often sharing them.
With regard to GPT-4/ChatGPT-4's software engineer role, we present a breakdown of the various modules and functionalities, indicating the AI's contribution. It is capable of writing the simpler modules on its own or with minimal human assistance, and only had serious problems with a small number of top-level functionalities, in particular "Simple C-LARA", which directly or indirectly involved most of the codebase.
We describe initial use cases, including trialling of C-LARA in a school classroom, integrating it into the experimental CALL platform Basm, and creating multimodal texts in the Oceanic languages Drehu and Iaai. A short section summarises our policy on ethical issues concerning the crediting of the AI as an author. The appendices present examples illustrating use of the Simple C-LARA and Advanced C-LARA versions of the platform, list functionalities and code files, and reproduce conversations with the AI about various aspects of the project.
Following on from the initial progress report, released in July 2023, we focus on new work carried out during the period August 2023 -- March 2024. The platform is far more usable. C-LARA is now packaged with a wizard-style interface ("Simple C-LARA") that allows the non-expert user to create a complete illustrated multimodal text by entering a prompt and approving default choices a few times, and the software is deployed on a fast dedicated server maintained by the University of South Australia. Other substantial new pieces of functionality are support for ``phonetic texts'', where words are automatically divided up into units associated with phonetic values; ``reading histories'', which support the combination of several texts into a single virtual document; and the social network, rudimentary in the first version, which now includes support for friending, an update feed, and email alerts.
To investigate the AI's abilities as a language processor, we present an experiment where we created six texts for each of five languages, using the same prompts for each language, and evaluated the accuracy of the language processing. We also give the results when some of the experiments were repeated five months later with a newer version of GPT-4, in the case of English revealing a dramatic reduction in error rates. A small questionnaire-based study probes users' subjective views of C-LARA projects they have created: in general, people are pleased with the results, to the extent that they are often sharing them.
With regard to GPT-4/ChatGPT-4's software engineer role, we present a breakdown of the various modules and functionalities, indicating the AI's contribution. It is capable of writing the simpler modules on its own or with minimal human assistance, and only had serious problems with a small number of top-level functionalities, in particular "Simple C-LARA", which directly or indirectly involved most of the codebase.
We describe initial use cases, including trialling of C-LARA in a school classroom, integrating it into the experimental CALL platform Basm, and creating multimodal texts in the Oceanic languages Drehu and Iaai. A short section summarises our policy on ethical issues concerning the crediting of the AI as an author. The appendices present examples illustrating use of the Simple C-LARA and Advanced C-LARA versions of the platform, list functionalities and code files, and reproduce conversations with the AI about various aspects of the project.
144 pages, ebook
Published March 21, 2024
12 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 2 of 2 reviews
Read
April 4, 2024[Available for free here.]
April 8, 2024
If you were wondering why I've been posting so little the last few weeks, then wonder no more. Available for free download here.
___________
But seriously...
ChatGPT-based Learning And Reading Assistant (C-LARA) is a project that's been taking up most of my time for the last year. The basic idea was to build a web platform that lets people create easy-to-read multimodal texts in foreign languages, and have ChatGPT-4 do as much of the work as possible. Chat appears in two roles. With its software component hat on, it writes the texts, cuts them up into roughly sentence-length pieces, adds glosses to the words, marks them with root forms, and adds TTS audio. With its software engineer hat on, it's written most of the codebase. This report gives you a detailed picture of what we've done.
You may think there are too many details - does it really need to be 144 pages long? But it hasn't primarily been written for you, it's been written for ChatGPT-4. Although the AI is responsible for the greater part of the work performed in the project, it periodically has to be reinitialised, and then I need to tell it who it is again. Having this report available makes the job easy: I can give it the text in half a dozen instalments, which takes a quarter of an hour, and then it's up to speed again.
For humans, here are some of the bits you might find interesting. First of all, we've made C-LARA easier to use. There's a new top level called "Simple C-LARA", which lets you create a multimodal text with an initial request and a couple of button presses. You choose the text language and the glossing language, provide a sentence or two telling the AI what to write, and it creates a short illustrated text for you. (The illustrations comes from DALL-E-3). There are straightforward options to edit and correct when it gets things wrong. You can also paste in an existing piece of text if you prefer, and tell the AI to annotate it instead. For example, that's how I created the multimodal Norwegian passage in my review of Jon Fosse's Melancholia I-II.
Second, we've done some work to evaluate C-LARA. We describe an experiment we presented at the ALTA 2023 conference late last yeat, where we created six texts of widely different kinds in English, Faroese, Farsi, Mandarin and Russian and carefully checked how often the AI was making mistakes as it wrote and annotated them. Not surprisingly, it's much better at some languages than others. It turns out to be nearly as good at Mandarin as it is at English, but it's clearly worse at Russian, worse again at Farsi, and having serious problems with Faroese. Of course, it's miraculous that it can do anything in Faroese, an obscure Scandinavian language spoken by about 50,000 people. We repeated the experiments a few months later, and found it had improved a good deal in English. We also analyse the codebase to quantify the AI's contribution to writing it. We found that it had written nearly all of the simple modules and done the greater part of the work on the middling difficult ones. There were a couple of top-level pieces of functionality, in particular "Simple C-LARA", where it couldn't deliver: they required an overview of the whole project, and its context window doesn't seem to be up to it yet. I'm curious to find out what happens in GPT-5.
Third, we present some case studies where people have started to use C-LARA. We have a primary school teacher in Holland who teaches a weekly class for Romanian kids whose parents don't want them to forget their heritage language. We'd never used C-LARA with Romanian, a language I know nothing about, but Lucretia just tried giving it requests in Dutch and said it produces cute, funny little Romanian stories that the kids like. We've also been collaborating with linguists at the University of New Caledonia, who are using it for a couple of the Indigenous languages there. Here, the AI can't help with the writing; they need to create the texts by hand, but that's possible too. They're pleased with the results, which are freely available on the web.
In the appendices we give more details, including examples with step-by-step screenshots showing how to create a C-LARA text. Try it out, the platform is already a whole lot of fun to play with! It's just amazing what you can do with an AI to help you.
___________
[Update, Apr 9 2024]
One of the most useful things about writing this kind of document is that it forces you to think carefully about what tasks you should be planning to do next. We put together a long list in the "Further work" section (§9.1), and we've already started. More about that in a recent post on the C-LARA blog.
___________
But seriously...
ChatGPT-based Learning And Reading Assistant (C-LARA) is a project that's been taking up most of my time for the last year. The basic idea was to build a web platform that lets people create easy-to-read multimodal texts in foreign languages, and have ChatGPT-4 do as much of the work as possible. Chat appears in two roles. With its software component hat on, it writes the texts, cuts them up into roughly sentence-length pieces, adds glosses to the words, marks them with root forms, and adds TTS audio. With its software engineer hat on, it's written most of the codebase. This report gives you a detailed picture of what we've done.
You may think there are too many details - does it really need to be 144 pages long? But it hasn't primarily been written for you, it's been written for ChatGPT-4. Although the AI is responsible for the greater part of the work performed in the project, it periodically has to be reinitialised, and then I need to tell it who it is again. Having this report available makes the job easy: I can give it the text in half a dozen instalments, which takes a quarter of an hour, and then it's up to speed again.
For humans, here are some of the bits you might find interesting. First of all, we've made C-LARA easier to use. There's a new top level called "Simple C-LARA", which lets you create a multimodal text with an initial request and a couple of button presses. You choose the text language and the glossing language, provide a sentence or two telling the AI what to write, and it creates a short illustrated text for you. (The illustrations comes from DALL-E-3). There are straightforward options to edit and correct when it gets things wrong. You can also paste in an existing piece of text if you prefer, and tell the AI to annotate it instead. For example, that's how I created the multimodal Norwegian passage in my review of Jon Fosse's Melancholia I-II.
Second, we've done some work to evaluate C-LARA. We describe an experiment we presented at the ALTA 2023 conference late last yeat, where we created six texts of widely different kinds in English, Faroese, Farsi, Mandarin and Russian and carefully checked how often the AI was making mistakes as it wrote and annotated them. Not surprisingly, it's much better at some languages than others. It turns out to be nearly as good at Mandarin as it is at English, but it's clearly worse at Russian, worse again at Farsi, and having serious problems with Faroese. Of course, it's miraculous that it can do anything in Faroese, an obscure Scandinavian language spoken by about 50,000 people. We repeated the experiments a few months later, and found it had improved a good deal in English. We also analyse the codebase to quantify the AI's contribution to writing it. We found that it had written nearly all of the simple modules and done the greater part of the work on the middling difficult ones. There were a couple of top-level pieces of functionality, in particular "Simple C-LARA", where it couldn't deliver: they required an overview of the whole project, and its context window doesn't seem to be up to it yet. I'm curious to find out what happens in GPT-5.
Third, we present some case studies where people have started to use C-LARA. We have a primary school teacher in Holland who teaches a weekly class for Romanian kids whose parents don't want them to forget their heritage language. We'd never used C-LARA with Romanian, a language I know nothing about, but Lucretia just tried giving it requests in Dutch and said it produces cute, funny little Romanian stories that the kids like. We've also been collaborating with linguists at the University of New Caledonia, who are using it for a couple of the Indigenous languages there. Here, the AI can't help with the writing; they need to create the texts by hand, but that's possible too. They're pleased with the results, which are freely available on the web.
In the appendices we give more details, including examples with step-by-step screenshots showing how to create a C-LARA text. Try it out, the platform is already a whole lot of fun to play with! It's just amazing what you can do with an AI to help you.
___________
[Update, Apr 9 2024]
One of the most useful things about writing this kind of document is that it forces you to think carefully about what tasks you should be planning to do next. We put together a long list in the "Further work" section (§9.1), and we've already started. More about that in a recent post on the C-LARA blog.
Displaying 1 - 2 of 2 reviews