By Gordon Rugg

There’s a lot of talk about Artificial Intelligence (AI) at the moment, usually framed in either/or terms. For anyone who worked with AI in the 1980s, this is depressingly familiar.

The brief version is that yes, AI will bring massive changes in some areas but not in the ways that most people are claiming, and no, it won’t bring massive changes in others that most people are worried about, and by the way, there are a lot of really useful things that it could be doing, but that have been marginalised or ignored or unknown for over forty years.

So, what’s the reality about AI?

The type of AI getting most attention at the moment involves a glorified version of predictive text. The AI knows which statements are most likely to follow a given initial statement, based on a huge corpus of text that acts as the source for those likelihoods (e.g. “In these 3 million exchanges on Twitter/X, this phrase followed that phrase on 231 occasions”). The full story is more complicated; for instance, you need some heavy duty grammatical rules to join the phrases together coherently, and you need ways of assessing which pieces of text to ignore. The core principle, though, is using correlations to string chunks of text together.

That’s the same underlying principle as the “Customers who bought this also bought that” approach, and it has the same strengths and weaknesses. When things go well, this principle can identify real correlations, including correlations that humans had missed. When things don’t go so well, it can identify real correlations, but in a way that completely misses the point. A common example in online shopping occurs when someone buys a gift for the legendary crazy uncle, and then buys themselves a reward that’s much more aligned with their own preferences. That’s how you can get recommendations like “Customers who bought Max Bygraves’ Greatest Hits also bought Soothing Nature Music”.

When things go badly, you can end up with results that are dangerously wrong. That’s bad enough on its own, but it’s doubly bad when the results look highly plausible.

Plausible but wrong results usually occur because the AI is just joining strings of letters together, without any understanding of what they actually mean. It’s a phenomenon that’s horribly familiar to anyone who has worked in education for a while. There’s a type of student that is very good at stringing together relevant buzzwords in a way that sounds initially plausible. Where it all goes wrong is when you ask them to apply what they’ve just said to a real problem. With students, you see a blank stare in some cases, and a look of horror in others, depending on whether or not they’re aware of why their knowledge isn’t joined up to the real world. That’s why I used extended case studies in my teaching, to make students apply what they were learning to real problems, and check that they really understand it.

This is why a lot of boring technical details are really, really important when setting up and using an AI system. If it’s getting its chunks of phrasing from the unfiltered Internet, for instance, then asking the AI who built the pyramids is likely to generate some very strange answers indeed. Even when you’re using text from peer-reviewed papers as your input for the AI, there’s no guarantee that their content will be okay; in fact, a key issue when human researchers do a proper survey of the literature on a topic involves critical assessment of that literature, not just repeating what was said within that literature. Incidentally, this is a significant problem with a lot of so-callled Systematic Literature Reviews; I’ve seen some appalling examples which were worse than useless, because they misunderstood key points in the texts that they were reviewing, and produced results that looked very plausible and scientific, but were actually profoundly wrong.

So, what is likely to happen with predictive text AI?

It will probably produce drastic changes in some fields that involve summarising and/or generating text. Examples include journalism and copy writing for routine copy. A key variable is whether the field involves significant risk of major legal action. If your AI generated text produces an unreliable claim in some advertising copy for a new type of coffee maker, then there’s not much risk of serious consequences. If, however, your AI generated text makes an error in a key point in an assessment of a medical procedure or a safety-critical software design, then the legal consequences could be very serious indeed. So, there’s a good chance that various fields will use AI to replace some of their staff, and a good chance that some of those fields will later seriously regret it and go back to the old ways.

What probably won’t happen is that predictive AI will be very efficient at generating new knowledge by stringing together bits of the existing literature. Yes, it will produce some new insights and associations, but those will need to be checked by a human in case they’re just coincidental. If you want to generate new insights and new knowledge, there are far more efficient ways of doing it by using other AI approaches.

Similarly, predictive text AI is unlikely to make all humans redundant. It does one thing very plausibly, and does it reasonably well most of the time, but most jobs don’t consist only of that one thing. Likewise, predictive text AI won’t turn into Skynet and obliterate us in a nuclear catastrophe; worrying about that risk is like worrying about whether your word processing package will make your household appliances try to kill you.

What depresses me about predictive text AI is the mediocrity of the claims about how it will be used. Yes, it will probably produce better chatbots. Yes, it will probably produce summaries of text, and answers to questions, that are as good as you’d get from a competent new hire. What gets me is the missed opportunities. Chatbots and text summarising might make short-term commercial sense, but they’re not likely to make many lives significantly better.

The key discoveries about AI were pretty well established forty years ago. Types of AI such as Artificial Neural Nets (ANNs) and genetic algorithms (GAs) and specialist systems emulating human expertise were all out-performing human experts in specialised tasks forty years ago. Instead of being taken up as powerful tools to support human experts, these technologies were largely used either for unglamorous low-level tasks such as optimising engine performance, or viewed as possible alternatives to human experts (and therefore as potential threats). There was far less interest in systematically identifying ways for humans and AI to work together on extending human creativity, or solving hard research problems, or generally making the world better.

That leads into the concept of task partitioning, which is about systematically breaking a task down into sub-tasks, and then allocating those sub-tasks to whichever route (human, AI, other software, hardware) is most suitable. This will be the topic of a future article.

Notes and links

You’re welcome to use Hyde & Rugg copyleft images for any non-commercial purpose, including lectures, provided that you state that they’re copyleft Hyde & Rugg.

You might also find our website useful: https://www.hydeandrugg.com/