Large language models have pushed their way through to the other side of the hype cycle. After the hype emerged a few years ago, a consequent negative reaction followed. But despite all the concerns, it is now clear they are here to stay.
It is easy to understand the appeal. People are finished with navigating through complex, bloated websites. Young people don’t even want to begin navigating them. Many demos of artificial intelligence tools centre their pitch on a promise to save time navigating complex information environments and large volumes of content.
People crave fast answers
This is the era of the answer engine.
The rise of “zero click” searches shows us this. An increasing number of web searches end without the user clicking on anything. One consultancy, Bain, estimates that “80% of consumers now rely on “zero-click” results in at least 40% of their searches”. Many websites are seeing organic search traffic plummet.
It is a complete role reversal from the job Google originally saw for itself. Google originally wanted users to spend as little time as possible on Google, seeing users’ goal as to get to a relevant webpage.
This trend appeared to begin even before Google started including “AI overviews” at the top of their search engine results pages. In a sense, this is a natural extension of their existing knowledge panels. These would display information from their knowledge graph directly within the search results, instead of just linking off to authoritative web resources.
Information systems exist to help people learn
But there is a clear tension between the convenience of a quick answer versus the usual purpose of an information system.
Normally, tan information system is there to help people gain greater knowledge. As per Jorge Arango’s information architecture first principles:
There are no ‘intuitive’ information structures, only learnable ones; our job is building scaffoldings that allow people to find their way to knowledge.
Learning happens over time
Strongly tied to this is the evidence, from research by Marcia Bates, about how people actually use information systems — through “berrypicking”.
In real-life searches in manual sources, end users may begin with just one feature of a broader topic, or just one relevant reference, and move through a variety of sources. Each new piece of information they encounter gives them new ideas and directions to follow and, consequently, a new conception of the query. At each stage they are not just modifying the search terms used in order to get a better match for a single query. Rather the query itself (as well as the search terms used) is continually shifting, in part or whole. This type of search is here called an evolving search.
A major purpose of using an information system is to gain more knowledge, which makes you better at using the information system to gain more knowledge… and so on, in an iterative process.
I certainly recognise this behaviour in the way I use searches to understand a new topic. My first searches are inevitably broad and naive. But the bits of information I pick up from those initial results help me form a better search query. This process continues until I feel satisfied that I have gained enough knowledge for now.
Fast answers hinder learning
For research like this, having a fast answer misses the point. To gain new knowledge, we must go through an active process of finding and interpreting information iteratively.
From this perspective, answer engines completely miss this point.
That is not to say that answer engines do not have a place. But we need to be clear about what their place is.
Fast answers are made up of ultraprocessed information
I’ve written before about the concept of a spectrum of information processing. Relatively unprocessed information is more like data. Relatively more processed information is more like content (where interesting or significant data points are packaged up and contextualised in formats like articles and infographics).
Because of the way large language models work, you cannot be guaranteed that the answers they give you is fully accurate. They are probabilistic, which is the artificial intelligence industry’s way of saying you cannot guarantee they will be right. They are based on linguistic pattern-matching, rather than semantic accuracy.
For this reason, I think of the output generated by large language models as ultraprocessed information.
Fast answers are like fast food
In the same way that a bad diet containing lots of ultraprocessed food might make you malnourished, a bad information diet can make you malinformed.
Just as with food, you don’t necessarily need to cut out the convenient options entirely. But it’s about knowing when you should best use an artificial intelligence tool as part of a balanced information diet.
Current AI tools can cause a malinformation spiral
One of the biggest issues with artificial intelligence tools appears to be around one of their apparent appeals — their open nature. Users start with a blank screen and complete freedom to type whatever they want.
This is in contrast to traditional information environments. In a browsable system, users are given a set of navigation options or other information facets that communicate the constraints and boundaries of possibilities. Meanwhile, in traditional search environments (like web search engines), you will rapidly obtain information that helps you refine your query — this is berrypicking in action.
But large language models tend to be sycophantic towards the user. They won’t faithfully show you the boundaries. They won’t necessarily correct your terminology. They might not inform you of an error in your prompt.
This all makes artificial intelligence users unable to accurately refine their prompt. The sycophancy of LLMs creates a barrier to effective berrypicking.
Instead, the naive user and the sycophantic AI tool lead each other down a merry malinformation spiral. This causes hallucinations to build over time, and responses become increasingly chaotic.
Instead of giving people answers, AI tools should ask people questions
More to the point, giving people an answer is not how people learn. People learn by doing, by critically engaging with a subject matter, and by asking questions among their peers.
We can even go back to Socratic reasoning. Socrates felt that the only way for a student to gain knowledge was to ask them probing questions. Teachers still rely on this technique today as a powerful learning tool. Giving people “the answer” is counterproductive if you want people to truly learn and gain knowledge.
So instead of pushing answer engines, we should perhaps pursue the development of question engines.
Large language models currently tend to tell the user: “you’re absolutely right”, no matter what.
What if artificial intelligence tools instead responded with a Socratic question that would help the user make their next question better?
Reposts