A dark future worth looking at

“Their knowledge was more in the realm of knowing how to ask machines to do things than in knowing how to do things themselves.”

I picked up Ken Liu’s All That We See or Seem on the recommendation of Anand Mahadevan, Executive Director of CAIS. He suggested it as a way to think more deeply about the darker possible futures of generative AI—not through another research report, policy paper, or breathless prediction, but through fiction.

It was an excellent recommendation.

The book is part of Liu’s Julia Z. series, and it works as a technological thriller. But what held my attention was not simply the plot. It was the world Liu builds around it: a world in which artificial intelligence has become so embedded in communication, commerce, creativity, security, and everyday life that people can no longer be entirely sure what is real, who is speaking, what they own, or whether they still understand the systems governing their lives.

The book takes some of the signals already around us—AI-generated content, surveillance capitalism, technological monopolies, misinformation, declining trust, outsourced thinking—and stretches them forward.

That is what makes it unsettling. It is also what makes it useful.

You would be interested in this book if…

• You are thinking about the future of generative AI in schools or society.
• You are concerned about declining trust in online information.
• You are interested in AI literacy, media literacy, and technological agency.
• You believe fiction can sometimes reveal a future more powerfully than a policy report.
• You want to explore what might happen if we continue outsourcing judgment, creativity, and cognitive struggle to machines.

Dark futures are not predictions

@sarahsseaofpages

All That We See or Seem by Ken Liu! Thank you for reading with me @@ literarymadmax everywhere 🍉 🫶🏼 @Saga Press Books 🛸🧙🏿‍♀️🔪 #allthatweseeorseem #authorkenliu #newbookrelease #bookreview #sagasayscrew

♬ original sound – sarahsseaofpages

This book builds nicely on something I took from Nicholas Badminton’s work: we should not only imagine the future we hope to create. We should also spend time exploring the futures we most want to avoid.

A dark future is not necessarily a prediction. It is a provocation. It allows us to take the trends, behaviours, incentives, and technologies already emerging and ask:

Key Questions: What happens if we keep going?

(1) What happens if we become increasingly comfortable allowing machines to think, create, filter, recommend, remember, and decide for us?
(2) What happens if we stop understanding the technologies we rely upon?
(3) What happens when convenience becomes more important than agency?
(4) And what happens when the internet becomes so saturated with synthetic content that being human becomes a market advantage?

Liu does not answer these questions through an argument. He builds a world in which the answers have already begun to emerge.

“Everyone knows the internet is dead”

One of the most haunting moments comes when a character describes loneliness as “the quintessential condition of modernity.”

They eat alone. Sleep alone. Do not know their neighbours. Their relationships with coworkers disappear outside the office. Even calling a parent feels like an obligation. Then comes the line:

“Everyone knows the internet is dead. It’s only bots talking to bots now.”

The internet was once imagined as a vast network of human connection. In Liu’s future, it has become a largely synthetic space populated by automated accounts, scammers, trolls, and machines producing content for other machines to process.

The tragedy is not only that the technology has become artificial. It is that people have become profoundly lonely within it. As Marshall McLuhan encourages us through his 4 Laws of Media – this is a look at how the Internet has reversed upon itself!

This feels like an important warning for schools. When we discuss AI, we can become overly focused on productivity, academic integrity, assessment, or employability. Those things matter. But there is also a much more human question:

What happens to connection when more and more communication is mediated, generated, filtered, and personalized by machines?

The book’s future is technologically connected and emotionally isolated. That is not difficult to imagine.

You do not really own the device

The book also explores the relationship between technology, power, and control through a fictional company called Bloom.

The devices people purchase are not really theirs. Users are effectively renting the right to operate them according to the manufacturer’s rules.

A phone may refuse to capture certain images. A photograph of a military installation may be blocked or even altered through AI. A laptop may prevent the installation of software deemed to spread “disinformation”—a label defined by those who hold institutional and corporate power.

The concern is about agency, not just privacy.

When a small number of technology companies control the hardware, operating systems, platforms, information flows, and increasingly the AI models themselves, they gain the ability to decide what users may see, record, install, share, or create.

Liu takes current concerns about technology monopolies and extends them into a future where censorship does not always look like censorship. Sometimes it looks like a device that simply refuses to obey.

This is one reason technological literacy matters. It is not enough to know how to operate a platform. We need to understand what the platform is doing, who controls it, what incentives shape it, and what forms of choice we may have quietly surrendered.

The surveillance panopticon

Another thread running through the novel is the near-total surveillance enabled by ordinary devices.

In one scene, phones are described as constantly listening—not simply to support AI assistants, but to capture sounds beyond the human audible range. Ultrasonic signals embedded in advertisements can be detected by devices, allowing companies to measure exposure and continue tracking people beyond conventional cellular networks.

The line that lands is:

“God forbid that anyone should be beyond the gaze of the surveillance panopticon for even just a second.”

Liu’s future is one where every device is simultaneously a tool and a sensor.

It serves the user, but it also observes the user. It listens, records, predicts, shapes, and reports. Again, this is not entirely fiction. The power of speculative storytelling is that it takes familiar practices and removes the reassuring language surrounding them. Personalized advertising becomes surveillance. Convenience becomes behavioural monitoring. An AI assistant becomes a microphone that never stops listening.

The book asks us to consider what privacy will mean when technologies are designed to gather information continuously—and when opting out may mean being unable to participate fully in society.

Knowing how to ask is not the same as knowing

For educators, one of the book’s most important passages concerns machine-assisted developers who have become so disconnected from the physical technology that they could not identify the chips for which they were supposedly writing software. They know how to ask machines to do things.

They no longer know how to do the things themselves.

Liu acknowledges that this distinction is philosophically complicated. Human beings have always relied on external tools and sources of knowledge. Knowing where to find an answer has long been part of knowing.

But the novel draws a meaningful distinction between using technology to extend understanding and using it to avoid understanding.

There is, the character explains, a responsible way to develop an AI: allow it to reflect only the things you have made the effort to understand. Let it become an extension of your intellectual journey.

Then there is the shortcut:

“To let the machine do all the work while you take the credit.”

That is the danger I keep returning to in conversations about generative AI.

The concern is not simply that students—or adults—will use AI to produce something. The greater concern is that they may gradually lose the ability to recognize when the output is wrong, shallow, manipulative, or disconnected from reality.

Prompting is a skill. It is not the same as judgment. And fluency is not the same as understanding.

When humanity becomes the loophole

The most disturbing idea arrives through the novel’s villain, known as the Prince.

The Prince operates a content farm, but not one run entirely by bots. His operation uses real people: migrants, runaways, the chronically underemployed, people experiencing mental illness, and others living at the edges of society.

They spend their days liking, posting, reviewing, commenting, and manipulating social media platforms.

Why use humans when machines could generate vastly more content?

Because the platforms have developed AI systems—referred to as Blade Runners—that identify and remove machine-generated material. Synthetic content is filtered out. Human behaviour is allowed through.

The Prince’s solution is horrifyingly clever: use exploited humans to create content with just enough randomness, creativity, and authenticity to bypass the machines trained to detect artificiality.

In his mind, he is “putting back the human element.”

Human beings become the mechanism through which automated systems are manipulated. Human authenticity becomes an industrial input.

This may be the darkest idea in the book because it reverses the usual conversation. We often worry that machines will replace people. Liu imagines a world in which people become valuable precisely because machines cannot perfectly replicate them—and that value becomes another reason to exploit them.

The Prince calls it innovation. 

Compared with a bot farm, his operation produces far less content. But because the content is authentically human, it is vastly more influential. What a coopting of the word ‘innovation’!

It is a stunning extension of today’s attention economy. When platforms reward engagement, someone will always find a way to manufacture it. When systems learn to block artificial content, someone will find people desperate enough to produce the human equivalent.

In this future, the human element has been commodified ~ not protected.

Julia Z. and the importance of remaining literate

What makes Julia Z. compelling is not simply her ability to use technology. It is her determination to understand it.

She maintains her knowledge of computers, hardware, networks, code, and systems. She does not accept that the technology around her is too complicated to question. Her literacy gives her a degree of agency that most people in this imagined world have surrendered.

This is where the novel becomes particularly relevant to education.

AI literacy cannot mean merely learning how to use AI tools. It must include understanding how they work, where they fail, who controls them, what data shapes them, and how they influence what we see and believe.

Technology literacy matters.

Media literacy matters.

Critical thinking matters.

Judgment matters.

And increasingly, the willingness to engage in cognitive struggle matters.

The struggle to understand something is not an inefficient obstacle to learning. Often, it is the learning.

When we outsource that struggle too quickly, we may still receive the answer. But we lose the intellectual architecture needed to evaluate it.

The future is still ours to influence

All That We See or Seem does not argue that technology is inherently destructive. Nor does it suggest that we can or should turn away from artificial intelligence.

Instead, it shows what may happen when powerful technologies are combined with familiar human weaknesses: greed, loneliness, convenience, inequality, the pursuit of influence, and the temptation to avoid difficult thought.

This is why dark futures are worth exploring.

They allow us to identify the signals before they become systems.

They help us ask better questions about decisions that may otherwise appear small or harmless.

They remind us that the future does not arrive fully formed. It is built through the choices, incentives, permissions, and habits we establish now.

Ken Liu’s novel is more like a warning flare than a roadmap… don’t expect answers.

It asks what may become possible if we allow machines to replace too much of our thinking, companies to control too much of our technology, platforms to mediate too much of our reality, and convenience to erode too much of our agency.

The future portrayed in this book is dark. But perhaps that is precisely why we need to look at it. Because once we can see the future we do not want, we may become more intentional about building the one we do.