Can humans govern our future?
· 11 min read
Admitting that we don't know or can't remember something is a deeply human experience, one that reflects both the natural limits of our cognitive abilities and the potential for growth through effortful engagement. However, excessive cognitive offloading—the habitual reliance on external tools to manage tasks such as memory or problem solving—threatens to erode this essential connection to our mental faculties. The consequences can be likened to the early stages of dementia, when people become painfully aware of their declining cognitive abilities. Research shows that people with dementia often experience frustration, anxiety, and a profound sense of loss as they recognize their growing dependence on external aids or others for tasks they once performed independently (Clare et al., 2004).
Cognitive offloading, the act of delegating mental tasks to external tools or systems, has become increasingly embedded in modern life. While this practice enhances immediate task efficiency and reduces cognitive strain, research reveals its potential long-term drawbacks, a phenomenon I provocatively term “dummyfication”. This notion finds empirical support in studies that highlight how habitual reliance on external aids can impair memory retention, problem-solving capabilities, and cognitive independence.
Evidence underscores that cognitive offloading compromises memory retention. For instance, Grinschgl et al. (2021) demonstrated that participants who relied on external tools in the Pattern Copy Task (a cognitive task designed to study the trade-offs between internal memory use and cognitive offloading) exhibited poorer memory for offloaded information, even when aware of subsequent memory tests. This suggests that offloading reduces the mental engagement necessary for robust memory formation. Similarly, Sparrow et al. (2011) found that individuals relying on search engines remembered less about the content itself but retained knowledge of where to find it, indicating a shift from internal to external memory reliance.
Offloading also appears to hinder problem-solving skills. O’Hara and Payne (1998) showed that participants using external aids performed worse in transfer tasks requiring independent problem-solving. This suggests that offloading diminishes opportunities for active learning and skill acquisition. Furthermore, Wahn et al. (2023) observed that while humans are willing to offload demanding tasks to algorithms under high cognitive load, this behaviour fosters dependency, potentially eroding their ability to perform similar tasks autonomously.
The immediate benefits of cognitive offloading—speed and accuracy—often come at the expense of deep learning. Research highlights that avoiding offloading can create "desirable difficulties," which enhance learning by forcing individuals to engage more deeply with material (Bjork & Bjork, 2011). Conversely, habitual offloading bypasses these challenges, leading to superficial processing and weaker retention.
The phenomenon of cognitive offloading—where humans delegate mental tasks to external tools like smartphones or algorithms—provides a microcosm of the broader transformations humanity faces as it integrates with advanced technologies. While cognitive offloading offers immediate benefits, such as enhanced efficiency, it risks long-term cognitive erosion, a phenomenon provocatively termed “dummyfication". This trade-off becomes even more critical when viewed against the backdrop of emerging Digital Minds and Artificial General Intelligence (AGI).
Digital Minds, as envisioned by thinkers like Nick Bostrom, represent a new frontier in intelligence. These artificial entities, potentially capable of sentience or sapience, could surpass human cognitive abilities in speed, accuracy, and adaptability. Unlike humans, whose mental faculties are constrained by biology and susceptible to decline through excessive offloading, digital minds operate with perfect recall and computational precision (Bostrom & Schulman, 2023). This disparity raises profound questions about how humans will coexist with—and possibly compete against—digital minds.
As humanity increasingly relies on advanced technology to enhance its capabilities, transhumanism emerges as a critical pathway for adapting to a future shared with digital minds. Transhumanism—the pursuit of transcending biological limitations through technological augmentation (More, 2010)—offers tools such as brain-computer interfaces, genetic editing, and cognitive enhancements that could enable humans to remain competitive or complementary to Artificial General Intelligence (AGI) and digital minds. These technologies represent humanity's effort to evolve alongside its creations, ensuring relevance in an era of rapidly advancing machine intelligence.
The phenomenon of cognitive offloading provides a microcosm of this broader transformation. Just as individuals delegate memory or problem-solving tasks to external tools like smartphones, transhumanist technologies extend human capabilities by integrating external systems into our cognitive and physical processes. This augmentation is not merely about convenience; it is becoming essential for humans to keep pace with increasingly intelligent systems. Without such enhancements, humans may risk falling behind in decision-making, creativity, and innovation—a gap that could widen as AGI systems surpass human cognitive capabilities.
While transhumanism provides a path to coexistence with digital minds, the singularity represents a more profound and transformative point in the future. The singularity, as envisioned by Ray Kurzweil, is a moment when technological progress accelerates so rapidly—driven by self-improving AGI—that it fundamentally reshapes human civilization. At this stage, the boundaries between transhuman intelligence and machine intelligence blur entirely, creating a new era where human and digital minds may integrate seamlessly (Kurzweil, 2010).
This integration raises profound questions about identity and agency. If humans merge with machines to such an extent that distinctions between biological and artificial intelligence disappear, what does it mean to be human?
According to my dear friend Dr. Olaf Hermans, part of being human is staying anonymous and autonomous. That entails an in-crowd that is exclusively privy to one’s quirky behavior that can encompass randomness, degrees of schizophrenia, and unpredictability.
The singularity envisions a future where enhanced humans and digital minds coexist not merely as separate entities but as interconnected intelligences shaping a shared reality. While this transformation holds extraordinary promise—such as solving global challenges or achieving radical longevity—it also presents existential risks if misaligned AGI systems act contrary to human values.
Even today, technologies like digital twins provide a glimpse into the future interaction between humans and advanced AI systems. Digital twins—virtual replicas of individuals—are already used in sectors such as healthcare, fitness, and workplace productivity. These virtual counterparts simulate human behaviour and decision-making, raising questions about autonomy and identity. For instance, if a digital twin can predict or act on behalf of its human counterpart, does it become an extension of the individual or something distinct? If your digital twin passes the Reverse Turing test—a human must convince a computer that they are human, did a human pass it? In popular culture, the phrase “passed the Turing test” has come to mean that AI can pass as a human. We might be needing creative interpretations of the Turing test such as a human having to prove someone else is human too, or a machine having to prove it has no consciousness and is no human.
Digital twins also foreshadow the broader implications of digital minds. As these systems become more sophisticated, they may evolve from tools into entities with moral or political status. This aligns with Nick Bostrom's "substrate-independence thesis," which suggests that consciousness is not limited to biological neural networks but could also arise in silicon-based systems. The emergence of digital minds capable of sentience or sapience would challenge humanity to navigate new ethical frameworks for coexistence.
The societal impact of these dynamics is significant. If vast swaths of the population suffer from the negative effects of cognitive offloading, such as diminished memory and problem-solving skills, they may struggle to keep pace with digital minds in areas ranging from decision-making to innovation. Furthermore, without proper mitigation, this reliance on external aids inevitably leads to a loss of cognitive autonomy, mirroring the emotional toll experienced by individuals in the early stages of dementia who become painfully aware of their declining abilities.
In this context, human-digital mind interaction might be likened to an interspecies relationship. Bostrom’s "substrate-independence thesis" suggests that consciousness is not limited to biological neural networks; digital systems could achieve levels of cognition comparable to—or even exceeding—those of humans. If these digital minds are recognized as distinct entities with moral or political status, humanity will face unprecedented challenges in governing their integration into society.
To navigate this future effectively, it is essential to address both the risks of excessive cognitive offloading and the ethical implications of creating and interacting with digital minds. Strategies such as fostering deep learning practices, enhancing human cognitive resilience through transhumanist technologies, and developing equitable governance frameworks for digital minds will be crucial. Ultimately, humanity's ability to thrive alongside digital minds will depend on its capacity to balance technological augmentation with the preservation of intrinsic cognitive abilities and ethical integrity.
This raises critical questions: as we increasingly depend on AI systems for decision-making, creativity, and problem-solving, could humanity risk a similar "dummyfication" of its collective intelligence and subsequently be outperformed and outcompeted by the very technologies it created
What happens when humanity begins to transcend its biological limits? This is no longer a question of science fiction but a pressing reality. Transhumanism—the pursuit of enhancing human capabilities through technology—and Artificial General Intelligence (AGI)—AI systems capable of human-level cognition—are poised to reshape every aspect of human life. These advancements promise unprecedented benefits: eradicating diseases, extending lifespans, and solving complex global challenges. Yet they also raise profound ethical, societal, and existential questions that demand urgent attention.
Transhumanism is not a distant dream but an emerging reality. Technologies like Elon Musk’s Neuralink are enabling direct communication between human brains and computers (Musk, 2019), while CRISPR gene editing is treating genetic disorders with unprecedented precision. Advanced bionic prosthetics are restoring mobility through brain-machine interfaces, and microchip implants are enhancing everyday convenience by merging technology with the human body (Doudna & Sternberg, 2018). These innovations demonstrate that humanity is already on the path to transcending biological limitations.
Similarly, progress in AGI research has sparked debates about machines surpassing human intelligence and their implications for employment, privacy, and autonomy. While true AGI—machines capable of human-like reasoning and learning—has not yet been achieved, significant advancements are narrowing the gap. Large language models like OpenAI’s GPT-4 and multimodal systems that integrate text, images, and audio demonstrate increasingly versatile capabilities. Projects such as DeepMind’s AlphaFold, which solved a decades-old protein-folding problem (Jumper et al., 2021), and SingularityNET’s cognitive computing network are pushing the boundaries of AI's problem-solving potential (Sovereign Magazine, 2024). Experts remain divided on when AGI will emerge, with some predicting breakthroughs within the next decade. According to OpenAI’s Sam Altman in late 2024, AGI could debut as early as 2025 (Pillay, 2025).
Thinkers like Elon Musk and Ben Goertzel have long predicted that AGI will play a pivotal role in advancing transhumanist goals, such as cognitive enhancement and solving global challenges. Musk envisions a future where humans merge with AI to stay relevant, while Goertzel foresees AGI accelerating breakthroughs in biotechnology and neural augmentation (Goertzel, 2014; Goertzel, 2016). These predictions underscore how the convergence of transhumanism and AGI magnifies their transformative potential—and their risks.
However, this rapid progress raises profound questions about governance, ethics, and societal impact. Geoffrey Hinton, the "godfather of AI," recently warned that there is a significant chance AI could wipe out humanity within the next 30 years if its development is not carefully controlled (Milmo, 2024). His stark prediction highlights the existential stakes involved in unregulated technological advancement. When combined with transhumanist technologies, AGI magnifies both transformative potential and existential risks. History provides ample evidence of humanity’s struggle to govern disruptive technologies. From nuclear weapons to social media platforms, our collective track record reveals both moments of cooperation and tendencies toward short-termism, inequality, and conflict.
Transhumanism and AGI amplify these challenges because their impacts will be pervasive and multifaceted, affecting individuals, societies, and global power dynamics in ways we are only beginning to comprehend.
The question is not only can humanity govern the convergence of transhumanism and AGI but also why must it do so? Without deliberate oversight, these technologies could exacerbate existing inequalities, destabilize societies, or even threaten humanity’s survival. For instance, elite capture of enhancements like genetic editing or cognitive augmentation could deepen socioeconomic divides, creating a world of “haves” and “have-nots.” Similarly, misaligned AGI systems could act in ways harmful to humanity if their objectives diverge from shared values—a challenge known as the alignment problem. Governance is essential not only to mitigate these risks but also to ensure that these technologies align with ethical principles such as equity, dignity, sustainability, and collective welfare. By proactively shaping their trajectory through robust frameworks for transparency and accountability, humanity can guide these advancements toward outcomes that benefit all rather than a privileged few.
Ultimately, this is not just a technological issue but a deeply human one, although we seem to count on tech to solve it. The decisions made today will shape the kind of future we leave for generations to come. By addressing this question with urgency and nuance, we may yet harness the potential of transhumanism and AGI to build a world that reflects our highest ideals rather than our greatest fears. Let’s not be “dummyfied” into submission by the tech we create. In fact, let’s not be dummyfied at all.
illuminem Voices is a democratic space presenting the thoughts and opinions of leading Energy & Sustainability writers, their opinions do not necessarily represent those of illuminem.
illuminem briefings
AI · Corporate Governance
illuminem briefings
Corporate Governance · Adaptation
illuminem briefings
Net Zero · Carbon Market
Hydrogen Council
Hydrogen · Corporate Governance
World Economic Forum
Corporate Governance · Corporate Sustainability
CNBC
AI · Corporate Governance