The institutional trap: Why governments and organisations struggle to manage complex risk
· 17 min read
This is article 4 in The Complexity Paradox series. Here is article 3.
Institutions exist because complexity must be organised before it can be governed. Governments divide responsibilities among ministries, departments, agencies, regulators, commissions, and local authorities. Companies do something similar through finance, operations, technology, legal, compliance, procurement, human resources, cybersecurity, sustainability, risk, and strategy. These divisions make complex systems manageable because no individual or institution can process every relevant variable at once. Problems are separated into categories, responsibilities are assigned, procedures are developed, expertise is concentrated, and accountability is distributed across organisational structures.
This architecture is necessary. A government cannot operate as a single undifferentiated body in which every official is simultaneously responsible for public health, infrastructure, defence, agriculture, financial regulation, energy, technology, education, and the environment. An enterprise cannot realistically expect the same team to manage accounting, logistics, information technology, human capital, legal exposure, operational risk, and strategic planning. Institutions simplify reality because some form of simplification is required for coordinated action.
The difficulty is that reality does not organise itself according to institutional charts. Climate affects water availability, water influences agriculture, public health, hydropower, industrial production, urban growth, and ecosystem stability, and electricity affects communications, financial systems, transportation, hospitals, water treatment, industry, and digital infrastructure. Employment influences household stability, fiscal revenues, consumption, migration, and political confidence, while cybersecurity touches finance, infrastructure, national security, health systems, manufacturing, government administration, and communications. A disruption that appears to belong to one department can quickly move through several others.
Institutions divide problems in order to manage them, but complex risk travels across the boundaries those institutions create. The result is an institutional paradox: the organisational structures designed to make complexity governable can also make systemic risk harder to see. The problem is not that governments and organisations lack intelligence, expertise, or effort. In many cases, the opposite is true. Modern institutions contain enormous concentrations of technical knowledge. Governments employ economists, engineers, scientists, diplomats, planners, lawyers, public-health specialists, security professionals, statisticians, and regulators. Large enterprises employ similarly sophisticated teams. Yet expertise itself is usually organised into domains, and those domains can become intellectual and operational boundaries.
The deeper challenge is therefore not simply whether institutions possess information, but whether they can connect information quickly enough to understand how risks interact across the system.
The bureaucratic organisation of modern society emerged because specialisation works. Max Weber's analysis of bureaucracy emphasised rational administration, clearly defined responsibilities, hierarchy, rules, technical competence, and predictable procedures. Compared with governance based on personal authority or informal relationships, bureaucratic systems could administer increasingly large and complicated societies with greater consistency.
This represented an extraordinary advance in institutional capacity. Tax systems could be standardised, infrastructure could be planned, laws could be implemented across large territories, and public services could operate according to established procedures rather than the discretion of individual officials. Large organisations could coordinate thousands, and eventually millions, of people.
Specialisation was central to this achievement. Complex tasks were divided so that expertise could develop within particular domains. A transportation ministry could focus on transportation, a central bank could specialise in monetary and financial stability, a health ministry could focus on health systems, and environmental agencies could develop scientific and regulatory expertise that other parts of government would struggle to reproduce. Organisations followed the same logic. Departments allowed expertise to deepen, responsibilities to become clearer, and decisions to move through defined channels. The modern corporation is possible partly because work can be separated into specialised functions while still contributing to a larger organisational objective.
But specialisation creates a trade-off. The deeper expertise becomes within a particular domain, the easier it can become to interpret problems primarily through that domain's language, incentives, data, and priorities. An environmental agency may view drought primarily through water availability and ecosystem impacts. An agricultural ministry may view the same drought through crop yields and farmer income. An energy ministry may focus on hydropower generation or cooling requirements for thermal plants. A finance ministry may focus on food inflation, fiscal support, and economic growth, while a national-security institution may examine migration or political instability.
Each perspective can be correct, yet the systemic risk exists in the interaction among them. No single institution may be wrong, and the overall system can still misunderstand the problem if the relationships between those perspectives remain weak.
Herbert Simon's concept of bounded rationality helps explain why this problem cannot simply be solved by asking decision-makers to consider everything. Human beings do not possess unlimited information, time, attention, or computational capacity, so we make decisions within constraints and use simplified models of reality. Institutions do the same.
Governments cannot analyse every possible relationship before acting. Businesses cannot model every conceivable future condition. Regulators cannot evaluate infinite combinations of risk. Decisions must be made using available information, established procedures, professional judgement, and assumptions about what matters most. Simon argued that decision-makers often "satisfice" rather than optimise, meaning they search for solutions that are good enough under the circumstances because identifying the theoretically optimal solution may be impossible.
This is not necessarily a failure. It is often the only realistic way to operate. The problem becomes more serious as systems grow more interconnected. A simplified model may work well when the problem is relatively contained, but become increasingly misleading when outcomes depend on interactions across multiple sectors.
A city planning new housing, for example, cannot think only about the number of residential units required. Housing changes transportation demand, water consumption, electricity use, school capacity, waste management, emergency services, land values, tax revenues, and infrastructure requirements. Those effects may then influence regional development patterns and future public spending. The planning agency may understand housing perfectly and still produce unintended consequences because housing is not only a housing issue.
Complex risk repeatedly creates this problem. Every institution must simplify reality to function, but those simplifications can conceal the relationships that matter most. The danger grows when organisations begin treating their administrative categories as if they were properties of the real world rather than tools created to help manage it.
The organisational silo is perhaps the most visible manifestation of the institutional trap. Silos form when departments develop their own priorities, information systems, cultures, terminology, budgets, incentives, and decision-making processes. Over time, these structures can become highly effective within their own domains while becoming increasingly disconnected from one another.
In government, this can produce fragmented policy. A climate strategy may be developed separately from industrial policy. Infrastructure investment may be planned separately from demographic change. Cybersecurity may be treated as a technical matter even though a digital attack could affect hospitals, transportation, financial services, government operations, and energy infrastructure simultaneously.
Businesses face the same problem. Enterprise risk may be divided among departments that rarely share a complete view of exposure. Cybersecurity sits within information technology, supply-chain risk belongs to procurement or operations, financial exposure belongs to finance, legal risk belongs to counsel, sustainability risk may sit within ESG or corporate responsibility, and geopolitical risk may be handled by strategy, government affairs, or security.
Yet a single event can move through all of them. Imagine a severe weather event disrupting a manufacturing region. Initially, it may appear to be a physical climate risk. If facilities close, it becomes an operational problem. If suppliers cannot deliver components, it becomes a supply-chain problem. If production falls, it becomes a revenue issue. If contracts cannot be fulfilled, it becomes a legal issue. If customers leave, it becomes a strategic problem, and if investors react negatively, it becomes a financial problem.
The organisation may possess teams responsible for every one of these risks and still fail to understand the total exposure because no one sees the entire chain. The central challenge of systemic risk is therefore not always missing information. Often, the information exists but is distributed across an organisation in ways that prevent it from becoming a coherent picture.
Institutions are also shaped by their own histories. Path dependence describes the way earlier decisions influence future choices, even when the original conditions that produced those decisions have changed. Once organisations build procedures, regulations, technologies, professional cultures, and infrastructure around a particular model, changing direction can become difficult.
This creates institutional inertia. A government agency created to solve one category of problem may continue operating according to its original mandate even as the nature of the problem evolves. Regulations designed for an earlier technological environment may remain in place after markets change, and organisational structures created during periods of stability may prove poorly suited to rapidly evolving risks.
The reason is not simply resistance to change. Institutions accumulate investments. Employees are trained around existing procedures, information systems are built around current categories, budgets reflect established priorities, laws define authority, professional expertise develops around particular models, careers become associated with existing structures, and political constituencies form around programmes. Changing the structure therefore involves costs.
The longer an institution operates, the more deeply embedded these arrangements can become. Past solutions become part of the operating environment, creating another complexity paradox in which institutions designed to solve earlier problems become part of the complexity facing future decision-makers.
A new policy rarely begins with a blank sheet of paper. It must interact with existing regulations, agencies, programmes, contracts, legal obligations, budgets, technologies, and political interests. Each new layer can address a legitimate problem while simultaneously increasing the complexity of the system. Over time, institutional capacity can become consumed by managing the accumulation of previous solutions.
Regulation is essential to modern economies. Without rules governing finance, environmental protection, health and safety, competition, consumer rights, infrastructure, and corporate behaviour, complex markets could generate enormous social costs. Yet regulation itself can become complex.
Rules accumulate because risks emerge incrementally. A failure occurs and new safeguards are introduced. Technology changes and new standards are created. Political priorities shift and additional reporting requirements appear. International obligations add another layer. Local, regional, national, and transnational authorities may all regulate overlapping dimensions of the same activity.
Each individual rule may be rational, while the combined system becomes difficult to understand. Organisations respond by creating compliance teams, reporting systems, audits, internal controls, documentation processes, and legal reviews. Governments create agencies to administer and enforce those requirements. Businesses hire specialists to interpret them.
A feedback loop can develop in which complexity generates regulation, regulation generates organisational complexity, organisational complexity creates new coordination problems, and those problems generate additional procedures. The result is not necessarily better governance. A highly complicated regulatory environment can sometimes obscure the purpose of regulation itself.
Organisations may become focused on demonstrating procedural compliance rather than understanding underlying risk. The question can shift from whether the institution is resilient to whether the required documentation has been completed. Those are not the same thing. Compliance can support resilience, but compliance is not resilience, and an organisation can satisfy formal requirements while remaining exposed to systemic risks that existing rules do not capture.
Regulation tends to respond to known categories, while complex systems continually generate interactions that do not fit neatly within those categories. This creates another institutional challenge: rules are often optimised for recognisable risks, while systemic failures emerge through relationships that no single rule was designed to address.
Complex institutions are also affected by information asymmetry. Those making decisions often depend on information produced by people closer to operations, while those closer to operations may not understand the strategic context in which decisions are being made. Information moves upward through layers of management, reporting, interpretation, and filtering, and at each stage some detail can be lost.
Senior officials cannot read every technical report. Executives cannot personally inspect every facility or supplier. Boards cannot examine every operational decision. Information must therefore be summarised, but summarisation is itself a form of simplification.
Important context can disappear when complex realities are reduced to dashboards, risk ratings, traffic-light indicators, or executive summaries. A risk reported as moderate may contain assumptions that senior leaders never see. A project reported as on track may face emerging vulnerabilities that do not yet meet formal escalation thresholds. Organisations therefore confront a fundamental information problem: leaders need simplified information because they cannot process everything, but excessive simplification can hide the very complexity they need to understand.
The opposite problem also exists. Decision-makers can receive so much information that meaningful signals become difficult to distinguish from noise. Modern organisations produce enormous quantities of data, reports, metrics, assessments, audits, forecasts, and alerts. More information does not automatically produce better understanding.
The real challenge is interpretation. Institutional intelligence depends not simply on collecting data, but on identifying relationships, patterns, dependencies, and emerging changes before they become crises.
Another difficulty arises from principal-agent problems. In organisations, one group often delegates responsibility to another. Citizens delegate authority to governments, legislatures delegate implementation to agencies, executives delegate responsibilities to managers, and boards delegate operational control to leadership teams.
The interests of principals and agents are not always perfectly aligned. A department may protect its budget even when organisational priorities shift. Managers may avoid reporting problems that could reflect poorly on performance. Regulators may focus on measurable compliance because systemic outcomes are more difficult to evaluate. Political leaders may favour visible short-term achievements over investments in resilience whose value may not become apparent for years.
These incentive structures matter because systemic risk frequently requires action before benefits are visible. A redundant supply route may appear unnecessary until the primary route fails. Backup infrastructure may look expensive when it is rarely used. Preventive maintenance may attract less political attention than building something new. Institutional capacity building may generate few immediate headlines.
Resilience suffers from a visibility problem because success often means that nothing dramatic happens. This can make it politically and organisationally difficult to sustain investment in prevention, redundancy, preparedness, and long-term capability. Institutions therefore tend to respond more easily to visible crises than to invisible vulnerabilities.
After a failure, resources become available, committees are created, rules are changed, and funding is approved. Before failure, the same risks may struggle to compete with more immediate priorities.
If silos create fragmentation, coordination is the obvious solution, yet coordination itself introduces complexity. Cross-agency committees, interdepartmental working groups, task forces, integrated planning processes, and joint reporting mechanisms are often created to connect institutional boundaries. These can improve information sharing and reduce fragmentation, but every coordination mechanism also requires time, authority, procedures, meetings, data, and decision rules.
As more institutions become involved, the number of relationships that must be managed grows rapidly. A problem involving two agencies requires relatively limited coordination, while a problem involving ten institutions introduces a much larger web of interactions. Add local governments, private companies, civil society, international organisations, and multiple regulatory jurisdictions, and coordination becomes a major governance challenge of its own.
This is one reason cross-cutting risks are so difficult to manage. Everyone may agree that coordination is necessary while disagreeing over who leads, who pays, whose data is authoritative, which objectives take priority, and who is accountable when outcomes fail.
Climate adaptation illustrates this problem. Effective adaptation may involve water authorities, agriculture departments, energy planners, transportation agencies, municipalities, disaster management authorities, environmental regulators, health systems, financial institutions, and private infrastructure operators. No single institution controls the entire system, and the same is true for cybersecurity, food security, pandemic preparedness, critical minerals, supply chains, and infrastructure resilience.
Modern risk increasingly demands governance without centralised control.
Private organisations often reproduce the same institutional weaknesses as governments. Many companies still manage risk through categories that correspond to organisational departments rather than the actual structure of exposure. Financial risk belongs to finance, technology risk belongs to IT, operational risk belongs to operations, sustainability risk belongs to ESG, and regulatory risk belongs to compliance.
This is administratively convenient, but potentially misleading. A company can have excellent departments and still possess weak systemic awareness.
Consider dependence on a critical software platform. Information technology may understand the technical architecture, procurement may understand the contract, finance may understand the cost, legal may understand liability, cybersecurity may understand vulnerabilities, and operations may understand what would happen if the platform failed. Unless those perspectives are connected, however, no one may fully understand how important that platform is to the organisation as a whole.
The same problem appears in supply chains. Procurement may know direct suppliers, sustainability teams may understand environmental exposure, geopolitical analysts may track instability, finance may monitor commodity prices, and operations may understand production requirements. The systemic risk lies in the intersection.
This is why enterprise risk management increasingly needs to move beyond registers of individual risks toward maps of relationships and dependencies. Organisations do not fail because risk categories exist. They fail when risks interact in ways those categories do not capture.
At a certain point, the complexity of the institution itself can become a source of vulnerability. Large organisations accumulate structures to solve problems. New risks lead to new committees. New regulations create new compliance functions. New technologies create new specialists. New strategies create new reporting requirements. New crises create new procedures.
Each addition may be individually reasonable, but collectively they can slow decision-making. Officials spend more time navigating internal processes, managers struggle to determine who has authority, multiple departments may need to approve relatively simple actions, information is duplicated across reporting systems, and employees spend increasing amounts of time managing the organisation itself.
This is where institutional complexity can begin to undermine institutional capacity. The paradox is profound because complexity requires organisation, but organisation creates additional complexity. Eventually, a system can reach the point where substantial effort is devoted to managing the mechanisms originally created to manage external problems.
This does not mean large institutions are inherently dysfunctional. Scale can produce enormous capabilities. Governments can mobilise resources no private actor could match, and large corporations can coordinate technological and financial resources across continents. The challenge is maintaining coherence as scale increases. Without deliberate effort, institutional growth can produce fragmentation faster than integration.
If traditional institutions struggle with complex risk, the solution is not to eliminate specialisation. Expertise remains essential. The challenge is to build institutions capable of connecting specialised knowledge.
This requires a shift from risk categories toward risk relationships. Instead of asking only what risks exist within each department, organisations must ask how risks move between them. What depends on what? Which functions share critical infrastructure? Which suppliers support multiple business units? Which external systems could create cascading effects? Where are the points at which multiple risks converge?
Governments need similar capabilities. Policy analysis must increasingly examine interactions across systems rather than treating sectors independently. Climate planning should consider water, agriculture, migration, infrastructure, finance, health, and energy together where relevant. Industrial policy should consider energy systems, critical minerals, workforce capacity, technology, trade, and national security.
This does not mean creating a single giant institution responsible for everything, because that would simply create a different form of complexity. The goal should be distributed expertise with integrated awareness.
Institutions need mechanisms that allow information to move horizontally rather than only vertically. Decision-makers need tools that reveal dependencies across departments. Planning processes must examine second- and third-order effects. Scenario analysis can help institutions explore how disruptions propagate through systems rather than assuming risks remain within predefined categories. Adaptive institutions must also be capable of changing their own structures, because if external systems evolve while institutional arrangements remain fixed, the gap between them will grow.
Modern governments and organisations were built through specialisation because specialisation made complexity manageable. That achievement should not be dismissed. Bureaucracies, departments, regulations, professional expertise, and organisational hierarchies allowed societies to manage levels of scale and complexity that would otherwise have been impossible.
But success produced a new challenge. The divisions created to make reality manageable can prevent institutions from seeing reality as an interconnected system. Climate does not stop at the boundary of an environmental agency, cyber risk does not stop at the information technology department, financial instability does not remain within financial markets, and food systems cannot be separated from water, energy, transportation, climate, and trade.
The categories remain useful, but the boundaries can become misleading. Institutions therefore face a transition from managing isolated risks toward managing relationships among risks. This requires more than better forecasting or additional regulation. It requires a different institutional mindset, one that recognises that systemic vulnerability often exists between departments, between agencies, between organisations, and between jurisdictions.
The greatest risks may not belong to anyone because they belong to everyone. That is the institutional trap: we created organisations to simplify complexity, then added structures to manage the problems produced by those organisations, followed by new rules to coordinate those structures and new institutions to oversee those rules.
At some point, the question becomes unavoidable: what happens when the institutions created to manage complexity become too complex to manage it?
The answer will shape whether governments and organisations can govern the increasingly interconnected systems upon which modern societies depend.
illuminem Voices is a democratic space presenting the opinions of leading Sustainability Thought Leaders, their views do not necessarily represent those of illuminem.
The world needs sustainability knowledge. At illuminem, no interest group or shareholder can influence our work. Thank you for supporting our mission to make high-quality and independent sustainability information free for all. Every contribution helps. Thank you for donating today.
John Elkington
Corporate Governance · Public Governance
Steven W. Pearce
Public Governance · Ethical Governance
illuminem briefings
Carbon Market · Public Governance
Climate Change News
Public Governance · Sustainable Finance
Politico
Carbon Regulations · Public Governance
Le Monde
Public Governance · Ethical Governance