Investor interest in environmental, social, and governance information has exploded over the last several years. By some estimates, roughly a third of all assets under management now consider some aspects of ESG information in their investment strategy.

However, doubts have lingered about the validity of ESG as an indicator of financial performance. Recently, these doubts have expanded to an apparent backlash against the promise of ESG investing, which can be summarized into two spheres of thought. The first is that any financially material ESG information is already captured by more-traditional market fundamentals. This is an iteration on the efficient-market hypothesis, in which prices are thought to reflect all information. Under this sphere of thought, the empirical studies that suggest a link between ESG and financial performance is either poorly constructed or measures proxy information for a market fundamental rather than a unique impact of ESG.

The second sphere of thought is that ESG aspects, by their nature, are externalities to the business, and accepting responsibility for externalities is inherently a cost and drag on financial performance. With roots in the ideals of Milton Friedman, this viewpoint argues that consideration of ESG aspects by businesses can only be financially beneficial when governments create penalties or incentives. In other words, the correlation between ESG and financial performance is only true in the presence of market-distorting regulation. This creates the scenario in which capital flow to sustainable investments is inherently less efficient, and so markets will seek more-efficient investments or circumvent the regulations.

While some may expect the repudiation of ESG as an investment fad, our view is that the current debate on the validity of ESG heralds not the end of ESG investing but instead a transition toward major improvement.

Market externalities do not disappear; they are integrated

Markets are not perfectly efficient. They have always encountered novel factors as evidenced by crashes such as the dot-com bubble. ESG represents an enormous and complex system of novel factors for markets. From the uncertainty around climate impact, to the consequences of ecosystem collapse, resource depletion, social instability, and political upheaval, global ESG factors are the epitome of information that will be interpreted differently by market players, resulting in inconsistent pricing. Moreover, these factors are changing rapidly as we see the impact of ESG factors on society and economies unfold.

At the corporate level, profound transformation is being driven by digitalization and decarbonization. Entire sectors are shifting away from industrial-era strategies, toward smarter, cleaner, and more-agile models. ESG plays a central role here. The price of human and environmental capital continues to rise, driving greater efficiencies in time and resource use. Governments are creating trillions of dollars in incentives to promote the renewable energy economy. ESG externalities represent one of the predominant nonmarket forces at play today.

While protecting environmental and social capital is an externality in the traditional sense, it is also a fundamental asset on which companies and economies rely. A rapidly changing world is reshaping the framework conditions within which corporations seek to compete and thrive, but the fundamental need to access human, social and ecological systems to create financial value will always be present. In fact, recent regulatory and technological changes, the impact of climate change, and evolving social norms captured by ESG factors make them ever more important for valuations.

Major transitions are never smooth or linear

ESG is messy. Not only is the data difficult to collect and assess, but the very nature of ESG challenges how we measure economic growth and value. Our efforts to understand ESG factors is deeply connected with corporate transformation and the use of digital technologies for data-driven assessments. This transformation affects all segments of society and all market players.

It is unsurprising therefore that we will encounter bumps in the road in the development, scaling, and integration of ESG into investing. Like other major market transformations before, the growth of ESG will be dotted with bubbles and backlashes. Some aspects of ESG will be fundamental to short-term financial success. Others will have longer-term impacts or will be felt by companies and investors through complex, intangible effects.

Over the short term, the relationship between ESG and financial performance is difficult to interpret, and it is likely that a relatively small number of ESG factors will have a significant effect. Some companies will continue to drive strong returns in the short and medium term by burning the globe, while we will see bubbles of performance in portfolios of more-sustainable stocks. Over longer periods, however, many analysts agree that the relationship between ESG and long-term financial performance is sufficiently established.

The time-dependency of market transformations has always allowed for multiple successful investing strategies. Short-term investors can get in and get out before long-term transformations take hold. However, with ESG, we face a different set of consequences for both markets and society. Failure or delay to decarbonize will create enormous, and potentially irreversible damage. Success will entail managing consistent performance against different timescales simultaneously.

ESG ratings have become part of the problem

How then do we identify ESG factors that build for short and long-term financial success? Traditionally, we look to analysts to comb data for relationships. However, there is now ample evidence that existing ESG ratings are imperfect and at times unhelpful. One of the predominant challenges is that the method by which ESG scores are developed is frequently based on backward-looking information and biased, or greenwashed, datasets. The resulting incoherence between raters has been widely criticized.

Efforts to impose quality standards on rating organizations have largely failed. Today, there are about 600 rating systems and despite recent industry consolidation, the market is still fragmented.

New technology is also accelerating smarter analysis of data through machine learning and AI. Digitalization will enable investors to have greater flexibility and customize data at lower cost.

Comparability in ESG data will also be driven by regulators. The European Union is in the process of defining minimum standards of ESG disclosure for companies and asset managers. The SEC has recently opened consultations for potential disclosure rules or guidance on climate change and ESG. Broader and more consistent disclosure, especially of carbon emissions and financial risks of climate change, will bring more clarity. These trends will ease the burden for companies. Today, businesses spend significant time and capital to respond to myriad requests for information from investors, raters, and other stakeholders. More consistent reporting expectations and new technology will allow companies to track ESG information vital to their business strategy and enhance their management of ESG risks and opportunities.

The current backlash against ESG will not stop the inevitable integration of ESG into corporate management and asset valuation. However, it is likely to have consequences—both constructive and destructive. Delay can be devastating, and there is a real danger that the backlash will slow the movement of capital toward more sustainable investments in the short term, just when that capital is most urgently needed.

On the other hand, greater scrutiny on ESG data and the correlation between ESG and financial performance can only serve to improve data quality and processes. Analyses of ESG factors will become more, not less relevant as data improves and these relationships become better defined. The barriers to these deeper understandings are today being lowered through technology and regulation. The result will be to create new mindsets for adjusting valuations in line with current and emerging global changes. ESG 2.0 is in the making.

This article is also published in Barron's. Energy Voices is a democratic space presenting the thoughts and opinions of leading Energy & Sustainability writers, their opinions do not necessarily represent those of illuminem.