Correlation vs. Causation Between Sustainability Reporting and Sustainability Performance

The relationship between standardized sustainability reporting and sustainability performance is complex and nuanced. Through our semester-long consulting project, we sought to explore whether enforcing standardized reporting frameworks directly drives measurable improvements in sustainability performance, particularly within the mining sector. While our findings revealed clear correlations, we could not establish definitive causation. This outcome, far from being a setback, offered valuable insights into the challenges of linking reporting practices to tangible sustainability outcomes in one of the world’s most resource-intensive industries.

The question of Quantitative vs. Qualitative

After our successful midterm presentation to the client early last month, our team quickly began working on developing the case studies. Building on our Corporate Sustainability Assessment (CSA)-based scoring framework, we faced a significant challenge regarding the integration of quantitative versus qualitative metrics. Some team members strongly advocated for retaining qualitative metrics, arguing that they provided a deeper narrative of a company's sustainability journey. Qualitative data highlights initiatives and proactive efforts, particularly for companies just beginning to implement standardized sustainability reporting. These stories, while not always tied to immediate, tangible results, illustrate the intent and groundwork for long-term impact.

However, mid-project, our primary client contact took a leave of absence, leaving us to recalibrate with a new point of contact. This transition caused a temporary communication gap, compounded by the learning curve on both sides. When we finally resumed discussions, the new client contact emphasized that impactful initiatives inherently lead to measurable results, therefore quantitative metrics are representative. According to their perspective, qualitative metrics might dilute the scoring framework's objectivity and rigor. This perspective challenged our approach, but we ultimately decided to align with the client's preference, opting to include qualitative insights as disclaimers in the final report rather than as part of the scoring system.

Looking back, this experience taught me the importance of balancing client expectations with professional judgment. While qualitative metrics add depth, focusing solely on quantifiable outcomes would help prevent greenwashing, especially in an industry as nuanced as mining.

The Challenges 

As we progressed in developing the case studies, another challenge became apparent: the glaring inconsistencies in data reported by companies across years. We found that companies sometimes alter the aspects of their operations they choose to report, making it difficult to track trends or derive meaningful insights. Even for indicators reported consistently, discrepancies arose. For instance, one company's 2021 water consumption data differed between its 2021 and 2022 reports. This raised questions about whether the changes stemmed from evolving regulatory scopes, internal errors, or other factors.

These inconsistencies reflect broader challenges in the mining sector. Unlike industries with standardized processes, mining operations vary significantly by location, resource type, and extraction method. Moreover, companies often operate in regions with differing regulations, leading to fragmented reporting practices. The lack of uniformity in reporting frameworks exacerbates these discrepancies, making it difficult to assess industry-wide progress on sustainability goals.

Without a direct contact person at these companies—ideally a sustainability officer—it was challenging to interpret these discrepancies or seek clarifications. I attempted to reach out to a contact person at the company I was analyzing but received no response. This underscored a limitation in our project timeline: had we started this outreach earlier or involved the client directly, we might have obtained clearer and more reliable data. Reflecting on this, I realize the critical role that communication channels and transparency play in sustainability consulting.

We became our own experts! 

Despite these challenges, one aspect of our process that stood out was the collaboration and mutual support within our team. During the earlier phase, each of us became an "expert" in specific indicators we were tasked with developing. This expertise proved invaluable during the case study development phase. Whenever a team member encountered a question or issue related to a particular indicator, we relied on the expertise of the individual responsible for it. This dynamic fostered a sense of trust and strengthened our collective output.

Additionally, synthesizing our findings and deriving sustainability performance scores for the five mining companies was a rewarding exercise in teamwork and analysis. Even when faced with limitations in data and methodological constraints, we worked together to produce the best possible insights for our client.

It's correlation, not causation 

After finalizing the sustainability performance scores, we conducted a regression analysis to evaluate whether these scores correlated with improvements over time. The results revealed that only 2 out of the 5 companies showed statistically significant improvements in their sustainability metrics, as indicated by meaningful p-values in our analysis. While this outcome was somewhat disappointing, it did not entirely reject our hypothesis. Instead, it highlighted the difficulty of establishing causation between sustainability reporting and performance improvement.

Our analysis showed that while causation was not evident, correlations existed. Standardized reporting frameworks appeared to help companies identify best practices, set benchmarks, and build credibility with stakeholders. This finding points to the value of these frameworks as tools for guidance and transparency, even if their direct impact on performance remains ambiguous.

Conclusion 

Reflecting on the final phase of our project, I recognize the complexity and nuance of sustainability consulting, particularly in data-heavy industries like mining. This experience has emphasized the importance of rigorous methodology, clear communication, and adaptability in navigating client relationships and project challenges.

While we could not establish a direct causal link between standardized reporting and performance improvement, our findings provide a foundation for future research and highlight the potential of standardized sustainability frameworks to drive industry-wide change. The client seemed satisfied with our conclusions, and their feedback further refined our deliverables.

This project reinforced the importance of perseverance, collaboration, and critical thinking in addressing the evolving challenges of sustainability consulting. Looking ahead, I hope to apply these lessons to future projects, contributing to meaningful progress in sustainable investing and beyond.