Beyond Expert Agreement: Measuring Originality in Human–AI Reasoning

Combining causal reasoning training with LLM access enhances multiple dimensions of intellectual output beyond standard metrics.
Scientific evaluation must distinguish intellectual conformity from genuine causal insight and originality.
Short, focused causal reasoning exercises offer distinct benefits not replicated by LLM access alone.

A randomized controlled study conducted at Bocconi University involving thirteen class sections and 1,053 first-year management students tasked participants with composing a concise, 180-word recommendation addressing increased alumni engagement with the university's merchandise shop. The experimental design allocated students across four conditions: access to ChatGPT, participation in causal reasoning training, a combination of both, or a control with neither. Analysis revealed that participants utilizing ChatGPT produced responses rated higher in alignment with expert solutions by evaluators. In contrast, only those who received causal reasoning training demonstrated the ability to explicitly identify conditions under which their proposals might fail, provide mechanistic explanations for expected outcomes, and generate ideas distinct from peer contributions. Notably, the group exposed to both interventions performed robustly across all evaluated metrics without deficiencies. This empirical evidence encapsulates a microcosm of the ongoing shift in intellectual work, wherein organizations increasingly seek individuals capable of formulating novel causal inquiries while adeptly leveraging AI systems to support their exploration and articulation.

Delineating Impacts of ChatGPT and Causal Reasoning Training

The study employed a rigorously pre-registered 2×2 factorial randomized design at the classroom level, splitting thirteen sections into four groups: causal reasoning training only, exclusive access to ChatGPT via the university's institutional subscription in a dedicated browser tab, the combination of both interventions, and a control condition without either. The causal training was concise, consisting of a twelve-question interactive exercise with immediate feedback, designed to cultivate explicit causal chains, delineate failure conditions, and require mechanistic explanations for proposed claims. Participants in the control condition engaged in an identical set of questions without receiving feedback. Outcome measures spanned eight distinct dimensions, including task performance evaluated blind by twenty trained graduate students (three evaluators per text), reasoning quality assessed through large language model ratings grounded in a pre-established rubric, and textual similarity quantified against three independently crafted expert solutions from a marketing professor, the university store manager, and a seasoned alumnus. This multifaceted evaluation framework enabled nuanced analysis differentiating performance, causal understanding, and originality.

Results revealed a clear divergence in effects. ChatGPT access improved conventional performance metrics, elevating evaluator scores and increasing text similarity to expert answers. Conversely, causal reasoning training did not significantly boost conventional performance nor augment ChatGPT's effect on it when combined. Both interventions, however, enhanced textual coherence and idea richness, with ChatGPT contributing more substantially in these dimensions.

The combined intervention yielded effects surpassing those of each individual treatment. Notably, in dimensions reflecting intellectual depth—such as articulating failure conditions, explicating causal mechanisms, and generating ideas distinct from peers—only participants who underwent causal reasoning training showed measurable advances. Access to ChatGPT alone did not facilitate any of these deeper analytical qualities, and its addition to causal training neither diminished nor diluted the training’s impact. Across all eight assessed outcomes, the group receiving both interventions outperformed the control group and consistently matched or exceeded the groups subjected to a single intervention. Moreover, the conjunction of ChatGPT use and causal training produced mutually reinforcing improvements in coherent logical structure, mechanistic explanation, and idea diversity, underscoring the complementary nature of AI assistance and human causal reasoning in elevating sophisticated intellectual performance.

A pivotal methodological consideration emerging from the Milan experiment is that evaluators primarily rewarded responses closely aligned with established expert solutions, a standard easily met by large language models generating polished, conventional answers to typical marketing challenges. While standardized rubrics serve practical purposes for scalable, ostensibly fair evaluations across large cohorts, they inherently privilege conformity over intellectual novelty. Such frameworks tend to undervalue first-principles analysis and the critical reasoning skill of explicitly articulating failure conditions—competencies crucial for robust causal understanding but often invisible within traditional scoring systems. Consequently, organizations relying on legacy evaluation criteria risk incentivizing superficially refined compliance that coincides with the democratization of AI-generated text, thereby overlooking rare yet vital abilities to critically assess when and why a proposal might fail.

Scientific Distinction: Superhuman Labor Through Human–AI Causal Collaboration

The notion of superhuman labor, articulated by SIAI in August 2026, captures the synergistic interaction between humans and AI tools producing intellectual outputs that surpass solo human capability in quality, efficiency, and cost-effectiveness, without invoking artificial general intelligence. In this partnership, the human contributes contextual judgment, frames meaningful questions, and interprets results, while the AI provides rapid information retrieval, synthesis, and coverage. Crucially, SIAI emphasizes that an autonomous feedback loop relying solely on AI-generated content risks cyclical repetition of existing material, stifling innovation without human introduction of novel questions or reframing. The Milan randomized controlled trial operationalizes this distinction: access to the large language model expanded the quantity of generated ideas and improved conformity to expert benchmarks, whereas causal reasoning training altered the substantive nature of those ideas, yielding explanations with mechanistic insight and articulate failure conditions. Importantly, only the combined intervention produced outputs that simultaneously achieved high evaluative ratings and intellectual originality. Thus, this experimental evidence represents the most rigorous empirical instantiation of superhuman labor to date, demonstrating how integrated human causal reasoning and AI tools conjointly elevate intellectual performance compared to isolated use or control.

Empirical workplace data substantiate the critical importance of combining causal reasoning capabilities with AI tool use for meaningful performance enhancements. A randomized study involving 758 consultants at the Boston Consulting Group found that those accessing GPT-4 completed 12.2% more tasks and did so 25.1% faster, achieving over 40% higher quality scores—provided that the tasks fell within the model’s competence boundaries. However, performance notably deteriorated, with a 19 percentage-point drop in correct solutions when GPT-4 was applied beyond these limits. As these boundaries are not explicitly delineated in the user interface, the capacity to recognize when a model’s answer fails—a skill directly supported by the causal reasoning training’s emphasis on identifying failure conditions—is essential to prevent overreliance on plausible but invalid outputs. Similarly, in an American software company, deploying an AI assistant among more than 5,000 customer service employees led to a 14% increase in requests resolved per hour on average and a 34% improvement for less experienced staff, but mainly for tasks replicating established knowledge embedded in expert conversations. Collectively, these findings illustrate that while AI tools substantially enhance throughput and consistency on routine tasks, human causal insight—especially the awareness of validity constraints—is indispensable for maintaining accuracy and exercising judgment in complex or novel problem contexts.

A contrasting outcome emerged from a field experiment in Kenya, where 640 entrepreneurs used a GPT-4-based advisory service via WhatsApp over a five-month period. Aggregate performance showed no statistically significant difference compared to a control group; however, this average masked two diverging trends. Entrepreneurs already performing well improved by over 15%, whereas those facing challenges experienced an approximate 10% decline. Importantly, the nature of interactions with the AI—both in queries posed and responses received—were comparable across groups. The divergence in outcomes arose from differential selection and application of the AI-generated advice in contexts lacking clearly defined correct solutions. These findings highlight that while AI can amplify intellectual judgment, successful outcomes depend critically on human evaluative and integrative reasoning in applying AI suggestions to complex, open-ended problems.

Evolving Workforce Demands: Integrating Analytical Thinking with AI Proficiency

Emerging labor market analyses reveal a growing valuation of workers who integrate analytical reasoning skills with proficiency in AI technologies, even before such combined profiles are explicitly defined by employers. The World Economic Forum's Future of Jobs 2025 report, surveying over 1,000 employers spanning 14 million workers across 55 countries, identifies analytical thinking as the foremost foundational skill, with nearly 70% of firms deeming it essential. Simultaneously, competencies in AI and big data exhibit the fastest acceleration in employer demand, with the proportion of companies rating AI-related skills as basic increasing by 17 percentage points since 2023. These ostensibly separate skill rankings instead reflect complementary facets of a singular, hybrid intellectual profile valued in contemporary workplaces. Notably, 63% of employers cite skill deficiencies as the principal impediment to transformative business evolution by 2030, prompting 85% to emphasize upskilling current personnel over onboarding candidates with pre-existing expertise—underscoring the strategic recognition that causal reasoning and AI fluency must be cultivated in tandem within the incumbent workforce to realize productive human–AI collaboration.

Empirical data from the US Census Bureau's Business Trends and Outlook survey, compiled between December 2023 and February 2024, illustrates prevailing practices in workforce upskilling amid AI adoption. Specifically, 20.8% of firms utilizing AI resources reported investing in training their existing employees, contrasted with only 2.0% hiring personnel already proficient in AI tools. This tenfold discrepancy reflects constrained labor supply of individuals who proficiently integrate domain expertise, critical judgment, and AI fluency, as well as a corporate preference for developing such competencies within trusted incumbent staff whose decision-making track record is known. Concurrently, demand for AI education is accelerating, evidenced by Coursera's report to shareholders indicating enrollment rates in AI-related courses increased from 8 per minute in 2024 to 20 per minute in early 2026, underscoring substantial scaling of scalable, modular training pathways that foster causal reasoning and AI integration among diverse learners.

The impact of generative AI adoption extends markedly to entry-level employment dynamics. Analysis of approximately 62 million employee records and job postings across 285,000 U.S. firms reveals a 7.7% decline in hiring new entrants within organizations implementing generative AI during the six quarters following early 2023, while senior management hiring remained stable. This contraction implies heightened expectations for immediate, high-caliber contributions from incoming talent. Notably, the profile of successful new hires increasingly resembles that of the Milan experimental cohort—individuals equipped with both causal reasoning training and large language model access—underscoring the emerging necessity for integrated human–AI competencies even at the workforce entry point.

Short, Modular MOOCs as Scalable AI and Causal Training Pathways

In response to the growing imperative to develop workforce capabilities integrating causal reasoning with expertise in large language models, organizations are increasingly adopting scalable, role-specific digital learning programs. For example, on October 5, 2026, TO THE NEW, a Singapore-based digital engineering company, announced a collaboration with Coursera that provides its more than 2,200 employees across India, the United States, the United Arab Emirates, and Australia access to Coursera’s comprehensive enterprise catalog. This partnership underpins an internal initiative to accelerate AI adoption within each technical discipline through curated learning pathways, systematically phased in across teams. TO THE NEW characterizes all its client projects as AI-enabled, indicating a mature integration of AI fluency within professional practice. Similar industry movements include BSI’s May 2026 global training alliance with Coursera, the May 2026 merger of Coursera and Udemy expanding combined enterprise customers to approximately 18,000 organizations, and the U.S. Department of Labor’s April 2026 launch of an AI skills integration platform for apprenticeship programs. These developments reflect a methodological shift toward brief, modular, digital courses designed for rapid upskilling—an approach motivated by the scarcity of ready-made professionals combining critical causal insight and AI proficiency at scale, and the high costs and lag times associated with external hiring. Such initiatives exemplify the emerging scientific challenge: building training paradigms and evaluation instruments that effectively foster and measure the complementary contributions of human causal reasoning and AI capabilities in complex intellectual work.

A prominent counterargument posits that training focused exclusively on AI tools suffices, as these technologies disproportionately enhance performance among less experienced individuals and ostensibly mitigate variability in intellectual output, potentially rendering dedicated causal reasoning instruction redundant. Empirical evidence from customer service settings affirms these benefits in structured, routine task environments. However, findings from the Milan randomized controlled trial challenge this notion: ChatGPT access alone failed to foster the articulation of failure conditions, mechanistic understanding, or nuanced idea generation. Similarly, in the Kenyan entrepreneurial experiment, AI assistance normalized performance for stronger participants but disadvantaged weaker ones in contexts requiring open-ended judgment. These outcomes underscore that mere proficiency in instruction writing or tool navigation without causal reasoning skill development tends to produce superficially polished but semantically uniform responses. Importantly, the concise causal training administered in Milan—comprising a dozen targeted, feedback-driven questions—easily integrates into modular MOOC curricula, providing an efficient means to cultivate foundational causal insight. For recruitment, this translates into brief case studies prompting applicants to specify scenarios where their proposals might fail; for workforce development, it delineates a scaffolded pedagogical sequence placing first-principles causal analysis prior to technical tool training, complemented by evaluation metrics that capture idea differentiation alongside conformity measures.

In the Milan randomized controlled trial, the subgroup receiving both causal reasoning training and large language model access uniquely surpassed the control group across every one of the eight intellectual performance metrics measured during a single, time-limited task following a concise twelve-question intervention. The relatively low cost of the training component, combined with institutional subscriptions for LLM access, renders this integrated approach feasible for organizations already employing extensive MOOC platforms at scale. However, publicly available information about TO THE NEW’s enterprise-wide AI upskilling initiative does not clarify whether their role-specific learning paths incorporate interactive causal reasoning exercises comparable to the Milan study’s training regimen, nor does it detail methodologies for assessing the development of nuanced reasoning and originality as engineers complete onboarding. Thus, the eventual evaluation results from such corporate programs will be critical to discerning whether these implementations foster superficial tool fluency alone or truly replicate the synergistic cognitive gains documented when combining explicit causal insight training with AI-assisted generation, as evidenced in rigorous experimental settings.

This article represents a considered methodological synthesis by the Gordon Institute Science Forum editorial team and does not constitute formal policy or institutional positions.

References

Asirvatham, H. et al. (2026) Training Novices to Think, or Giving Them LLMs? Evidence from an RCT. CEPR Discussion Paper 21882.
Bonney, K. et al. (2024) Tracking Firm Use of AI in Real Time. CES Working Paper 24-16, U.S. Census Bureau.
Brynjolfsson, E., Li, D. and Raymond, L. (2025) 'Generative AI at work', The Quarterly Journal of Economics, 140(2), pp. 889-942.
BSI (2026) 'BSI and Coursera partner on global workforce skills development', press release, 5 May.
Coursera, Inc. (2026a) Q1 2026 Shareholder Letter.
Coursera, Inc. (2026b) 'Coursera completes combination with Udemy', press release, 11 May.
CXOtoday News Desk (2026) 'TO THE NEW partners with Coursera to bring AI learning to its entire workforce', CXOtoday, 5 October.
Dell'Acqua, F. et al. (2023) Navigating the Jagged Technological Frontier. Harvard Business School Working Paper 24-013.
Fumagalli, C. and Gambardella, A. (2026) 'Training novices to think in the age of large language models', VoxEU.org, 1 October.
Hosseini Maasoum, S.M. and Lichtinger, G. (2025) Generative AI as Seniority-Biased Technological Change. SSRN Working Paper 5425555.
Leopold, T.A. et al. (2025) The Future of Jobs Report 2025. World Economic Forum.
Otis, N.G. et al. (2026) 'The uneven impact of generative artificial intelligence on entrepreneurial performance', Management Science.
SIAI Editor (2026) 'Superhuman labor and the feedback loop no one is watching', SIAI AI Memo, 17 August.
U.S. Department of Labor (2026) 'US Department of Labor launches website to build artificial intelligence skills', news release, 29 April.

Adapted from Causal Reasoning and LLMs: Why Employers Now Want Both.