Counterfactual Reasoning in WFM

From WFM Labs

Counterfactual Reasoning in WFM is reasoning about what would have happened under circumstances that did not actually occur — the top rung of the ladder of causation. Where association describes what is observed and intervention predicts the effect of a deliberate change, a counterfactual concerns a specific case under a hypothetical alternative: "would this agent have left if their schedule had not changed?" or "what would service level have been last quarter without the new forecast model?" Counterfactuals are the language of attribution, credit, and blame, and they are the hardest causal questions to answer because the alternative outcome is, by definition, never observed.[1]

The fundamental problem

For any unit and any decision, only one outcome is ever seen: the agent either left or stayed, the quarter ran with the new model or it did not. The unobserved alternative — the counterfactual — must be inferred, never measured. This is the fundamental problem of causal inference, articulated formally in the potential-outcomes framework: the effect on an individual is the difference between two outcomes, only one of which can exist.[2] Because the comparison is to something that did not happen, every counterfactual claim depends on a model or an assumption about how the alternative world would have unfolded.

Two framings

Two complementary traditions make counterfactuals rigorous. The potential-outcomes (Neyman–Rubin) framing defines, for each unit, the outcome under treatment and the outcome under control, and treats the missing one as a missing-data problem to be solved by design (randomization, matching, weighting).[2] The structural (Pearl) framing computes counterfactuals from a causal model in three steps — abduction (use the observed evidence to update what is known about the case), action (alter the model to reflect the hypothetical), and prediction (read off the result). Both answer the same kind of question; the first is dominant in program evaluation, the second in causal modeling.

Why it matters in WFM

Counterfactuals appear whenever WFM moves from "what happened" to "what was responsible":

  • Attribution of savings. "The new model saved 400 hours" is a counterfactual claim — it compares actual hours to the hours that would have occurred under the old model, which were never observed. A simple before-and-after difference is a weak proxy and is vulnerable to regression to the mean and to coincident changes.
  • Individual retention. "Would this agent have churned without the intervention?" is an individual-level counterfactual — the hardest kind, since it concerns one person, not an average. Honest answers come as probabilities, not certainties.
  • Incident post-mortems. "Would service level have been missed without that lever?" requires reconstructing the unobserved alternative, and is where hindsight bias does the most damage by making the alternative feel obvious.
  • Scheduling and policy what-ifs. Comparing the realized plan to a plan that was never run is counterfactual; causal-inference designs and simulation are how such comparisons are made credibly rather than rhetorically.

Estimating counterfactuals credibly

Because the alternative is unobserved, the credibility of a counterfactual rests entirely on how the comparison world is constructed:

  • Controlled experiments supply a real, observed comparison group whose average stands in for the counterfactual, which is why a randomized A/B test is the strongest basis for an attribution claim.
  • Quasi-experimental designs (difference-in-differences, matched comparisons, synthetic controls) build a comparison from observational data under stated assumptions; these belong to causal inference.
  • Structural models and simulation generate the counterfactual from an explicit causal model — only as trustworthy as the model and its assumptions.

The discipline is to state the comparison world explicitly. A counterfactual claim with no stated alternative — "the change worked" — is rhetoric, not evidence.

Maturity Model Position

In the WFM Labs Maturity Model™, how an operation handles attribution is a direct read on its causal maturity.

  • Level 1–2 (Emerging / Foundational) — attribution is asserted from before-and-after movement; savings and successes are claimed without a stated comparison world.
  • Level 3 (Progressive) — attribution claims name the counterfactual and support it with a comparison group or a quasi-experimental design rather than a raw before-and-after.
  • Level 4–5 (Advanced / Pioneering) — counterfactual estimation via experiments, structural models, or simulation is routine, and automated systems reason about alternatives rather than assuming their own decisions caused the outcome.

See also

References

  1. ↑ Pearl, J., & Mackenzie, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books. ISBN 978-0-465-09760-9.
  2. ↑ 2.0 2.1 Holland, P. W. (1986). "Statistics and Causal Inference". Journal of the American Statistical Association, 81(396), 945–960.
Retrieved from "https://wiki.wfmlabs.org/w/index.php?title=Counterfactual_Reasoning_in_WFM&oldid=3367"