When we started building Coachvyne, one of the first design questions we faced was where to put the feedback: during the simulation or after it? The natural instinct in most learning design is to let the activity complete and then debrief. Running the exercise, then reviewing. Playing the game, then analyzing the play. This is the dominant pattern in executive education, management training, and most simulation-based learning.
We made a different choice, and the data from the first year of sessions has reinforced it in ways that shaped how we think about feedback design more broadly.
The Learning Science Case for In-Session Feedback
The research on feedback timing in skill acquisition has been fairly consistent for decades: feedback is most effective when it arrives proximate to the behavior it's responding to. This holds across motor skills, cognitive skills, and interpersonal skills. The mechanism is straightforward — the closer the feedback is to the behavior, the more precisely the learner can connect what they did to the signal they received, and the more likely the behavioral encoding is to stick.
The complication in leadership development is that "real-time feedback" has typically meant a coach or observer interrupting a scenario to comment, which is disruptive and often breaks the psychological engagement that makes the scenario valuable in the first place. Most simulation-based programs resolve this by moving feedback to the end — preserving the scenario's integrity at the cost of the feedback timing.
We tried to find a third option: feedback that is delivered within the flow of the session, attached to specific moments, but delivered in a way that doesn't interrupt the scenario's momentum. Whether we succeeded is something the data can help answer.
What We Compared
In the first phase of Coachvyne sessions, we ran two variants. In one, leaders received dimension scores and behavioral commentary during the session — surfaced as brief, specific observations at natural pause points within the simulation. In the other, they received the same information in a structured debrief at the end of the session. The scoring content was identical; only the timing differed.
We tracked behavioral change across the two groups using repeat sessions — meaning the same leaders ran a second simulation of the same type two to four weeks later, and we measured how much their dimension scores changed.
The in-session feedback group showed faster dimension score improvement in the repeat session, particularly on dimensions that involve real-time behavioral adjustment: Directness Under Social Pressure, Emotional Regulation, and Narrative Coherence. The post-session debrief group showed more improvement on dimensions that are cognitively oriented — Prioritization Logic, Accountability Framing, and Decision Under Uncertainty.
The pattern makes sense once you look at what each dimension requires. Behavioral dimensions — the ones where what matters is what you do in the moment — respond better to feedback that arrives in the moment. Cognitive dimensions — where the quality of your thinking matters as much as your real-time behavior — respond well to the kind of reflective analysis that a post-session debrief enables.
The 2.8x Figure and What It Actually Means
The headline finding — that in-simulation feedback changes behavior 2.8x faster than end-of-session summaries on behavioral dimensions — deserves some context before it becomes a marketing claim we overuse.
This is from our internal data, not a published, peer-reviewed study. The sample is relatively small and the populations aren't perfectly matched. We're not publishing this as a scientific finding; we're reporting it as a directional signal that confirmed the design choice we made and informed how we weight the two feedback modes in a given session.
What the finding does tell us, with reasonable confidence, is that for the specific behavioral dimensions we measure, the timing of feedback is a meaningful variable — not an aesthetic choice, but a design variable that affects outcomes. Changing from post-session debrief to in-session feedback for Emotional Regulation and Directness Under Social Pressure made a measurable difference in how quickly leaders adjusted in their next session.
We're not saying post-session debrief is less valuable overall. For building the kind of metacognitive awareness that makes leaders better at self-diagnosing and self-correcting over time, the structured retrospective conversation is genuinely important. The most effective sessions we've seen combine both: in-session signals at key moments, followed by a more integrative post-session discussion that connects what happened to the broader dimension profile.
The Memory Problem in Post-Session Debrief
There's a practical problem with relying entirely on post-session debrief that the learning science literature doesn't always emphasize: leaders often don't remember the specific moments that the debrief is trying to address.
A simulation session is cognitively dense. A leader who has just navigated 20 minutes of a board challenge or a difficult termination conversation has processed a large volume of information and made a large number of micro-decisions. When the debrief begins, the coach or scoring system might reference a specific moment — "at around minute 14, when the simulated board member challenged your Q3 forecast, you said X" — and the leader often can't fully reconstruct what they were thinking or feeling at that moment. The experience is already partially abstracted.
In-session feedback anchors to the moment while the moment is still live. "You just used passive voice to describe the revenue miss — that's a low-Accountability-Framing signal" means something different when it arrives 30 seconds after the moment than when it arrives 25 minutes later in a debrief. The leader has full access to the context: what they were thinking, what they were feeling, what other options occurred to them. That context is the medium in which feedback becomes behavior change.
Designing Feedback That Doesn't Break the Scenario
The reason most simulation-based programs don't use in-session feedback is that it's genuinely difficult to execute well without disrupting the experience. An interruption at the wrong moment breaks immersion, reduces the psychological reality of the scenario, and can trigger defensiveness that makes the feedback harder to absorb.
We spent a significant amount of time on this problem. The approach we settled on uses pause points that are built into the scenario structure — natural moments where the conversation has reached a node (a question has been answered, a decision has been made) before the next phase begins. Feedback surfaces at these nodes as brief, specific, behavioral observations: not evaluation, not instruction, just a precise description of what the leader did and what dimension it scored on.
This keeps the observation close to the behavior, maintains the session's momentum, and avoids the interpretive leap ("you need to be more direct") that generic feedback tends to require. Leaders can use the observation in the next segment of the scenario — applying it immediately while the session is still active — or they can hold it for the post-session debrief, where it will have an anchor they can actually remember.
What This Changes About How We Think About Coaching ROI
One of the persistent challenges in coaching ROI measurement is the gap between learning and behavioral transfer. Leaders can finish a development program feeling they've learned something valuable and still show no measurable change in their leadership behavior six months later. The learning happened. The transfer didn't.
Feedback timing is one variable in the transfer problem. It's not the only variable — practice frequency, the relevance of scenarios to real current challenges, and the quality of follow-through all matter. But it's a variable that's underweighted in how most programs are designed.
When we track dimension scores across repeat sessions, the leaders showing the fastest improvement are consistently those who received in-session feedback on their behavioral dimensions and used the post-session debrief to build the metacognitive layer. They change behavior because the feedback arrived when they could do something with it, and they understand why they changed behavior because the debrief gave them a model for it.
That combination — in-session signal, post-session understanding — is the design we've landed on, and it's the basis for how we structure the feedback component in every Coachvyne scenario. The alternative, which is the standard coaching and simulation model, gives you the understanding but misses the signal at the moment of behavior. Given the choice, we'd rather not leave that on the table.