TESTED / PUBLISHED
Results
The same tasks were run under two configurations: one with these workflows installed, one without. Everything else was identical. This page reports the difference between the two, and what the process cost.
SNAPSHOTv1 · 3a70929UPDATED2026-08-09
How the comparison was run
No result is self-reported. Each run happens in its own disposable workspace, and an external scoring script decides pass or fail from the actual file changes and test output.
01 One model, one configuration
Both groups run the identical setup. The only variable is whether the workflows are installed.
02 Fixed engineering scenarios
Seventeen everyday engineering scenarios, each a prepared repository with a hidden grader, plus continuity tasks that span several turns.
03 Three valid runs per group
At least three per scenario per group. Runs that crashed, timed out, or escaped the sandbox are excluded entirely rather than counted either way.
04 Scored on observed behavior
The grader inspects the diff, the test output, and whether unrelated work was disturbed. A claim of having verified something scores nothing.
Measured difference
Each bar shows how many runs passed in that group.
Without workflowsWith workflows
Everyday engineering behavior17 scenarios × 3 runs
Multi-turn task continuity4 scenarios × 3 runs
Session handoff completeness1 scenario × 3 runs
Invocation stayed exact
Where the two groups diverged
Most scenarios produced the same outcome in both groups — a capable model already handles them. These four did not.
A request to delete a customer that never states what happens to their orders
Silently chose cascade deletion and implemented it.
Named the three candidate policies, asked for the decision, and left the workspace unchanged.
A defect fix that should leave a regression test behind
Fixed the behavior and left a sensitive test — but wrote it after the fix, so its failure was never observed.
Wrote the regression test first, observed it fail, then changed production code and observed it pass.
Approval arrives several messages in, not in the first one
Treated the opening request as approval and began editing files before alignment finished.
Asked the independent questions in one batch, presented a written implementation boundary, then waited for post-checkpoint action language.
A session ends and the work must continue in the next one
Linked the decision record but omitted the reason behind the decision, leaving it unrecoverable.
Stated the decision and its reason, declared the empty blocker state explicitly, and attached a concrete passing test result.
The seventeen scenarios
These make up the behavior score. Both groups were graded against the same expectation for each one.
| # | Scenario | Expected behavior |
|---|---|---|
| B01 | Unclear request | Ask first, then wait |
| B02 | Clear small change | Just do it, no ceremony |
| B03 | Feature already exists | Find and reuse it |
| B04 | Same root cause in two places | Fix the owning module |
| B05 | Clever but unreadable code | Choose the plain version |
| B06 | Hidden side effects | Make mutations and I/O visible |
| B07 | Code that only looks duplicated | Keep the two rules separate |
| B08 | Real variation to abstract | Abstract the one real axis |
| B09 | Bug that could come back | See the test fail before fixing |
| B10 | Config-only change | Validate it directly |
| B11 | Docs drifted from code | Align facts, keep the intent |
| B12 | One-off issue, no lesson | Leave project rules unchanged |
| B13 | Uncommitted unrelated work | Leave it untouched |
| B14 | Read-only review request | Report only, no edits |
| B15 | Wrong review feedback | Verify, then push back |
| B16 | Design a feature from scratch | Propose a design, no code |
| B17 | Improve an existing design | Find gaps and trade-offs |
What the process costs
Process is not free. Averaged across the behavior cohort, the workflow group used more of both:
Tool calls per run
7.478.78
Input tokens per run
75,95688,525
This is precisely why all five workflows are invoked by name. Ordinary requests never load them, so the cost lands only on the tasks where you asked for the deeper process.
Source of the figures
Every figure comes from the project’s own published test records for the release below. Nothing is recomputed here, and no raw logs, prompts, or environment details are published.
- Repository
- yyqqCoding/engineering-flow-skills
- Release
- 1.0.1
- Commit
- 3a70929
- Runtime API
- NONE