In “Public Explanation as a Test of an Engineering Solution”, I wrote that a claim such as “the solution works” needs an observable fact to support it. But there is another question: when should we decide which fact is sufficient?
If verification is designed after implementation, the answer can easily become influenced by what has already been built.
A developer writes the code, sees how it works, and then chooses a convenient way to demonstrate that the task is complete. Verification is no longer an independent criterion. It partly becomes confirmation of the chosen solution.
This is why implementation and verification are worth separating before the first code change.
Done should not be defined by the implementation
A task usually has an expected result. Implementation introduces a particular way to achieve it.
These are different things.
Suppose a service configuration needs a new option. After implementing it, it is easy to say: the new parameter is parsed, the parser test passes, so the task is done.
But the original task may require more. The value must not only be parsed but actually affect the service behavior. Existing configurations must continue to work. Invalid values must be rejected. Other parameters must behave as before.
If these conditions are formulated only after the code has been written, the definition of done can be adjusted to fit the implementation.
This is a common form of confirmation bias: once a solution has been chosen, it is easier to look for evidence that supports it than for conditions under which it would be wrong.
Promising to “test it properly later” does not solve this problem. The evidence required for acceptance should be defined in advance.
Define the result before choosing how to achieve it
Before implementation, it is useful to record three things:
what should change
→ what should not change
→ how to prove it
At this point, the structure of the future code does not need to be known.
For a configuration change, the criteria might be:
the new value changes the required behavior
configuration without the new parameter
preserves the previous behavior
an invalid value is rejected
the rest of the configuration behaves as before
Only then does implementation begin.
The solution may require changing the configuration schema. It may need value transformation or a new connection between components. Those are implementation decisions.
The acceptance criteria should not depend on which implementation was chosen.
This separation does not require different people or teams. The same engineer can define the criteria, implement the change, and verify it. The important boundary is temporal: the criteria exist before the implementation result is known.
Three levels of verification answer different questions
One type of verification is usually not enough.
The first level is acceptance criteria. They describe observable system behavior and the boundaries of the change. They form the task contract: what must become true after implementation.
The second level is automated verification. It turns some of those criteria into reproducible evidence. A test can be run again after another change to confirm that a previously verified property still holds.
The third level is a manual scenario. It is useful when several components need to be observed together or when the actual environment matters. For a configuration change, this might mean starting the service with the new value and checking its real behavior.
These levels do not replace one another.
A parser unit test can prove that a configuration value is read correctly, but not that the service actually uses it. A manual run with one new value may demonstrate the main scenario while saying nothing about backward compatibility.
A good verification plan therefore starts from the properties that need to be proven, not from the tests that happen to be easy to write.
Verification should be able to disprove the solution
A useful measure of verification quality is whether it can show that the implementation is wrong.
If almost any implementation is guaranteed to pass a chosen check, the evidence is weak.
For example, after adding a new parameter, we could verify only that the service starts successfully with that parameter. This confirms the main scenario but misses several classes of failure: the value may be ignored, existing configurations may stop working, or invalid values may be accepted silently.
Verification becomes stronger when it covers not only the expected positive result but also the boundaries of the change.
This is why “what should not change?” matters as much as “what should start working?”
The distinction is particularly useful for small changes. They often have an obvious positive scenario and a less visible risk of regression in neighboring behavior.
Code review does not start with the diff
A code review can easily begin with the implementation: open the diff, inspect the structure, and decide whether the change looks reasonable.
Then verification is again being built around an already chosen solution.
A better starting point is to reconstruct the task contract: what should change, what should remain unchanged, and what evidence the author considers sufficient.
Only then does the diff answer one of the questions: how was the required result achieved?
Tests and other checks answer another: how do we know that the result was actually achieved?
This makes the review less dependent on how convincing the implementation itself looks. Well-written code does not prove that the task is complete, while an unusual implementation is not necessarily wrong if it satisfies the defined contract without violating its constraints.
This boundary matters even more for AI agents
It is easy to give an AI agent a task like:
implement the change and verify that it works
But this makes the same executor both choose the solution and decide what counts as sufficient verification after making the change.
The agent can write the code, add a test tailored to its own implementation, run it, and produce a completely consistent but overly narrow set of evidence.
A better sequence for agentic work is:
task
→ acceptance criteria
→ verification plan
→ implementation
→ run the predefined checks
→ map each criterion to its result
This does not mean the verification plan can never change.
Implementation may reveal a new constraint or show that an initial assumption was wrong. In that case, the criteria should genuinely be reconsidered. But this should be a separate decision, not a silent weakening of verification because the original requirement became inconvenient.
The same structure helps when the result is handed to another agent or a person for review. The reviewer does not need to infer the author’s intention from the diff: the expected result and the evidence required to support it are already explicit.
A simple template
Before implementing a small engineering task, it is usually enough to write down:
What should change:
<observable result>
What should not change:
<important invariants and backward compatibility>
How to prove it:
<automated checks>
<manual scenario, if needed>
After implementation, the actual result can be compared against this contract.
If defending the solution requires changing the definition of done, that is a reason to reconsider either the criterion or the implementation first.
Verification is most useful when it exists independently of the solution it is supposed to verify.