When Building Gets Faster Than Learning

Break a large job into pieces. Start. Learn from what the work reveals. Adjust what comes next. A house remodel is a familiar version of the same pattern: open a wall and find something unexpected; a job that should have taken an afternoon consumes a week; nothing goes wrong and someone simply discovers a better way to finish the rest. Meanwhile the house is still someone's home. People still need to sleep, cook, and use the bathroom. The plan was not necessarily wrong. The project simply knows things now that the planner could not know when the plan was written.

Decomposition helps us decide how to begin. Incremental execution gives us repeated opportunities to improve how we continue.

That is ordinary craftsmanship. Generative AI did not invent it. GenAI did change the clock. It arrived with little shared guidance, while many organizations were left to figure out the practice for themselves and bosses pressed for an immediate return on the investment. Work that once took months can sometimes take weeks or days. That is an advantage worth taking.

It also changes which conditions we encounter while building.

Suppose an organization is replacing a major reporting system. A project that once might have taken two years can now produce a working version in weeks.

We can compress implementation without compressing the business calendar.

On projects I've worked on, we began comparing the new system's output against the legacy system as soon as the new components could produce data. When that parallel comparison was possible, validation began well before the replacement was complete.

Over a two-year project, components ready early enough could be exercised through monthly closes, quarterly reporting, and year-end. Discrepancies sent us back to investigate, correct, and compare again. Each cycle taught us more about both the software and the business it served.

A working version built in a few weeks might encounter none of those events before release. That has consequences for how we plan projects, decide what is ready for use, and structure the engagement that follows deployment.

The loops were already there

The delivery practices I'm describing already include feedback, testing, and revision. Teams build components, compare results, investigate surprises, and adjust the remaining work. Not every project does this well, but assume those loops are in place.

The question is what happens when we run them over a few weeks instead of two years.

We may complete more implementation cycles in less time, while encountering a much narrower range of business conditions. Running another hundred tests against ordinary transactions doesn't teach us what happens during year-end processing.

Time itself was part of the test suite: while development continued, the business supplied new conditions against which we could compare the parts of the replacement that were ready. The calendar gave our existing validation loops more to work with.

If year-end fell within development, we could use it. If the build now takes place in March, that opportunity has moved beyond the implementation schedule.

Friday's code doesn't contain two years of Tuesdays.

Learning runs on different clocks

Some learning comes from building. We discover a better abstraction, an unexpected constraint, a repeated pattern, or a simpler architecture. GenAI can help us explore alternatives and learn from them sooner.

Some learning comes from use. People discover that a workflow is confusing, that two pieces of information belong together, or that the new process doesn't match how the organization works. Releasing a useful increment can bring that feedback forward while also reducing manual effort or delivering other value.

Some learning comes from operating through business cycles. A reporting system encounters closing adjustments, annual calculations, late corrections, and exceptions that ordinary daily use may never exercise.

View larger

Close

We can bring some of those conditions into testing. Historical data, parallel runs, simulated loads, and deliberate exception tests all help. For year-end, however, recreating the relevant data, configuration, dependencies, and sequence of operations can require substantial effort. A saved set of transactions may be only part of what made that year's close behave as it did.

When the real event occurs during development, it gives us a validation opportunity we would otherwise have to reconstruct or wait for. Even a careful reconstruction leaves a question about how well it represents the next live event.

This is also why comparison with the legacy system needs judgment. Agreement is useful evidence; a discrepancy needs investigation. The old system can have defects too.

Faster implementation gives us more choices about when to deliver value. We still need to account for the evidence those choices leave outstanding.

Keep learning connected to the plan

The feedback loops inside implementation still matter. Their findings need to influence what happens next.

A coding agent can work through a multi-step plan, testing and correcting each component as it goes. We should also give it opportunities to reconsider the plan itself:

Knowing what we know now, would we still approach the remaining work the same way?

Otherwise, we can end up with fourteen well-implemented components that all reproduce a decision Component 2 gave us a reason to reconsider.

That is the remodel question again, asked of software. A short review after a meaningful increment may reveal a simpler approach, a mistaken assumption, or useful value we can release before the rest is complete.

That review should also ask what the increment has actually encountered. Passing its tests and serving its first users may establish confidence for everyday work while leaving quarterly or annual behavior unresolved.

The plan needs to carry those unanswered questions forward.

Plan for the validation that remains

Suppose the reporting system is ready for a limited release in March. Daily reporting has been tested, users can complete the intended workflows, and results reconcile against the agreed comparisons. Year-end remains unobserved in live operation.

We have known, unresolved uncertainty. We can plan around it.

Before release, the team and the business owner should agree on:

• What the system is ready to do, and what remains unvalidated.
• Which tests or historical comparisons can provide evidence now.
• Which operating events will provide additional evidence, and what will be collected.
• Who will review the results and decide whether they meet the agreed criteria.
• What happens if they don't, including any limits on use or fallback arrangements.
• What capacity will remain available to investigate and make changes.

Some uncertainty may be too consequential to accept for the proposed release. The response could be more testing, a narrower release, continued parallel operation, or delaying the affected capability. In other cases, useful work can begin while later validation is scheduled.

The amount of validation should reflect the consequences of being wrong and the uncertainty that remains. Size alone is a poor guide: a small automation can still perform consequential work.

“Ready for this use” is a decision we can support with evidence. “Fully proven under every meaningful condition” is a much harder promise to keep.

Carry the loop beyond deployment

When a business event arrives, the loop continues:

Operate → Observe → Evaluate → Adapt → Verify → Decide

At the first quarterly close, for example, we collect the agreed results and exceptions. We compare them with the expected outcomes, investigate differences, and decide what they mean. A required change goes through testing and verification before we rely on it. If no change is needed, the evidence still strengthens our confidence.

The decision may be to expand use, continue observation, restrict a capability, or stop using it until a problem is resolved. Each review also updates the remaining validation plan. A successful quarterly close does not settle every question about annual processing.

Some reviews therefore belong on the business calendar: first monthly close, first quarterly close, first seasonal peak, first year-end. Others can be scheduled exercises, such as a restore test, without waiting for an outage to make them urgent.

Trust grows as evidence supports intended uses and the people responsible understand the system's limits. The passage of time creates opportunities for that work. We have to preserve the ability to do it.

What this should mean for a vendored engagement

For projects with meaningful uncertainty tied to business cycles, the vendor should agree with the client at the outset how the engagement will continue through operational validation.

The work can have three parts, which may overlap as capabilities are released:

• Implementation and release. Build and test the system, establish the evidence for its initial use, and record the questions that remain open.
• Initial operation. Support early use and correct defects against the agreed requirements.
• Planned operational validation. Return at the agreed business events, review evidence with the client, and act on the findings.

For the reporting-system example, the vendor and the client's reporting owner could agree before release on a first-close review: which reports and totals to reconcile, acceptable tolerances, how exceptions will be treated, and who can authorize broader use. The vendor retains the technical context and capacity to investigate; the client provides the business expertise, access, and decision authority needed to judge the results.

That agreement makes remaining uncertainty visible before deployment and gives the client a clear picture of how validation continues after go-live. The commercial detail—warranty boundaries, scope decisions when something new appears, and how completion is defined—belongs in the proposal, not in this article.

Preserve the capacity to act

A validation date on a calendar is of little use if everyone who understands the system has moved on by the time it arrives.

The plan needs to preserve budget, access to the relevant people, test environments, comparison data, and the reasoning behind earlier decisions. That does not require keeping a full project team idle. It does require knowing who will be available, what work they can undertake, and how decisions will be made.

If a working version arrives in weeks, we should make use of that speed. We should also recognize when part of the validation that once occurred during development now belongs later in the engagement, and budget for it from the beginning.

The question at release becomes:

What haven't we had the opportunity to learn yet, when will we be able to learn it, and have we preserved the capacity to act when we do?