The resume fix that changed nothing

We built a browser test for the product loop RezScore is supposed to deliver.

A person uploads a resume, sees the important problems, targets it to a job, reviews a grounded change, applies it, sees the score improve, and reaches export. The test uses a fake AI provider and a disposable database, but it drives the real buttons, pages, revisions, scoring code, and paywall.

The first serious run made it through almost the whole path. Upload returned 200. The target proposal returned 201. Preview rendered. Apply returned 200. Scoring returned 200.

The score before the edit was 56.34. The score after the edit was also 56.34.

Every service had done what we asked. The product had not done what we promised.

A perfectly grounded no-op

The proposed change was not invented. Our deterministic provider copied this existing bullet from the resume:

Led a 12-person launch program that cut onboarding time by 30%.

It placed the sentence in the professional subheadline and attached an exact evidence reference to the original work-experience bullet. The proposal validator confirmed that the words, number, and meaning came from the source. The preview showed the same change the server later applied. Apply created one new Builder revision. The score service evaluated that exact revision.

The edit was truthful. It was also irrelevant to the rubric.

Our current scoring projection includes the subheadline because it is visible on the resume, but the active scoring categories do not award points for it. The headline affects positioning and top-of-page signal. The subheadline does not. Moving a good bullet into an unscored field changed the document without changing the score.

This is why an API test could not settle the release question. Every API contract was correct in isolation. Only the full browser path asserted the claim that tied them together: after a person accepts the proposed fix, the product can show a real improvement.

The tempting fix was to weaken the test

We could have changed the assertion from “after is greater than before” to “a score response exists.” That would have made the suite green immediately.

It also would have changed the product contract. A completed score request is not evidence of improvement. It only proves that the scorer ran.

We kept the strict assertion and changed the fixture instead. The test resume now starts without a professional headline. Its work history already contains the title Operations Manager. The deterministic proposal copies that exact title into the headline and cites /work_experience/0/title as its evidence.

No new claim is created. The work title was already in the source. The edit fixes a real top-of-page problem, and the rubric scores the field it changes. A focused test now evaluates the actual Builder projection before and after the patch and requires the score to increase.

That is a better fixture because it represents the promise under test. It begins with a specific, fixable problem. The proposed change uses existing evidence. The result affects the same rubric the user sees.

Grounding and usefulness are separate checks

This failure clarified a distinction that AI product work often blurs.

Grounding asks whether the system has the right to say something. Each number, employer, title, and result must trace to the person’s resume, and the saved words must match the proposal the person approved.

Usefulness asks whether the change solves the problem the product identified. The destination field must affect a dimension we actually measure, such as scanability, evidence, positioning, or credibility, and the product must be able to explain why the new version is better.

A change can be grounded and useless. It can also be useful-sounding and ungrounded. A safe transformation loop needs both tests.

We already had strong machinery for the first one: canonical source Builders, exact revisions, evidence references, durable proposals, idempotent Apply, and revision-bound scoring. The browser failure showed that the second check cannot be inferred from those controls.

A funnel needs semantic checkpoints

The same lesson appeared later in the flow.

Our free export path returns a paywall before it generates a file. That means the truthful sequence is export intent, checkout, successful charge, then completed export. Calling the pre-checkout request an export would make the funnel look complete when the user had only asked for a file.

Likewise, calling a score request “improvement” would turn activity into outcome. Calling an Apply click “saved” would ignore whether the transaction committed. Calling checkout-session completion “revenue” would ignore whether a charge succeeded.

The event names are not clerical details. Each one is a claim about what happened. The release test is useful because it forces those claims to line up with visible browser state and durable server state.

Where the work stands

The focused provider, fixture, and scoring tests now pass with the grounded headline change. The full browser release gate still needs another run, and the separate hard-kill journey has not passed yet. Organic rollout remains at zero.

That is the honest stopping point for this post. We found a failure, rejected the easy green check, and made the test case represent a real improvement. The next browser run gets to decide whether the complete product loop agrees.