False success flag

• An integration server sat between the factory floor and an ERP, pushing operations across tens of sites, on the order of thousands of rows a day.
• For each operation it wrote a status row and flipped a flag to "success" the moment the request was sent, not when the committed result came back.
• So a row could read "done" while the real operation downstream had actually failed.
• Logs didn't help reconcile: lines were sometimes missing, sometimes the same operation logged twice.
• Root cause: success measured at send time, plus non-idempotent logging that broke on retries.
• Fix: set the flag from the actual committed downstream result, and make logging idempotent so one operation maps to one reconcilable log line no matter how many retries.

// BUG: trusted HTTP 200 as success
if (resp.statusCode() == 200) markSuccess();   // ERP returned 200 with an error body!

// FIX: read the ERP business result, not the transport status
var r = parse(resp.body());
if (resp.statusCode() == 200 && "S".equals(r.resultCode)) markSuccess();
else markFailed(r.message);                     // a 200 is not a business success

• ·We had an integration server between the factory floor and an ERP, doing thousands of rows a day across tens of sites.
• ·The status table marked rows as success, but it set that flag when the request was sent, not when the result came back committed.
• ·So rows said done while the real operation had failed, and the table and the floor disagreed.
• ·The logs couldn't save us either, lines were sometimes missing and sometimes duplicated, so neither source could be trusted alone.
• ·The fix was to flip the flag only on the committed downstream result and make logging idempotent, one operation, one reconcilable line.

• Define success as the confirmed committed downstream result, never as "request sent" or "request accepted."
• Make logging idempotent with a stable operation key so retries produce exactly one reconcilable line, not missing or duplicated entries.
• Add a reconciliation check that compares the status table against actual downstream state so disagreements surface automatically instead of operation by operation.