When I first pasted a messy Orders API into an AI chat and typed "make this production-ready," I felt like I'd unlocked a cheat code. Clean names. Proper DI. Happy-path tests green in seconds.

But let's be honest - AI does not know what your code is supposed to do. It knows what code looks like. Those are different skills. I learned that the hard way (on a teaching demo I built on purpose - more on that honesty note below).

This article walks you through what I hit, the prompt that caused the damage, the better prompt that fixed it, and the checklist I now refuse to merge without.

---

In this article

• ▶ Run it - sample pack on GitHub: https://github.com/
• 📋 Steal the prompt - toolkit/Steal-These-Prompts.md in the same GitHub repo
• ✅ Checklist - toolkit/AI-Refactor-Review-Checklist.md in the same GitHub repo

---

🙋 Honesty first

The Orders API in this pack is a condensed teaching demo (a few hundred lines of Controllers-style ASP.NET Core 10). It is not a fake 10k-line production dump I "found in a drawer."

I built the quirks in so we can learn together. Same failure modes you'll hit on a real system - employee discounts, banker's rounding, shipped no-ops, off-by-one paging, auth stubs cleaned away - just small enough that you can download the pack, run it, and see the red/green yourself.

---

The scenario I started with

I inherited (okay - I wrote) a small but ugly Orders API:

• Poor names (OrdCtrl, DoStuff, StuffHelper, x1, tmp)
• Duplicated validation and totals math
• Fat controller with business rules inline
• Static helpers instead of DI
• Almost no tests
• Hidden rules that still matter (see samples/LegacyOrdersApi/README.md):

Rule	Behavior
A. Shipped + remove-line	Silent no-op - order unchanged
B. EMP- discount	Only if CustomerType == Internal; otherwise ignore (no error)
C. EU tax rounding	MidpointRounding.ToEven (banker's) to 2 decimals
D. Rate limit	Max 3 creates per customer id per minute

export PATH="$HOME/.dotnet:$PATH"
cd samples/LegacyOrdersApi
dotnet build
dotnet run --urls http://localhost:5080

Legacy ASP.NET Core Orders API build succeeding

📌 Tip: Install the .NET 10 SDK. If dotnet isn’t on your PATH after install, add it (on Linux/macOS that’s often export PATH="$HOME/.dotnet:$PATH"). Document the same SDK in CI.

---

Prompt v1 - the one that caused damage

Saved as samples/PROMPT_USED.md:

Refactor this ASP.NET Core Orders API for production readiness.
Improve naming, remove duplication, introduce proper DI, modern C# patterns,
add unit tests, and clean up anything that looks unused or outdated.
Preserve behavior.

That last sentence *sounds* responsible. Models are great at the first four bullets. They are not automatically great at the fifth - especially when "unused looking" code is load-bearing.

---

What the AI improved (I won't deny it)

The refactored tree (samples/AiRefactoredOrdersApi/) is genuinely nicer in several ways:

Naming and structure

// Before
public class OrdCtrl : ControllerBase { ... public IActionResult DoStuff(...) }

// After
public class OrdersController : ControllerBase { ... public async Task<IActionResult> Create(...) }

DI instead of static grab-bags

builder.Services.AddSingleton<IPricingService, PricingService>();
builder.Services.AddScoped<IOrderService, OrderService>();

Deduplicated pricing/totals into PricingService / OrderService.

Happy-path unit tests that pass and give you false confidence:

dotnet test samples/CorrectBehaviorTests --filter FullyQualifiedName~AiHappyPathTests
# Passed: 3

If you only look at structure and green happy paths… yeah. You would merge this. I almost did.

---

🚨 I almost merged this

Here's the beat I want you to feel, because I felt it.

Happy-path tests: green. Naming: OrdersController, IPricingService, IOrderService - looked senior. DI wired. Swagger still opened. Create-order returned 201.

I hovered over "Approve PR."

Then I ran the characterization / regression suite I had written against the legacy rules - and watched it light up red:

dotnet test samples/CorrectBehaviorTests \
  --filter "FullyQualifiedName~AiRegressionTests|FullyQualifiedName~AiSecurityRegressionTests"
# Failed: 6  ← that is the point

Regression tests failing on the AI-refactored Orders API

🎯 The lesson: green happy paths mean the wiring works. They do not mean EMP- on an external customer still behaves. I noticed EMP- applying to an External customer only because the characterization test said Expected 0, Actual 15.

Learn from my mistakes: write those tests before you ask the model to refactor - or you will accidentally encode the new (wrong) behavior.

---

What the AI got wrong (with live numbers)

Every mistake below is real code in the demo, caught by CorrectBehaviorTests.

1. "Simplified" the EMP- discount quirk

Legacy (StuffHelper.CalcDisc) - employee codes only for internal customers:

if (code.StartsWith("EMP-", StringComparison.OrdinalIgnoreCase))
{
    if (customerType != "Internal")
        return 0m; // ignore without error
    return Math.Round(sub * 0.15m, 2);
}

AI (PricingService.CalculateDiscount) - always applies 15%:

if (code.StartsWith("EMP-", StringComparison.OrdinalIgnoreCase))
    return Math.Round(subtotal * 0.15m, 2);

Before and after EMP- discount quirk: legacy vs AI simplification

📌 Live number AI can't fake: External customer + code EMP-SAVE on ₹100 subtotal → correct discount ₹0 → AI wrong discount ₹15

Finance notices weeks later. The model followed "clean up special cases" energy from the prompt - and violated "Preserve behavior."

Proof: AiRegressionTests.EmpDiscount_ExternalCustomer_MustBeIgnored_AIBreaksThis

---

2. Deleted EU banker's rounding as "dead code"

Legacy - looks redundant next to a plain Math.Round, but MidpointRounding differs on .5 edges:

if (region == "EU")
{
    // DO NOT delete - finance reconciliation depends on this
    return Math.Round(raw, 2, MidpointRounding.ToEven);
}
return Math.Round(raw, 2, MidpointRounding.AwayFromZero);

AI - one path for all regions:

return Math.Round(raw, 2, MidpointRounding.AwayFromZero);

Before and after EU banker’s rounding tax logic

📌 Live number AI can't fake: taxable 10.125, EU rate 20% → raw 2.025 → ToEven = 2.02 (correct) → AwayFromZero = 2.03 (AI)

Now multiply that 1-cent drift by × 50,000 invoices. Suddenly "dead looking" code is a reconciliation war room.

Proof: AiRegressionTests.EuTax_MustUseBankersRounding_AIBreaksThis

---

3. Subtle behavioral bugs while "cleaning up"

Pagination off-by-one - legacy is 1-based (Skip((page - 1) * size)). AI used Skip(page * size), so page=1 skips the first page.

Shipped orders mutated - legacy silent no-op when Status == "Shipped". AI compared to "shipped" (lowercase), so the guard never hits and lines get removed from shipped orders.

if (order.Status == "shipped") // never true for real "Shipped"
{
    return (order, null, 200);
}
// falls through and removes the line

Proof: RemoveLine_OnShippedOrder_MustBeSilentNoOp_AIBreaksThis, Pagination_Page1_MustReturnFirstItems_AIBreaksThis

---

4. Over-engineered abstractions for an in-memory list

AI introduced IRepository<T> + IUnitOfWork wrapping a static List<Order>. For this demo store that is ceremony without benefit.

Over-engineering is not a test failure; it is a review failure. Push back.

---

5. Security stub removed; internal fields still leaked

Legacy mutating endpoints required X-Api-Key: legacy-demo-key. AI deleted OkKey() as "outdated." Creates now return 201 with no key.

The model also kept CreditScoreHint on the public JSON shape - improved naming, same data leak.

Proof: AiSecurityRegressionTests

---

Prompt v2 - the one I wish I'd used first

Full text: samples/PROMPT_V2_FIXED.md (also side-by-side in toolkit/Steal-These-Prompts.md).

Refactor this ASP.NET Core Orders API for production readiness.

BEFORE changing production code:
1. Read and respect every characterization / regression test already in the repo.
2. Do not delete, merge, or "simplify" branches that have finance, auth, rate-limit,
   or "do not remove" comments - treat those as load-bearing.
3. If a branch looks redundant (e.g. MidpointRounding.ToEven vs AwayFromZero),
   KEEP IT unless a test proves it is dead.

Preserve these exact business quirks (non-negotiable):
- EMP- discount codes apply ONLY when CustomerType == "Internal".
  Example: External + EMP-SAVE on subtotal 100 → discount 0 (NOT 15).
- EU tax uses MidpointRounding.ToEven … raw 2.025 → EU 2.02 (not 2.03).
- Status == "Shipped" remove-line is a silent no-op.
- Pagination is 1-based: Skip((page - 1) * size).
- Mutating endpoints require X-Api-Key: legacy-demo-key.
- Do NOT expose CreditScoreHint / InternalNotes on public JSON - use a response DTO.

Allowed improvements: naming, DI, dedupe. Prefer simple services over generic
Repository + UnitOfWork for an in-memory list.
Preserve behavior. Prove it with the existing characterization suite.

Contrast: v1 vs v2 outcomes

	Prompt v1 → AiRefactoredOrdersApi	Prompt v2 → PromptFixedOrdersApi
Naming / DI	✅ nicer	✅ nicer
EMP- External ₹100	❌ ₹15	✅ ₹0
EU raw 2.025	❌ 2.03	✅ 2.02
Shipped no-op	❌ mutates	✅ silent no-op
Page 1	❌ skips first page	✅ first page
Auth header	❌ removed	✅ kept
creditScoreHint	❌ leaked	✅ stripped via OrderResponse
Regression suite	fails	passes

Prompt-fixed Orders API regression tests all passing

When I applied the better prompt (and kept characterization tests as the source of truth), I got clean structure without deleting the quirks. That's the whole game.

---

Testing AI's refactoring (step by step)

Layer	What to run	What it catches
Happy-path unit	AiHappyPathTests	Naming/wiring - *not* legacy quirks
AI regression	AiRegressionTests + AiSecurityRegressionTests	EMP-, EU rounding, shipped, paging, auth, leak
Prompt-fixed	PromptFixedRegressionTests + PromptFixedSecurityTests	Same quirks - expect pass
Legacy baseline	LegacyBehaviorTests	Original rules still hold
CI	.github/workflows/ci.yml	Legacy + prompt-fixed must pass; job ai-refactor-should-fail documents the lesson

export PATH="$HOME/.dotnet:$PATH"
cd dnw-ai-refactor-article   # folder from the sample pack (or your repo root)

# 1) Legacy - expect PASS
dotnet test samples/CorrectBehaviorTests --filter FullyQualifiedName~LegacyBehaviorTests

# 2) AI happy path - expect PASS (false confidence)
dotnet test samples/CorrectBehaviorTests --filter FullyQualifiedName~AiHappyPathTests

# 3) AI regressions - expect FAIL (that's the point)
dotnet test samples/CorrectBehaviorTests \
  --filter "FullyQualifiedName~AiRegressionTests|FullyQualifiedName~AiSecurityRegressionTests"

# 4) Prompt-fixed - expect PASS
dotnet test samples/CorrectBehaviorTests \
  --filter "FullyQualifiedName~PromptFixedRegressionTests|FullyQualifiedName~PromptFixedSecurityTests"

✅ Write characterization tests against legacy behavior before you ask the model to refactor.

---

✅ Practical checklist (steal from toolkit/)

Copy/paste the full version: toolkit/AI-Refactor-Review-Checklist.md in the same GitHub repo (https://github.com/YOUR_USER/YOUR_REPO, update this URL later).

Quick version:

1. Diff every deleted branch - finance comments, MidpointRounding, silent ignore
2. Search EMP- / partner codes / "ignore without error"
3. Compare status casing - "Shipped" vs "shipped"
4. Re-check paging - page=1 must be the first page
5. Auth still required on mutating endpoints?
6. Response DTOs - no creditScoreHint
7. Run regression on the AI output, not only AI's happy paths
8. Refuse Repository + UoW over one List<T> unless you have a second implementation
9. Keep "preserve behavior" measurable - characterization tests are that measurement
10. Prefer prompt v2 (PROMPT_V2_FIXED.md) once tests exist

Starter xUnit file: toolkit/Characterization-Test-Template.cs

---

Core lesson

AI is excellent at changing code. It is not automatically good at deciding which code should change.

When I treat model output like a junior PR - useful, fast, and never merge-ready without characterization tests - I sleep better. When I don't, External customers get employee pricing and EU finance starts Slack-paging me about cents.

---

🙋 FAQ

Q: Do I need to ban AI from refactors? A: No. I still use it for naming, DI wiring, and dumping duplication. I just don't let it be the source of truth for quirks.

Q: Why not only review the diff by eye? A: Because I almost merged this. Eyes love pretty names. Tests love boring numbers like ₹0 vs ₹15.

Q: Is PromptFixedOrdersApi "what the AI would produce with v2"? A: It's the teaching outcome of applying v2's constraints - good structure, quirks preserved, leak closed. Use the prompt on your own codebase; use this sample to verify your review habits.

Q: Where's CI? A: .github/workflows/ci.yml - builds the solution, runs legacy + prompt-fixed (must pass), and a separate job ai-refactor-should-fail that succeeds only when the AI regression suite fails as expected.

---

Reproduce this article's demo

export PATH="$HOME/.dotnet:$PATH"
cd dnw-ai-refactor-article

dotnet build OrdersApiDemo.sln
dotnet test samples/CorrectBehaviorTests --filter FullyQualifiedName~LegacyBehaviorTests
dotnet test samples/CorrectBehaviorTests --filter FullyQualifiedName~AiRegressionTests
dotnet test samples/CorrectBehaviorTests --filter FullyQualifiedName~PromptFixedRegressionTests

Files to open first:

• samples/PROMPT_USED.md - v1 (damage)
• samples/PROMPT_V2_FIXED.md - v2 (safer)
• samples/LegacyOrdersApi/Helpers/StuffHelper.cs - EMP- + EU rounding
• samples/AiRefactoredOrdersApi/Services/PricingService.cs - simplified (wrong)
• samples/PromptFixedOrdersApi/Services/PricingService.cs - quirks restored
• samples/CorrectBehaviorTests/ - the truth
• toolkit/ - checklist, template, steal-these-prompts

---

Your turn:

🎯 Which legacy quirk has AI deleted in your codebase - a silent no-op, a rounding mode, an auth stub, a 1-based page?

Drop it in the comments. I'm collecting stories for Part 2.