# 500 integration tests later: what regression coverage actually caught

About a third of the suite has never found a bug. I know that because after a year of writing PHPUnit tests against attendance, overtime, leave and penalty rules, I went back through a year of failures to see which tests had ever failed for a reason worth knowing about, and the answer sorted itself into three piles very quickly.

There are five hundred of them because HR logic punishes guessing. Overtime depends on the shift, the shift depends on the calendar, the calendar has three kinds of holiday, and every company on the platform has an exception it considers too obvious to mention.

## Integration over unit, for this domain

Almost none of these are unit tests. They hit the database, run through the real service classes, and assert on the numbers a payroll export would show. Mocking the calendar would have made them faster and would also have removed the only part I did not trust.

# the loop that actually finds bugs: same case, every shift type
$ php artisan test --filter=Overtime
  PASS  Tests\Feature\OvertimeTest
  ✓ pays night differential on a rotating shift
  ✓ ignores minutes below the daily grace period
  ✓ stacks holiday rate over weekend rate

  Tests: 512 passed (1384 assertions)

## Which tests failed for a good reason

• ▸Date and boundary cases. Month ends, shifts crossing midnight, the week a country moved its weekend: this group caught real regressions over and over, and it is worth every second of runtime it costs.
• ▸Permission and visibility tests. Cheap to write, and they failed every time someone refactored a policy. Two of those failures would have been customer-visible data leaks.
• ▸Happy-path CRUD tests. Roughly a third of the suite, and I cannot point to one bug they found. They fail when the code changes, which is not the same as finding a bug.

## What I changed after looking

I stopped writing CRUD tests for new endpoints unless the endpoint has real rules behind it, and I started writing the boundary cases first, before the implementation, because that is where I get the requirements wrong. The suite grew slower after that, and caught more.

## The thirty I would write again

If the suite were deleted tomorrow I would not try to recreate five hundred tests. I would recreate about thirty, and I can name them, because they all came from the same rule: when a bug reaches production, the first commit of the fix is a failing test that reproduces it.

Those thirty are the only tests in the repository I can prove are load-bearing, because every one of them exists as a receipt for something that already broke in front of a customer. The other four hundred and seventy assert that I understood the requirements. Weaker claim.