Billing
Pend 0Skip 98Runs 2
11
1.13%
11.21%
972
Executive Snapshot
9
Daily Runs
2/9
Daily Green
21m 48s
Avg Daily Runtime
16
Smoke Attempts
14/16
Smoke Green
3m 14s
Avg Smoke Runtime
3m 09s
Median Smoke Time
1
Current Green Streak
Executive Analysis
Bottom line: the regression system is informative but not calm. The data suggest repeatable problem areas rather than random breakage, which means focused ownership should move the needle quickly.
What Matters
- Daily regression passed 2 of 9 runs (22.2%), with a current green streak of 1 and a best streak of 1 in this window.
- Smoke passed 14 of 16 attempts (87.5%) across 9 production pipelines. 2 pipeline(s) recovered on rerun, which is useful for continuity but also a sign that first-pass deploy signal is noisier than it should be.
- Failure concentration is not random: Frontend has the highest strict failure ratio at 3.56%, while Billing has the broadest non-pass footprint at 11.21%.
- University is the weakest smoke surface in this window at 4/5 green (80.0%).
- Daily-suite runtime averaged 21m 48s.
Engineering Analysis
- A release gate should fail loudly for product regressions and quietly for infrastructure noise. Rerun recoveries plus incomplete daily or smoke attempts suggest those two failure modes are still partially mixed together.
- The failure profile is concentrated enough to act on. Frontend and Billing are carrying the strongest signal, which means reliability work should be assigned by category ownership instead of treating the suite as one undifferentiated problem.
- The broader daily suite is carrying more instability than smoke, which usually means product regressions are escaping into wider coverage areas even when the narrow deploy gate looks acceptable.
Recommended Actions
- Assign one owner to Frontend for the next cycle and expect a short written burn-down: top failing tests, suspected root causes, flake versus regression breakdown, and what gets fixed or quarantined first.
- Treat the daily regression suite like an operations queue until it is calm again: triage failures after each red run, close known-noise items fast, and avoid letting multiple unrelated red signals pile up between runs.
- Put University smoke under closer guardrails for the next release cycle. It is the best place to improve first-pass deploy confidence quickly.
Improvement Ideas
- Introduce a small reliability budget for tests: every flaky or quarantined case needs an owner and an expiry, and the team should review that budget weekly the same way it reviews bugs or incidents.
- Track first-fail to root-cause time as a core metric. Fast diagnosis is as important as raw pass rate because the practical value of a test gate depends on how quickly it helps the team recover.
- Define a runtime budget per suite and require justification when test count or duration grows. Reliable feedback systems stay trusted when they remain both stable and proportionate.
Category Execution Ratios
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
Strict Failure Ratio
Share of category executions that ended in failed across all daily runs in this window.
Billing1.13%
Web1.98%
Frontend3.56%
Library2.97%
Non-pass Ratio
Share of category executions that ended in failed, pending, or skipped across all daily runs in this window.
Billing11.21%
Web11.11%
Frontend10.64%
Library11.11%
Category Aggregate Table
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
| Category | Total | Failed | Pending | Skipped | Failure Ratio | Non-pass Ratio | Runs With Failures |
|---|---|---|---|---|---|---|---|
| Billing | 972 | 11 | 0 | 98 | 1.13% | 11.21% | 2 |
| Web | 7020 | 139 | 0 | 641 | 1.98% | 11.11% | 1 |
| Frontend | 2388 | 85 | 0 | 169 | 3.56% | 10.64% | 6 |
| Library | 774 | 23 | 0 | 63 | 2.97% | 11.11% | 1 |
CatFailF%NP%Tot
Web
Pend 0Skip 641Runs 1
139
1.98%
11.11%
7020
Frontend
Pend 0Skip 169Runs 6
85
3.56%
10.64%
2388
Library
Pend 0Skip 63Runs 1
23
2.97%
11.11%
774
Recent Runs
Recent Daily Suite Runs
| Date | Pipeline | Suites | Status | Summary |
|---|---|---|---|---|
| 2026-05-17 18:27 | 156183 | BillingWebFrontendLibrary | FAILED | Total 1255 | Passed 1252 | Failed 3 |
| 2026-05-18 18:26 | 156473 | BillingWebFrontendLibrary | FAILED | Total 1255 | Passed 1250 | Failed 5 |
| 2026-05-19 18:28 | 156720 | BillingWebFrontendLibrary | FAILED | Total 1255 | Passed 1250 | Failed 5 |
| 2026-05-20 18:28 | 156907 | BillingWebFrontendLibrary | FAILED | Total 1255 | Passed 1252 | Failed 3 |
| 2026-05-21 18:24 | 157062 | BillingWebFrontendLibrary | PASSED | Total 1208 | Passed 1208 | Failed 0 |
| 2026-05-22 18:11 | 157199 | BillingWebFrontendLibrary | FAILED | Total 1208 | Passed 0 | Failed 237 |
| 2026-05-23 18:24 | 157201 | BillingWebFrontendLibrary | FAILED | Total 1208 | Passed 1207 | Failed 1 |
| 2026-05-24 18:25 | 157212 | BillingWebFrontendLibrary | PASSED | Total 1255 | Passed 1255 | Failed 0 |
FAILED
T 1255 | P 1252 | F 3 | Pend 0
FAILED
T 1255 | P 1250 | F 5 | Pend 0
FAILED
T 1255 | P 1250 | F 5 | Pend 0
FAILED
T 1255 | P 1252 | F 3 | Pend 0
PASSED
T 1208 | P 1208 | F 0 | Pend 0
FAILED
T 1208 | P 0 | F 237 | Pend 0
FAILED
T 1208 | P 1207 | F 1 | Pend 0
PASSED
T 1255 | P 1255 | F 0 | Pend 0
Recent Smoke Attempts
| Date | Suite | Pipeline | Job | Status | Passed | Failed | Duration |
|---|---|---|---|---|---|---|---|
| 2026-05-18 15:02 | Frontend | 156306 | Frontend smoke | PASSED | 110 | 0 | 3m 08s |
| 2026-05-18 17:08 | Frontend | 156455 | Frontend smoke | PASSED | 110 | 0 | 3m 12s |
| 2026-05-18 22:21 | Frontend | 156479 | Frontend smoke | PASSED | 110 | 0 | 3m 09s |
| 2026-05-18 23:10 | Frontend | 156481 | Frontend smoke | PASSED | 110 | 0 | 3m 20s |
| 2026-05-19 12:54 | Frontend | 156608 | Frontend smoke | FAILED | 80 | 1 | 5m 01s |
| 2026-05-19 12:59 | Frontend | 156608 | Frontend smoke | PASSED | 110 | 0 | 4m 02s |
| 2026-05-20 23:05 | Frontend | 156608 | Frontend smoke | PASSED | 110 | 0 | 3m 31s |
| 2026-05-21 14:43 | University | 157003 | University smoke | PASSED | 60 | 0 | 2m 13s |
| 2026-05-21 14:50 | Frontend | 157003 | Frontend smoke | PASSED | 110 | 0 | 3m 49s |
| 2026-05-21 16:13 | University | 157030 | University smoke | PASSED | 60 | 0 | 2m 13s |
| 2026-05-21 16:19 | Frontend | 157030 | Frontend smoke | PASSED | 110 | 0 | 3m 06s |
| 2026-05-22 15:37 | University | 157139 | University smoke | PASSED | 60 | 0 | 2m 10s |
| 2026-05-22 15:41 | Frontend | 157139 | Frontend smoke | PASSED | 110 | 0 | 3m 08s |
| 2026-05-22 16:56 | University | 157198 | University smoke | FAILED | 59 | 1 | 3m 22s |
| 2026-05-22 16:56 | Frontend | 157198 | Frontend smoke | PASSED | 110 | 0 | 3m 53s |
| 2026-05-22 16:59 | University | 157198 | University smoke | PASSED | 60 | 0 | 2m 25s |
PASSED
P 110 | F 0 | 3m 08s
PASSED
P 110 | F 0 | 3m 12s
PASSED
P 110 | F 0 | 3m 09s
PASSED
P 110 | F 0 | 3m 20s
FAILED
P 80 | F 1 | 5m 01s
PASSED
P 110 | F 0 | 4m 02s
PASSED
P 110 | F 0 | 3m 31s
PASSED
P 60 | F 0 | 2m 13s
PASSED
P 110 | F 0 | 3m 49s
PASSED
P 60 | F 0 | 2m 13s
PASSED
P 110 | F 0 | 3m 06s
PASSED
P 60 | F 0 | 2m 10s
PASSED
P 110 | F 0 | 3m 08s
FAILED
P 59 | F 1 | 3m 22s
PASSED
P 110 | F 0 | 3m 53s
PASSED
P 60 | F 0 | 2m 25s
Smoke Suite Breakdown
Frontend
11 attempts across 9 pipelines
Passed10
Failed1
Incomplete0
Avg runtime3m 34s
Median passing runtime3m 16s
Pipelines9
University
5 attempts across 4 pipelines
Passed4
Failed1
Incomplete0
Avg runtime2m 28s
Median passing runtime2m 13s
Pipelines4