Billing
Pend 0Skip 0Runs 3
3
0.40%
0.40%
756
Executive Snapshot
7
Daily Runs
0/7
Daily Green
10m 11s
Avg Daily Runtime
25
Smoke Attempts
19/25
Smoke Green
5m 08s
Avg Smoke Runtime
4m 40s
Median Smoke Time
0
Current Green Streak
Executive Analysis
Bottom line: release confidence is unstable in both the broad regression path and the deploy smoke path. The immediate job is to separate real product regressions from execution noise, then burn down the concentrated failure clusters.
What Matters
- Daily regression passed 0 of 7 runs (0.0%), with a current green streak of 0 and a best streak of 0 in this window. The latest daily run (161005) failed, so the system is ending the week under tension rather than in a clean state.
- Smoke passed 19 of 25 attempts (76.0%) across 22 production pipelines. 1 pipeline(s) recovered on rerun, which is useful for continuity but also a sign that first-pass deploy signal is noisier than it should be.
- Failure concentration is not random: Social has the highest strict failure ratio at 8.73%, while Social has the broadest non-pass footprint at 8.73%.
- Frontend is the weakest smoke surface in this window at 16/22 green (72.7%).
- Daily-suite runtime averaged 10m 11s, while observed daily test volume moved from 1,281 to 1,304.
Engineering Analysis
- A release gate should fail loudly for product regressions and quietly for infrastructure noise. Rerun recoveries plus incomplete daily or smoke attempts suggest those two failure modes are still partially mixed together.
- The failure profile is concentrated enough to act on. Social and Social are carrying the strongest signal, which means reliability work should be assigned by category ownership instead of treating the suite as one undifferentiated problem.
- The broader daily suite is carrying more instability than smoke, which usually means product regressions are escaping into wider coverage areas even when the narrow deploy gate looks acceptable.
Recommended Actions
- Assign one owner to Social for the next cycle and expect a short written burn-down: top failing tests, suspected root causes, flake versus regression breakdown, and what gets fixed or quarantined first.
- Treat the daily regression suite like an operations queue until it is calm again: triage failures after each red run, close known-noise items fast, and avoid letting multiple unrelated red signals pile up between runs.
- Put Frontend smoke under closer guardrails for the next release cycle. It is the best place to improve first-pass deploy confidence quickly.
Improvement Ideas
- Introduce a small reliability budget for tests: every flaky or quarantined case needs an owner and an expiry, and the team should review that budget weekly the same way it reviews bugs or incidents.
- Track first-fail to root-cause time as a core metric. Fast diagnosis is as important as raw pass rate because the practical value of a test gate depends on how quickly it helps the team recover.
- Define a runtime budget per suite and require justification when test count or duration grows. Reliable feedback systems stay trusted when they remain both stable and proportionate.
Category Execution Ratios
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
Strict Failure Ratio
Share of category executions that ended in failed across all daily runs in this window.
Billing0.40%
Web0.26%
Frontend1.73%
Library0.00%
University0.00%
Subscriptions0.00%
Admission0.00%
Social8.73%
Non-pass Ratio
Share of category executions that ended in failed, pending, or skipped across all daily runs in this window.
Billing0.40%
Web0.26%
Frontend2.57%
Library0.00%
University0.00%
Subscriptions0.00%
Admission0.00%
Social8.73%
Category Aggregate Table
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
How computed
Category total executions means the sum of that category's observed test executions across every daily-suite run in the selected window.
Strict Failure Ratio = failed executions for that category divided by total executions for that category across the window.
Non-pass Ratio = (failed + pending + skipped) executions for that category divided by total executions for that category across the window.
Example: if Billing executed 800 times across the week and 2 of those executions failed, Billing strict failure ratio is 0.25%. That does not mean 0.25% of pipelines failed; it means 0.25% of observed Billing executions ended in failed.
| Category | Total | Failed | Pending | Skipped | Failure Ratio | Non-pass Ratio | Runs With Failures |
|---|---|---|---|---|---|---|---|
| Billing | 756 | 3 | 0 | 0 | 0.40% | 0.40% | 3 |
| Web | 5397 | 14 | 0 | 0 | 0.26% | 0.26% | 5 |
| Frontend | 2143 | 37 | 0 | 18 | 1.73% | 2.57% | 6 |
| Library | 602 | 0 | 0 | 0 | 0.00% | 0.00% | 0 |
| University | 21 | 0 | 0 | 0 | 0.00% | 0.00% | 0 |
| Subscriptions | 7 | 0 | 0 | 0 | 0.00% | 0.00% | 0 |
| Admission | 7 | 0 | 0 | 0 | 0.00% | 0.00% | 0 |
| Social | 126 | 11 | 0 | 0 | 8.73% | 8.73% | 1 |
CatFailF%NP%Tot
Web
Pend 0Skip 0Runs 5
14
0.26%
0.26%
5397
Frontend
Pend 0Skip 18Runs 6
37
1.73%
2.57%
2143
Library
Pend 0Skip 0Runs 0
0
0.00%
0.00%
602
University
Pend 0Skip 0Runs 0
0
0.00%
0.00%
21
Subscriptions
Pend 0Skip 0Runs 0
0
0.00%
0.00%
7
Admission
Pend 0Skip 0Runs 0
0
0.00%
0.00%
7
Social
Pend 0Skip 0Runs 1
11
8.73%
8.73%
126
Recent Runs
Recent Daily Suite Runs
| Date | Pipeline | Suites | Status | Summary |
|---|---|---|---|---|
| 2026-06-13 18:12 | 160133 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1281 | Passed 1279 | Failed 2 |
| 2026-06-14 18:12 | 160138 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1281 | Passed 1280 | Failed 1 |
| 2026-06-15 18:12 | 160308 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1281 | Passed 1276 | Failed 5 |
| 2026-06-16 18:13 | 160489 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1304 | Passed 1285 | Failed 12 |
| 2026-06-17 18:13 | 160727 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1304 | Passed 1288 | Failed 11 |
| 2026-06-18 18:14 | 160902 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1304 | Passed 1295 | Failed 9 |
| 2026-06-19 18:14 | 161005 | BillingWebFrontendLibraryUniversitySubscriptionsAdmissionSocial | FAILED | Total 1304 | Passed 1273 | Failed 25 |
FAILED
T 1281 | P 1279 | F 2 | Pend 0
FAILED
T 1281 | P 1280 | F 1 | Pend 0
FAILED
T 1281 | P 1276 | F 5 | Pend 0
FAILED
T 1304 | P 1285 | F 12 | Pend 0
FAILED
T 1304 | P 1288 | F 11 | Pend 0
FAILED
T 1304 | P 1295 | F 9 | Pend 0
FAILED
T 1304 | P 1273 | F 25 | Pend 0
Recent Smoke Attempts
| Date | Suite | Pipeline | Job | Status | Passed | Failed | Duration |
|---|---|---|---|---|---|---|---|
| 2026-06-15 13:04 | Frontend | 160207 | Frontend smoke | FAILED | 85 | 25 | 13m 19s |
| 2026-06-15 17:38 | University | 160307 | University smoke | PASSED | 60 | 0 | 3m 57s |
| 2026-06-15 17:42 | Frontend | 160307 | Frontend smoke | PASSED | 110 | 0 | 4m 57s |
| 2026-06-16 10:59 | Frontend | 160350 | Frontend smoke | PASSED | 110 | 0 | 4m 34s |
| 2026-06-16 13:00 | Frontend | 160382 | Frontend smoke | FAILED | 30 | 3 | 5m 10s |
| 2026-06-16 14:25 | University | 160415 | University smoke | PASSED | 60 | 0 | 3m 12s |
| 2026-06-16 14:28 | Frontend | 160415 | Frontend smoke | FAILED | 108 | 2 | 5m 15s |
| 2026-06-16 14:41 | Frontend | 160418 | Frontend smoke | FAILED | 108 | 2 | 5m 38s |
| 2026-06-16 15:15 | Frontend | 160437 | Frontend smoke | FAILED | 108 | 2 | 5m 29s |
| 2026-06-16 16:40 | Frontend | 160472 | Frontend smoke | PASSED | 110 | 0 | 4m 34s |
| 2026-06-16 17:37 | Frontend | 160487 | Frontend smoke | PASSED | 110 | 0 | 4m 30s |
| 2026-06-16 17:41 | University | 160487 | University smoke | PASSED | 60 | 0 | 3m 30s |
| 2026-06-16 18:43 | Frontend | 160491 | Frontend smoke | PASSED | 110 | 0 | 4m 54s |
| 2026-06-17 14:53 | Frontend | 160654 | Frontend smoke | PASSED | 110 | 0 | 4m 47s |
| 2026-06-17 16:30 | Frontend | 160702 | Frontend smoke | PASSED | 110 | 0 | 4m 36s |
| 2026-06-17 23:51 | Frontend | 160732 | Frontend smoke | PASSED | 110 | 0 | 4m 46s |
| 2026-06-18 16:14 | Frontend | 160868 | Frontend smoke | FAILED | 101 | 9 | 7m 44s |
| 2026-06-18 16:27 | Frontend | 160875 | Frontend smoke | PASSED | 110 | 0 | 4m 32s |
| 2026-06-18 17:11 | Frontend | 160882 | Frontend smoke | PASSED | 110 | 0 | 4m 40s |
| 2026-06-18 18:19 | Frontend | 160900 | Frontend smoke | PASSED | 110 | 0 | 4m 55s |
| 2026-06-18 18:31 | Frontend | 160904 | Frontend smoke | PASSED | 110 | 0 | 4m 40s |
| 2026-06-18 20:46 | Frontend | 160908 | Frontend smoke | PASSED | 110 | 0 | 4m 35s |
| 2026-06-19 09:40 | Frontend | 160911 | Frontend smoke | PASSED | 110 | 0 | 4m 42s |
| 2026-06-19 16:51 | Frontend | 161003 | Frontend smoke | PASSED | 110 | 0 | 4m 41s |
| 2026-06-19 21:05 | Frontend | 161014 | Frontend smoke | PASSED | 110 | 0 | 4m 45s |
FAILED
P 85 | F 25 | 13m 19s
PASSED
P 60 | F 0 | 3m 57s
PASSED
P 110 | F 0 | 4m 57s
PASSED
P 110 | F 0 | 4m 34s
FAILED
P 30 | F 3 | 5m 10s
PASSED
P 60 | F 0 | 3m 12s
FAILED
P 108 | F 2 | 5m 15s
FAILED
P 108 | F 2 | 5m 38s
FAILED
P 108 | F 2 | 5m 29s
PASSED
P 110 | F 0 | 4m 34s
PASSED
P 110 | F 0 | 4m 30s
PASSED
P 60 | F 0 | 3m 30s
PASSED
P 110 | F 0 | 4m 54s
PASSED
P 110 | F 0 | 4m 47s
PASSED
P 110 | F 0 | 4m 36s
PASSED
P 110 | F 0 | 4m 46s
FAILED
P 101 | F 9 | 7m 44s
PASSED
P 110 | F 0 | 4m 32s
PASSED
P 110 | F 0 | 4m 40s
PASSED
P 110 | F 0 | 4m 55s
PASSED
P 110 | F 0 | 4m 40s
PASSED
P 110 | F 0 | 4m 35s
PASSED
P 110 | F 0 | 4m 42s
PASSED
P 110 | F 0 | 4m 41s
PASSED
P 110 | F 0 | 4m 45s
Smoke Suite Breakdown
Frontend
22 attempts across 22 pipelines
Passed16
Failed6
Incomplete0
Avg runtime5m 21s
Median passing runtime4m 41s
Pipelines22
University
3 attempts across 3 pipelines
Passed3
Failed0
Incomplete0
Avg runtime3m 33s
Median passing runtime3m 30s
Pipelines3