Engineering Council Test Reliability Report

Executive Snapshot

29

Daily Runs

12/29

Daily Green

23m 14s

Avg Daily Runtime

66

Smoke Attempts

48/66

Smoke Green

3m 10s

Avg Smoke Runtime

3m 04s

Median Smoke Time

1

Current Green Streak

Council Narrative

Daily suite reliability finished strong: the last 1 daily runs were all green.
In the last 10 daily runs, total test volume grew from 1,183 to 1,189.
Library has the highest strict failure ratio at 0.87%, while Billing has the broadest non-pass footprint at 1.54% and showed up in 9 failed runs.
The average daily-suite runtime, measured from GitLab start and finish timestamps, was 23m 14s.
Smoke runs stayed fast when healthy: average duration was 3m 10s and the median passing run was 3m 04s.
3 production pipelines clearly recovered on rerun after an initial smoke failure: 145743, 147232, 147447.
5 failed smoke attempts never reached test execution in GitLab; they are counted as failed attempts, with per-test counts marked unavailable.

Category Failure Ratios

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

Category Aggregate Table

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

Category	Total	Failed	Pending	Skipped	Failure Ratio	Non-pass Ratio	Runs With Failures
Billing	2526	39	0	0	1.54%	1.54%	9
Web	12661	20	0	7	0.16%	0.21%	6
Frontend	4812	17	6	0	0.35%	0.48%	6
Library	1486	13	0	7	0.87%	1.35%	4

Billing

Pend 0Skip 0Runs 9

39

1.54%

2526

Web

Pend 0Skip 7Runs 6

20

0.16%

0.21%

12661

Frontend

Pend 6Skip 0Runs 6

17

0.35%

0.48%

4812

Library

Pend 0Skip 7Runs 4

13

0.87%

1.35%

1486

Recent Runs