Project Onboarding Procedure
Below are step-by-step instructions on how to onboard a project to report test metrics.
Prerequisites
To report test metrics for a project, ensure the following requirements are met:
- Test results must be in JUnit format.
- Coverage results must be in JSON format.
- Supported test frameworks are listed in the Metric Interpretation Guide.
1. Setup CICD to push test result and coverage data to GCS
- Create a directory for the repository in the ecosystem-test-eng-metrics GCP Cloud Bucket.
- Set up a service account for the project with
Storage Object Creator
andStorage Object Viewer
permissions. Store the credentials in the ETE 1Password vault. - Update project CICD jobs to push Coverage JSON files and JUnit XML files to the GCS repository
directory, under
coverage
andjunit
subdirectories respectively. - Coverage JSON files must follow a strict naming convention:
{job_number}__{utc_epoch_datetime}__{repository}__{workflow}__{test_suite}__coverage.xml
- Example:
15592__1724283071__autopush-rs__build-test-deploy__integration__coverage.json
- Example:
- JUnit XML files must follow a strict naming convention:
{job_number}__{utc_epoch_datetime}__{repository}__{workflow}__{test_suite}__results{-index}.xml
- Example:
15592__1724283071__autopush-rs__build-test-deploy__integration__results.xml
- The index is optional and can be used in cases of parallel test execution
- Example:
2. Create and Populate Tables in the ETE BigQuery Dataset
- Create Tables:
- For test results, create one empty table named
{project_name}_results
. - For coverage results, create one empty table named
{project_name}_coverage
. - These tables should be created in the
test_metrics
dataset of the ETE BigQuery instance. Reference the official documentation to create empty tables with schema definitions. Schemas can be copied from existing project tables.
- For test results, create one empty table named
- Populate Tables:
- Execute the following command to populate the tables with data:
make run_metric_reporter
- Execute the following command to populate the tables with data:
3. Create a Looker Dashboard
-
License Requirement: A developer license is required to create and edit dashboards in Looker. Instructions for obtaining a license and resources for learning Looker are available on the Mozilla Confluence. Additional help can be found in the
#data-help
and#looker-platform-discussion
Slack channels. -
Update Looker Project:
- Update the ecosystem-test-eng Looker project Looker project model and add the required views for the new project test data. Related repository: looker-ecosystem-test-eng.
-
Create Dashboard: Create a new dashboard, populate it with looks for the new project data, and add it to the ETE Test Metrics Board.