Adding Glean to your project

Before using Glean

Products using the Glean SDK to collect telemetry must:

  • add documentation for any new metric collected with the library in its repository (see an example);
  • go through data review for the newly collected data by following this process;
  • provide a way for users to turn data collection off (e.g. providing settings to control Glean.setUploadEnabled()).

Usage

Integrating with your project

Setting up the dependency

Glean is published on maven.mozilla.org. To use it, you need to add the following to your project's top-level build file, in the allprojects block (see e.g. Glean's own build.gradle):

repositories {
    maven {
       url "https://maven.mozilla.org/maven2"
    }
}

Each module that uses Glean needs to specify it in its build file, in the dependencies block. Add this to your Gradle configuration:

implementation "org.mozilla.components:service-glean:{latest-version}"

Important: the {latest-version} placeholder in the above link should be replaced with the version number of the Glean SDK used by the project.

For example, if version 6.0.2 is used, then the include directive becomes:

implementation "org.mozilla.components:service-glean:6.0.2"

The Glean SDK is released as part of android-components. Therefore, it follows android-components' versions. The android-components release page can be used to determine the latest version.

Setting up the dependency

Glean can be consumed through Carthage, a dependency manager for macOS and iOS. For consuming the latest version of Glean, add the following line to your Cartfile:

github "mozilla/glean" "{latest-version}"

Important: the {latest-version} placeholder should be replaced with the version number of the latest Glean SDK release. You can find the version number on the release page.

Then check out and build the new dependency:

carthage update --platform iOS

Integrating with the build system

For integration with the build system you can follow the Carthage Quick Start steps.

  1. After building the dependency one drag the built .framework binaries from Carthage/Build/iOS into your application's Xcode project.

  2. On your application targets' Build Phases settings tab, click the + icon and choose New Run Script Phase. If you already use Carthage for other dependencies, extend the existing step. Create a Run Script in which you specify your shell (ex: /bin/sh), add the following contents to the script area below the shell:

    /usr/local/bin/carthage copy-frameworks
    
  3. Add the path to the Glean framework under "Input Files":

    $(SRCROOT)/Carthage/Build/iOS/Glean.framework
    
  4. Add the paths to the copied framework to the "Output Files":

    $(BUILT_PRODUCTS_DIR)/$(FRAMEWORKS_FOLDER_PATH)/Glean.framework
    

It is recommended that you use a virtual environment for your work to isolate the dependencies for your project. There are many popular abstractions on top of virtual environments in the Python ecosystem which can help manage your project dependencies.

The Python Glean bindings currently have prebuilt wheels on PyPI for x86_64 Linux only.

If you're running that platform and have your virtual environment set up and activated, you can install Glean into it using:

$ python -m pip install glean_sdk

If you are not on x86_64 Linux, you will need to build the Glean Python bindings from source using these instructions.

Adding new metrics

All metrics that your project collects must be defined in a metrics.yaml file.

The format of that file is documented with glean_parser. To learn more, see adding new metrics.

Important: as stated before, any new data collection requires documentation and data-review. This is also required for any new metric automatically collected by the Glean SDK.

In order for the Glean SDK to generate an API for your metrics, two Gradle plugins must be included in your build:

The Glean Gradle plugin is distributed through Mozilla's Maven, so we need to tell your build where to look for it by adding the following to the top of your build.gradle:

buildscript {
    repositories {
        maven {
            url "https://maven.mozilla.org/maven2"
        }

        dependencies {
            classpath "org.mozilla.telemetry:glean-gradle-plugin:{latest-version}"
        }
    }
}

Important: as above, the {latest-version} placeholder in the above link should be replaced with the version number of the Glean SDK used by the project.

The JetBrains Python plugin is distributed in the Gradle plugin repository, so it can be included with:

plugins {
    id "com.jetbrains.python.envs" version "0.0.26"
}

Right before the end of the same file, we need to apply the Glean Gradle plugin:

apply plugin: "org.mozilla.telemetry.glean-gradle-plugin"

There are additional parameters that can be set to control the behavior of the Glean Gradle plugin, but they are rarely needed for normal use.

Note: Earlier versions of Glean used a Gradle script (sdk_generator.gradle) rather than a Gradle plugin. Its use is deprecated and projects should be updated to use the Gradle plugin as described above.

The metrics.yaml file is parsed at build time and Swift code is generated. Add a new metrics.yaml file to your Xcode project.

Follow these steps to automatically run the parser at build time:

  1. Download the sdk_generator.sh script from the Glean repository:

    https://raw.githubusercontent.com/mozilla/glean/{latest-release}/glean-core/ios/sdk_generator.sh
    

    Important: as above, the {latest-version} placeholder should be replaced with the version number of Glean SDK release used in this project.

  2. Add the sdk_generator.sh file to your Xcode project.

  3. On your application targets' Build Phases settings tab, click the + icon and choose New Run Script Phase. Create a Run Script in which you specify your shell (ex: /bin/sh), add the following contents to the script area below the shell:

    bash $PWD/sdk_generator.sh
    
  4. Add the path to your metrics.yaml and (optionally) pings.yaml under "Input files":

    $(SRCROOT)/{project-name}/metrics.yaml
    $(SRCROOT)/{project-name}/pings.yaml
    
  5. Add the paths to the generated code files to the "Output Files":

    $(SRCROOT)/{project-name}/Generated/{YourCategory}.swift
    

    Important: The parser generates one file per category. If you are unsure which files those are, skip this step and add the files after they are generated for the first time.

  6. If you are using Git, add the following line to your .gitignore file:

    {project-name}/Generated
    

Important: The Python bindings are already being used internally for testing of Mozilla's data platform. However, they are not ready for production or inclusion in applications that are distributed to end users. Progress on this support is being tracked in this bug.

For Python, the metrics.yaml file must be available and loaded at runtime.

If your project is a script (i.e. just Python files in a directory), you can load the metrics.yaml using:

from glean import load_metrics

metrics = load_metrics("metrics.yaml")

# Use a metric on the returned object
metrics.your_category.your_metric.set("value")

If your project is a distributable Python package, you need to include the metrics.yaml file using one of the myriad ways to include data in a Python package and then use pkg_resources.resource_filename() to get the filename at runtime.

from glean import load_metrics
from pkg_resources import resource_filename

metrics = load_metrics(resource_filename(__name__, "metrics.yaml"))

# Use a metric on the returned object
metrics.your_category.your_metric.set("value")

Adding custom pings

Please refer to the custom pings documentation.

Important: as stated before, any new data collection requires documentation and data-review. This is also required for any new metric automatically collected by the Glean SDK.

Testing metrics

In order to make testing metrics easier 'out of the box', all metrics include a set of test API functions in order to facilitate unit testing. These include functions to test whether a value has been stored, and functions to retrieve the stored value for validation. For more information, please refer to Unit testing Glean metrics.

Adding metadata about your project to the pipeline

In order for data to be collected from your project, its application id must be registered in the pipeline.

File a data engineering bug to enable your product's application id.