Glean Pings

A ping is a bundle of related metrics, gathered in a payload to be transmitted. The ping payload will be encoded in JSON format and contains one or more of the common sections with shared information data.

If data collection is enabled, the Glean SDK provides a set of built-in pings that are assembled out of the box without any developer intervention.

Ping sections

Every ping has the following keys at the top-level:

  • The ping_info section contains core metadata that is included in every ping.

  • The client_info section contains information that identifies the client. It is included in most pings (including all built-in pings), but may be excluded from pings where we don't want to connect client information with the other metrics in the ping.

  • The metrics section contains the submitted values for all metric types except for events. It has keys for each of the metric types, under which is data for each metric.

  • The events section contains the events recorded in the ping.

See the payload documentation for more details for each metric type in the metrics and events section.

The ping_info section

The following fields are included in the ping_info section, for every ping. Optional fields are marked accordingly.

Field nameTypeDescription
seqCounterA running counter of the number of times pings of this type have been sent
experimentsObjectOptional. A dictionary of active experiments
start_timeDatetimeThe time of the start of collection of the data in the ping, in local time and with minute precision, including timezone information.
end_timeDatetimeThe time of the end of collection of the data in the ping, in local time and with minute precision, including timezone information. This is also the time this ping was generated and is likely well before ping transmission time.
reasonStringOptional. The reason the ping was submitted. The specific set of values and their meanings are defined for each metric type in the reasons field in the pings.yaml file.

All the metrics surviving application restarts (e.g. seq, ...) are removed once the application using the Glean SDK is uninstalled.

The experiments object

This object (included in the ping_info section) contains experiment annotations keyed by the experiment id. Each annotation contains the experiment branch the client is enrolled in and may contain a string to string map with additional data in the extra key. Both the id and branch are truncated to 30 characters. See Using the Experiments API on how to record experiments data.

{
  "<id>": {
    "branch": "branch-id",
    "extra": {
      "some-key": "a-value"
    }
  }
}

The client_info section

The following fields are included in the client_info section. Optional fields are marked accordingly.

Field nameTypeDescription
app_buildStringThe build identifier generated by the CI system (e.g. "1234/A"). For language bindings that provide automatic detection for this value, (e.g. Android/Kotlin), in the unlikely event that the build identifier can not be retrieved from the OS, it is set to inaccessible. For other language bindings, if the value was not provided through configuration, this metric gets set to Unknown.
app_channelStringOptional The product-provided release channel (e.g. "beta")
app_display_versionStringThe user-visible version string (e.g. "1.0.3"). The meaning of the string (e.g. whether semver or a git hash) is application-specific. In the unlikely event this value can not be obtained from the OS, it is set to "inaccessible". If it is accessible, but not set by the application, it is set to "Unknown".
architectureStringThe architecture of the device (e.g. "arm", "x86")
client_idUUIDOptional A UUID identifying a profile and allowing user-oriented correlation of data
device_manufacturerStringOptional The manufacturer of the device
device_modelStringOptional The model name of the device. On Android, this is Build.MODEL, the user-visible name of the device.
first_run_dateDatetimeThe date of the first run of the application, in local time and with day precision, including timezone information.
osStringThe name of the operating system (e.g. "linux", "Android", "ios")
os_versionStringThe user-visible version of the operating system (e.g. "1.2.3")
android_sdk_versionStringOptional. The Android specific SDK version of the software running on this hardware device (e.g. "23")
telemetry_sdk_buildStringThe version of the Glean SDK
localeStringOptional. The locale of the application during initialization (e.g. "es-ES"). If the locale can't be determined on the system, the value is "und", to indicate "undetermined".

All the metrics surviving application restarts (e.g. client_id, ...) are removed once the application using the Glean SDK is uninstalled.

Ping submission

The pings that the Glean SDK generates are submitted to the Mozilla servers at specific paths, in order to provide additional metadata without the need to unpack the ping payload.

A typical submission URL looks like

"<server-address>/submit/<application-id>/<doc-type>/<glean-schema-version>/<document-id>"

where:

  • <server-address>: the address of the server that receives the pings;
  • <application-id>: a unique application id, automatically detected by the Glean SDK; this is the value returned by Context.getPackageName();
  • <doc-type>: the name of the ping; this can be one of the pings available out of the box with the Glean SDK, or a custom ping;
  • <glean-schema-version>: the version of the Glean ping schema;
  • <document-id>: a unique identifier for this ping.

Limitations

To keep resource usage in check, the Glean SDK enforces some limitations on ping uploading and ping storage.

  • Rate limiting: only up to 10 ping submissions every 60 seconds are allowed. There are no exposed methods to change these rate limiting defaults yet, follow Bug 1647630 for updates.
  • Request body size limiting: the body of a ping request may have up to 1MB. Pings that exceed this size are discarded and don't get uploaded. Size and number of discarded pings are recorded on the internal Glean metric glean.upload.discarded_exceeding_pings_size.
  • Storage quota: Pending pings are stored on disk. The storage directory is scanned every time Glean is initialized and upon scanning Glean checks its size. If this directory exceeds a size of 10MB or 250 pending ping files, pings are deleted to get the directory back to an accepted size. Pings are deleted oldest first, until the directory size is below the quota. The number of deleted pings due to exceeding storage quota is recorded on the metric glean.upload.deleted_pings_after_quota_hit and the size of the pending pings directory is recorded (regardless on whether quota has been reached) on the metric glean.upload.pending_pings_directory_size Note Deletion request pings are stored in a different directory, are not subject to this limitation and never get deleted.

Submitted headers

A pre-defined set of headers is additionally sent along with the submitted ping:

HeaderValueDescription
Content-Typeapplication/json; charset=utf-8Describes the data sent to the server
User-AgentDefaults to e.g. Glean/0.40.0 (Kotlin on Android), where 0.40.0 is the Glean SDK version number and Kotlin on Android is the name of the language used by the binding that sent the request plus the name of the platform it is running on.Describes the application sending the ping using the Glean SDK
Datee.g. Mon, 23 Jan 2019 10:10:10 GMT+00:00Submission date/time in GMT/UTC+0 offset
X-Client-TypeGleanCustom header to support handling of Glean pings in the legacy pipeline
X-Client-Versione.g. 0.40.0The Glean SDK version, sent as a custom header to support handling of Glean pings in the legacy pipeline
X-Debug-IDOptional, e.g. test-tagDebug header attached to Glean pings when using the debug tools
X-Source-TagsOptional, e.g. automation, perfA list of tags to associate with the ping, useful for clustering pings at analysis time, for example to tell data generated from CI from other data.