Pings

A ping is a bundle of related metrics, gathered in a payload to be transmitted. The ping payload is encoded in JSON format and contains one or more of the common sections with shared information data.

If data collection is enabled, the chosen Glean SDK may provide a set of built-in pings that are assembled out of the box without any developer intervention.

Table of contents

Payload structure

Every ping payload has the following keys at the top-level:

  • The ping_info section contains core metadata that is included in every ping.

  • The client_info section contains information that identifies the client. It is included in most pings (including all built-in pings), but may be excluded from pings where we don't want to connect client information with the other metrics in the ping.

The following keys are only present if any metrics or events were recorded for the given ping:

  • The metrics section contains the submitted values for all metric types except for events. It has keys for each of the metric types, under which is data for each metric.

  • The events section contains the events recorded in the ping.

See the payload documentation for more details for each metric type in the metrics and events section.

The ping_info section

Metadata about the ping itself. This section is included in every ping.

The following fields are included in the ping_info section. Optional fields are marked accordingly.

seq

Type: Counter, Lifetime: User

A running counter of the number of times pings of this type have been sent.

start_time

Type: Datetime, Lifetime: User

The time of the start of collection of the data in the ping, in local time and with minute precision, including timezone information.

end_time

Type: Datetime, Lifetime: Ping

The time of the end of collection of the data in the ping, in local time and with minute precision, including timezone information. This is also the time this ping was generated and is likely well before ping transmission time.

reason (optional)

The reason the ping was submitted. The specific set of values and their meanings are defined for each metric type in the reasons field in the pings.yaml file.

experiments (optional)

A dictionary of active experiments.

This object contains experiment annotations keyed by the experiment id. Each annotation contains the experiment branch the client is enrolled in and may contain a string to string map with additional data in the extra key.

Both the id and branch are truncated to 30 characters. See Using the Experiments API on how to record experiments data.

{
  "<id>": {
    "branch": "branch-id",
    "extra": {
      "some-key": "a-value"
    }
  }
}

The client_info section

A limited amount of metrics that are generally useful across products. The data is provided by the embedding application or automatically fetched by the Glean SDK. It is collected at initialization time and sent in every ping afterwards. For historical reasons it contains metrics that are only useful on a certain platform.

Additional metrics require a proposal

Adding new metrics maintained by the Glean SDKs team will require a full proposal and details on why that value is useful across multiple platforms and products and needs Glean SDKs team ownership.

The Glean SDKs are not taking ownership of new metrics that are platform- or product-specific.

The following fields are included in the client_info section. Optional fields are marked accordingly.

app_build

Type: String, Lifetime: Application

The build identifier generated by the CI system (e.g. "1234/A"). If the value was not provided through configuration, this metric gets set to Unknown.

app_channel (optional)

Type: String, Lifetime: Application

The product-provided release channel (e.g. "beta").

app_display_version

Type: String, Lifetime: Application

The user-visible version string (e.g. "1.0.3"). The meaning of the string (e.g. whether semver or a git hash) is application-specific. If the value was not provided through configuration, this metric gets set to Unknown.

build_date (optional)

Type: Datetime, Lifetime: Application

architecture

Type: String, Lifetime: Application

The architecture of the device (e.g. "arm", "x86").

client_id (optional)

Type: UUID, Lifetime: User

A UUID identifying a profile and allowing user-oriented correlation of data.

device_manufacturer (optional)

Type: String, Lifetime: Application

The manufacturer of the device the application is running on. Not set if the device manufacturer can't be determined (e.g. on Desktop).

device_model (optional)

Type: String, Lifetime: Application

The model of the device the application is running on. On Android, this is Build.MODEL, the user-visible marketing name, like "Pixel 2 XL". Not set if the device model can't be determined (e.g. on Desktop).

first_run_date

Type: Datetime, Lifetime: User

The date of the first run of the application, in local time and with day precision, including timezone information.

os

Type: String, Lifetime: Application

The name of the operating system (e.g. "Linux", "Android", "iOS").

os_version

Type: String, Lifetime: Application

The user-visible version of the operating system (e.g. "1.2.3"). If the version detection fails, this metric gets set to Unknown.

android_sdk_version (optional)

Type: String, Lifetime: Application

The Android specific SDK version of the software running on this hardware device (e.g. "23").

telemetry_sdk_build

Type: String, Lifetime: Application

The version of the Glean SDK.

locale (optional)

Type: String, Lifetime: Application

The locale of the application during initialization (e.g. "es-ES"). If the locale can't be determined on the system, the value is "und", to indicate "undetermined".

Ping submission

The pings that the Glean SDKs generate are submitted to the Mozilla servers at specific paths, in order to provide additional metadata without the need to unpack the ping payload.

URL

A typical submission URL looks like

"<server-address>/submit/<application-id>/<doc-type>/<glean-schema-version>/<document-id>"

where:

  • <server-address>: the address of the server that receives the pings;
  • <application-id>: a unique application id, automatically detected by the Glean SDK; this is the value returned by Context.getPackageName();
  • <doc-type>: the name of the ping; this can be one of the pings available out of the box with the Glean SDK, or a custom ping;
  • <glean-schema-version>: the version of the Glean ping schema;
  • <document-id>: a unique identifier for this ping.

Limitations

To keep resource usage in check, the Glean SDK enforces some limitations on ping uploading and ping storage.

Rate limiting

Only up to 15 ping submissions every 60 seconds are allowed.

For the JavaScript SDK that limit is higher and up to 40 ping submissions every 60 seconds are allowed.

Request body size limiting

The body of a ping request may have up to 1MB (after compression). Pings that exceed this size are discarded and don't get uploaded. Size and number of discarded pings are recorded on the internal Glean metric glean.upload.discarded_exceeding_pings_size.

Storage quota

Pending pings are stored on disk. Storage is scanned every time Glean is initialized and upon scanning Glean checks its size. If it exceeds a size of 10MB or 250 pending pings, pings are deleted to get the storage back to an accepted size. Pings are deleted oldest first, until the storage size is below the quota.

The number of deleted pings due to exceeding storage quota is recorded on the metric glean.upload.deleted_pings_after_quota_hit and the size of the pending pings directory is recorded (regardless on whether quota has been reached) on the metric glean.upload.pending_pings_directory_size.

Deletion request pings are not subject to this limitation and never get deleted.

Submitted headers

A pre-defined set of headers is additionally sent along with the submitted ping.

Content-Type

Describes the data sent to the server. Value is always application/json; charset=utf-8.

Date

Submission date/time in GMT/UTC+0 offset, e.g. Mon, 23 Jan 2019 10:10:10 GMT+00:00.

User-Agent (deprecated)

Up to Glean v44.0.0 and Glean.js v0.13.0 this contained the Glean SDK version and platform information. Newer Glean SDKs do not overwrite this header. See X-Telemetry-Agent for details.

Clients might still send it, for example, when sending pings from browsers it will contain the characteristic browser UA string.

This header is parsed by the Glean pipeline and can be queried at analysis time through the metadata.user_agent.* fields in the ping tables.

X-Telemetry-Agent

The Glean SDK version and platform this ping is sent from. Useful for debugging purposes when pings are sent to the error stream. as it describes the application and the Glean SDK used for sending the ping.

It's looks like Glean/40.0.0 (Kotlin on Android), where 40.0.0 is the Glean Kotlin SDK version number and Kotlin on Android is the name of the language used by the SDK that sent the request plus the name of the platform it is running on.

X-Debug-Id (optional)

Debug header attached to Glean pings by using the debug APIs, e.g. test-tag.

When this header is present, the ping is redirected to the Glean Debug View.

X-Source-Tags (optional)

A list of tags to associate with the ping, useful for clustering pings at analysis time, for example to tell data generated from CI from other data e.g. automation, perf.

This header is attached to Glean pings by using the debug APIs.