Metrics for Relying Parties
Last updated: March 24th, 2023
Subscriptions Services Data Sources
Subscription Platform provides product metrics in BigQuery by obtaining subscriptions data from the following sources: Stripe, mobile app store providers, and FxA logs.
Stripe
The bulk of Subscriptions Services data comes from Stripe and is synced to BigQuery using Fivetran and its Stripe connector. This data reflects the current subscription state of all active subscriptions paid for via Stripe or PayPal. It also includes information regarding subscriptions and customers, as well as products, plans, invoices, cards, charges, discounts, coupons, and promotion codes.
Some of the customer information includes the customer's postal code, state, and country for a subscription, which helps resolve issues for customers across a particular dimension (e.g., country, payment provider, etc.). However, as it intentionally excludes any personal identifiable information of customers (e.g., name, street address, phone number, email address), it can be challenging to use to help on an individual customer level. Other uses for this data is for retrieving all-time counts (e.g., total number of active subscriptions).
Fivetran preserves historical data within subscription_history by comparing the current state of the subscription with the most recent record in the history table and adding a new record if they vary. However, there are limitations with the Fivetran Stripe sync, as it only records historical state/changes for top-level subscription fields and not the associated subscription items which indicate the subscribed plan. As we have to rely on the subscription metadata fields previous_plan_id and plan_change_date, which only records the most recent plan change, we can potentially miss/lose history if there are any problems with the Fivetran sync.
The Stripe Firestore DB was set up in October 2021. As of February 2023, Subscription Platform also syncs Stripe Firestore Customer and Subscriptions collections in BigQuery, including the latest data and changelogs.
Mobile app store providers
Subscriptions Services data is also obtained from two mobile app store providers: Apple and Google. This includes current and historical IAP subscription states, as well as information, such as date of billing and expired dates, subscription change event, etc. Apple provides limited data - it does not provide the country, price, or currency of subscriptions.
As of December 2022, Subscription Platform supports Apple IAP subscriptions, and stores current subscription state from that point forward in the Firestore database and synced in BigQuery.
Similar to Apple, Subscription Platform also handles the current state of Google IAP subscriptions, which are also stored in Firestore and synced in BigQuery.
These collections of IAP subscriptions synced in BigQuery each have changelogs that also get synced into a BigQuery table. These changelogs contain the historical state for IAP subscriptions - there is a BigQuery view for each that represents the current state of the collection calculated based on the latest changelog for each document. Since the historical state and changes are only recorded by the BigQuery sync itself, there are only records of the changes that occurred after the BigQuery syncs were set up.
- Google IAP sync started 2021-10-18
- Apple IAP sync started 2022-12-15
The general use of this data is to understand what the subscription state was over time, as well as sequences of events.
FxA logs
FxA logs include account and subscription management event data that can help provide insights regarding the customers' journey through the FxA sign-in/sign-up process. It can also help identify potential drivers (e.g., call-to-action, medium, paid search term, etc.) that directed customers to a page for acquisition and attribution purposes, if query parameters are set up properly (see below for more information).
Relying-Party Hosted Email Form
Reliers must do either one of the following when integrating with Mozilla accounts: 0. Self-host the first step in the FxA authentication flow themselves (e.g. the form capturing the user's email) 0. Send users to a FxA-hosted form at https://accounts.firefox.com/.
In the first case, when the email entry form is hosted by the relying party, the relying party must:
- When the page that hosts the FxA form loads, have it make an XHR call to
https://accounts.firefox.com/metrics-flow. The domain name of the request should match the FxA page that is being redirected to (e.g. https://accounts.firefox.com). You can usefetchto get this info. - Include the following query parameters in the above request (see chart below for descriptions):
entrypointentrypoint_experimententrypoint_variationutm_sourceutm_campaignform_type- An example:
https://accounts.firefox.com/metrics-flow?entrypoint=my_page&utm_source=my_referrer&utm_campaign=my_campaign&form_type=email
- The response from
metrics-flowwill be a JSON object that contains the fieldsflowIdandflowBeginTime. These values will need to be propagated to FxA as query parameters, which can be done using hidden form fields with the namesflow_idandflow_begin_time. You can see an example of how the about:welcome page does this by looking here.
Following these instructions will provide FxA and the relying party with the data needed to ensure a healthy user flow.
Relying-Party "Engage" Events
Note: this is a limited, temporary solution for cross-product metrics that is due to be replaced in early 2020. Please contact the FxA team if you think you need access.
The metrics that the Mozilla accounts platform sends to Amplitude reflect mainly direct interactions with FxA. These are mostly authentication events (registering, logging in, etc) or events related to account management (e.g. changes to a user’s account settings). This means that interaction events within “relying” products of FxA (such as Firefox Monitor) that do not involve authentication are not logged to the FxA amplitude metrics system. To address this shortcoming, FxA-relying products can log metrics about product usage directly via the FxA metrics system. FxA has not previously allowed for the direct logging of these types of metrics by relying products, but we feel that this change is necessary to ensure that company-level metrics accurately reflect product usage.
Only one event is allowed per RP, and the required query parameters are different from other requests:
- When the event of interest occurs, the RP server (not the user's browser) should submit a GET request to
https://accounts.firefox.com/metrics-flowwith theOriginheader set to the RP's registered FxA OAuth domain. - Include the following query parameters in the request:
event_type- the static string “engage” - this ping tells us that a user engaged with a service in some way that we’ve defined out of bandservice- the oauth client identifier for the RP, this is an opaque 8-byte hex string that isn’t privateuid- the Mozilla accounts user id - this is an opaque hex string that identifies the user across all FxA relying parties. Here, it’s the user who has engaged with the service in some way. (In the future, we plan to replace this with an anonymous / pseudonymous identifier supplied by ecosystem telemetry)
Note that the RP's domain needs to be manually added to the FxA allowed_metrics_flow_origins list before these events will be accepted. Otherwise, they will be silently dropped.
Metrics-Related Query Parameters
The values that are passed in the parameters below are subject to validation via regular expressions. If the parameter values do not conform to their associated regexes in this file then all metrics events associated with the non-conforming parameters will not be logged!
| Name | Description | Example Values | Validation regex | Amplitude Chart Example |
|---|---|---|---|---|
entrypoint | The specific page or browser UI element containing the first step of the FxA sign-in/sign-up process (e.g., enter email form) | firstrun | ^[\w.:-]+$ | Firstrun form views |
entrypoint_experiment | Identifier for the experiment the user is part of (if any) | ^[\w.:-]+$ | ||
entrypoint_variation | Identifier for the experiment variation the user is part of (if any) | ^[\w.:-]+$ | ||
form_type | For self-hosted forms only (see above) the type of form that the user submits to begin the FxA flow | either: email if the form captures the user's email, otherwise other | NA | |
utm_source | Unambiguous identifier of site or browser UI element that linked to the page containing the beginning of the FxA sign-in/sign-up process | blog.mozilla.org | ^[\w/.%-]+$ | Registration form views segmented by utm_source |
utm_campaign | More general label for the campaign that the site is part of | firstrun | ^[\w/.%-]+$ | TBD |
utm_content | Used to track the name of an A-B test | my-experiment | ^[\w/.%-]+$ | TBD |
utm_term | Used to track the cohort or variation in an A-B test | my-experiment-var-a | ^[\w/.%-]+$ | TBD |
utm_medium | What type of link was used to direct to the page, if it came through a marketing campaign | email, cpc | ^[\w/.%-]+$ | TBD |
context | Not relevant to metrics, but this is required to be set to one of fx_desktop_v3, fx_fennec_v1 or fx_ios_v1 if service=sync. Please use the value that reflects the most likely operating system of the user. | fx_desktop_v3, fx_fennec_v1, fx_ios_v1 | ^[0-9a-z_-]+$/ | NA |
Note these may not be all the parameters you need to pass for your integration to work! A more exhaustive but less detailed list can be found here. What is documented above are only the parameters that are relevant for metrics analysis in (e.g.) amplitude.
Other Notes:
-
You must have access to the mozilla amplitude account to see the example charts. If you are a Mozilla employee, please contact Leif for information on gaining access to amplitude.
-
Regarding
utm_term: note that the current usage of this parameter is different from what is typical. In most scenarios, it is used to track the search terms that led the users to the page. If you would like to use the parameter in this way, please inform the Mozilla accounts team.
User Opt-out
Users may opt-out of metrics from the FxA settings page. If they do, FxA will not collect or store metrics and relying parties must not either.
User profiles include the metricsEnabled boolean. When the value is false relying parties should not collect any metrics tied to the user. Relying parties should check this value every time the profile is requested.