Upload mechanism

The glean-core Rust crate does not handle the ping upload directly. Network stacks vastly differ between platforms, applications and operating systems. The Glean SDK leverages the available platform capabilities to implement any network communication.

Glean core controls all upload and coordinates the platform side with its own internals. All language bindings implement ping uploading around a common API and protocol.

The upload module in the language bindings

classDiagram
    class UploadResult {
    }

    class HttpResponse {
        int statusCode
    }

    class UnrecoverableFailure {
    }

    class RecoverableFailure {
    }

    class PingUploader {
        <<interface>>
        +Upload() UploadResult
    }

    class BaseUploader {
        +BaseUploader(PingUploader)
        ~TriggerUploads()
        ~CancelUploads()
    }

    class HttpUploader {
        +Upload() UploadResult
    }

    UploadResult <|-- HttpResponse
    UploadResult <|-- UnrecoverableFailure
    UploadResult <|-- RecoverableFailure
    PingUploader <|-- HttpUploader
    PingUploader o-- BaseUploader

PingUploader

The PingUploader interface describes the contract between the BaseUploader and the SDK or user-provided upload modules.

BaseUploader

The BaseUploader component is responsible for interfacing with the lower level get_upload_task calls and dealing with the logic in a platform-coherent way.

• Every Glean instance will always have a single BaseUploader instance.
• The BaseUploader is fed, at Glean initialization, with an instance of an implementation of the PingUploader interface.
• Whenever BaseUploader thinks it should perform an upload, it will call the provided instance of the PingUploader interface and call upload with the data it's getting from Glean.
• Any throttling happens at this layer: the core will orchestrate the throttling, while this layer will be responsible to abide to what the core is telling it to do.
• Any logic for aborting uploads or triggering uploads is provided by this object.

HttpClientUploader

The HttpClientUploader is the default SDK-provided HTTP uploader. It acts as an adapter between the platform-specific upload library and the Glean upload APIs.

Note that most of the languages have now diverged, due to the many iterations, from this design. For example, in Kotlin, the BaseUploader is mostly empty and its functionalities are spread in the PingUploadWorker.

Upload task API

The following diagram visualizes the communication between Glean core (the Rust crate), a Glean language binding (e.g. the Kotlin or Swift implementation) and a Glean end point server.

sequenceDiagram
    participant Glean core
    participant Glean wrapper
    participant Server

    Glean wrapper->>Glean core: get_upload_task()
    Glean core->>Glean wrapper: Task::Upload(PingRequest)
    Glean wrapper-->>Server: POST /submit/{task.id}
    Server-->>Glean wrapper: 200 OK
    Glean wrapper->>Glean core: upload_response(200)
    Glean wrapper->>Glean core: get_upload_task()
    Glean core->>Glean wrapper: Task::Done

Glean core will take care of file management, cleanup, rescheduling and rate limiting¹.

Available APIs

#![allow(unused)]
fn main() {
/// Gets the next task for an uploader.
fn get_upload_task(&self) -> PingUploadTask

/// Processes the response from an attempt to upload a ping.
fn process_ping_upload_response(&self, uuid: &str, status: UploadResult)
}

/// Gets the next task for an uploader.
PingUploadTask glean_get_upload_task();

/// Processes the response from an attempt to upload a ping.
void glean_process_ping_upload_response(string uuid, UploadResult result);