Memory Distribution

Memory distributions are used to accumulate and store memory sizes.

Memory distributions are recorded in a histogram where the buckets have an exponential distribution, specifically with 16 buckets for every power of 2. That is, the function from a value \( x \) to a bucket index is:

\[ \lfloor 16 \log_2(x) \rfloor \]

This makes them suitable for measuring memory sizes on a number of different scales without any configuration.

Note Check out how this bucketing algorithm would behave on the Simulator.

Recording API

`accumulate`

Accumulates the provided sample in the metric.

import org.mozilla.yourApplication.GleanMetrics.Memory

fun allocateMemory(nbytes: Int) {
    // ...
    Memory.heapAllocated.accumulate(nbytes / 1024)
}

import org.mozilla.yourApplication.GleanMetrics.Memory;

fun allocateMemory(nbytes: Int) {
    // ...
    Memory.INSTANCE.heapAllocated().accumulate(nbytes / 1024);
}

import Glean

func allocateMemory(nbytes: UInt64) {
    // ...
    Memory.heapAllocated.accumulate(nbytes / 1024)
}

from glean import load_metrics
metrics = load_metrics("metrics.yaml")

def allocate_memory(nbytes):
    # ...
    metrics.memory.heap_allocated.accumulate(nbytes / 1024)

use glean_metrics::memory;

fn allocate_memory(bytes: u64) {
    // ...
    memory::heap_allocated.accumulate(bytes / 1024);
}

import * as memory from "./path/to/generated/files/memory.js";

function allocateMemory() {
    // ...
    memory.heapAllocated.accumulate(nbytes / 1024);
}

C++

#include "mozilla/glean/GleanMetrics.h"

mozilla::glean::memory::heap_allocated.Accumulate(bytes / 1024);

JavaScript

Glean.memory.heapAllocated.accumulate(bytes / 1024);

Recorded errors

invalid_value: If recording a negative memory size.
invalid_value: If recording a size larger than 1 TB.

Testing API

`testGetValue`

Gets the recorded value for a given memory distribution metric.
Returns a struct with counts per buckets and total sum if data is stored.
Returns a language-specific empty/null value if no data is stored. Has an optional argument to specify the name of the ping you wish to retrieve data from, except in Rust where it's required. None or no argument will default to the first value found for send_in_pings.

import org.mozilla.yourApplication.GleanMetrics.Memory

// Get snapshot
val snapshot = Memory.heapAllocated.testGetValue()

// Does the sum have the expected value?
assertEquals(11, snapshot.sum)

// Usually you don't know the exact memory values,
// but how many should have been recorded.
assertEquals(2L, snapshot.count)

import org.mozilla.yourApplication.GleanMetrics.Memory;

// Get snapshot
val snapshot = Memory.INSTANCE.heapAllocated().testGetValue();

// Does the sum have the expected value?
assertEquals(11, snapshot.sum);

// Usually you don't know the exact memory values,
// but how many should have been recorded.
assertEquals(2L, snapshot.getCount());

// Get snapshot
let snapshot = try! Memory.heapAllocated.testGetValue()

// Does the sum have the expected value?
XCTAssertEqual(11, snapshot.sum)

// Usually you don't know the exact memory values,
// but how many should have been recorded.
XCTAssertEqual(2, snapshot.count)

from glean import load_metrics
metrics = load_metrics("metrics.yaml")

# Get snapshot.
snapshot = metrics.memory.heap_allocated.test_get_value()

# Does the sum have the expected value?
assert 11 == snapshot.sum

# Usually you don't know the exact memory values,
# but how many should have been recorded.
assert 2 == snapshot.count

use glean::ErrorType;
use glean_metrics::memory;

// Get snapshot
let snapshot = memory::heap_allocated.test_get_value(None).unwrap();

// Does the sum have the expected value?
assert_eq!(11, snapshot.sum);

// Usually you don't know the exact timing values,
// but how many should have been recorded.
assert_eq!(2, snapshot.count);

import * as memory from "./path/to/generated/files/memory.js";

// Get snapshot
const snapshot = await memory.heapAllocated.testGetValue();

// Does the sum have the expected value?
assert.equal(11, snapshot.sum);

// Usually you don't know the exact memory values,
// but know how many should have been recorded.
assert.equal(2, snapshot.count);

C++

#include "mozilla/glean/GleanMetrics.h"

// Does it have an expected values?
const data = mozilla::glean::memory::heap_allocated.TestGetValue().value().unwrap()
ASSERT_EQ(11 * 1024, data.sum);

JavaScript

const data = Glean.memory.heapAllocated.testGetValue();
Assert.equal(11 * 1024, data.sum);

`testGetNumRecordedErrors`

Gets the number of errors recorded for a given memory distribution metric.

import org.mozilla.yourApplication.GleanMetrics.Memory

// Did this record a negative value?
assertEquals(
    0,
    Memory.heapAllocated.testGetNumRecordedErrors(ErrorType.INVALID_VALUE)
)

import org.mozilla.yourApplication.GleanMetrics.Memory;

// Assert that no errors were recorded.
assertEquals(
    0,
    Memory.INSTANCE.heapAllocated().testGetNumRecordedErrors(ErrorType.INVALID_VALUE)
);

// Did this record a negative value?
XCTAssertEqual(0, Memory.heapAllocated.testGetNumRecordedErrors(.invalidValue))

from glean import load_metrics
metrics = load_metrics("metrics.yaml")

# Did this record a negative value?
assert 0 == metrics.memory.heap_allocated.test_get_num_recorded_errors(
    ErrorType.INVALID_VALUE
)

use glean::ErrorType;
use glean_metrics::pages;

assert_eq!(
    0,
    pages::page_load.test_get_num_recorded_errors(ErrorType::InvalidValue)
);

import * as memory from "./path/to/generated/files/memory.js";
import { ErrorType } from "@mozilla/glean/<platform>";

// Did this record a negative value?
assert.equal(
    0,
    await memory.heapAllocated.testGetNumRecordedErrors(ErrorType.InvalidValue)
);

Metric parameters

Example memory distribution metric definition:

memory:
  heap_allocated:
    type: memory_distribution
    memory_unit: kilobyte
    description: >
      The heap memory allocated
    bugs:
      - https://bugzilla.mozilla.org/000000
    data_reviews:
      - https://bugzilla.mozilla.org/show_bug.cgi?id=000000#c3
    notification_emails:
      - me@mozilla.com
    expires: 2020-10-01

Extra metric parameters

`memory_unit`

Memory distributions have an optional memory_unit parameter, which specifies the unit the incoming memory size values are recorded in. The allowed values for memory_unit are:

byte (default)
kilobyte (= 2^10 = 1,024 bytes)
megabyte (= 2^20 = 1,048,576 bytes)
gigabyte (= 2^30 = 1,073,741,824 bytes)

Limits

The maximum memory size that can be recorded is 1 Terabyte (2⁴⁰ bytes). Larger sizes will be truncated to 1 Terabyte.

Data questions

What is the distribution of the size of heap allocations?

Generate uniformly distributed data

Use custom data

Properties

Memory unit (memory_unit)

Note The data provided, is assumed to be in the configured memory unit. The data recorded, on the other hand, is always in bytes. This means that, if the configured memory unit is not byte, the data will be transformed before being recorded. Notice this, by using the select field above to change the memory unit and see the mean of the data recorded changing.

Glean SDKs - Cross-platform Telemetry Libraries