# Memory Distribution

Memory distributions are used to accumulate and store memory sizes.

Memory distributions are recorded in a histogram where the buckets have an exponential distribution, specifically with 16 buckets for every power of 2. That is, the function from a value $$x$$ to a bucket index is:

$\lfloor 16 \log_2(x) \rfloor$

This makes them suitable for measuring memory sizes on a number of different scales without any configuration.

Note Check out how this bucketing algorithm would behave on the Simulator

## Configuration

Memory distributions have a required memory_unit parameter, which specifies the unit the incoming memory size values are recorded in. The units are the power-of-2 units, so "kilobyte" is more correctly a "kibibyte".

- kilobyte == 2^10 ==         1,024 bytes
- megabyte == 2^20 ==     1,048,576 bytes
- gigabyte == 2^30 == 1,073,741,824 bytes


If you wanted to create a memory distribution to measure the amount of heap memory allocated, first you need to add an entry for it to the metrics.yaml file:

memory:
heap_allocated:
type: memory_distribution
description: >
The heap memory allocated
memory_unit: kilobyte
...


## API

Now you can use the memory distribution from the application's code.

Note The data provided to the accumulate method is in the configured memory unit specified in the metrics.yaml file. The data recorded, on the other hand, is always in bytes.

For example, to measure the distribution of heap allocations:

import org.mozilla.yourApplication.GleanMetrics.Memory

fun allocateMemory(nbytes: Int) {
// ...
Memory.heapAllocated.accumulate(nbytes / 1024)
}


There are test APIs available too. For convenience, properties sum and count are exposed to facilitate validating that data was recorded correctly.

Continuing the heapAllocated example above, at this point the metric should have a sum == 11 and a count == 2:

import org.mozilla.yourApplication.GleanMetrics.Memory

// Was anything recorded?
assertTrue(Memory.heapAllocated.testHasValue())

// Get snapshot
val snapshot = Memory.heapAllocated.testGetValue()

// Does the sum have the expected value?
assertEquals(11, snapshot.sum)

// Usually you don't know the exact memory values, but how many should have been recorded.
assertEquals(2L, snapshot.count)

// Did this record a negative value?
assertEquals(1, Memory.heapAllocated.testGetNumRecordedErrors(ErrorType.InvalidValue))

func allocateMemory(nbytes: UInt64) {
// ...
Memory.heapAllocated.accumulate(nbytes / 1024)
}


There are test APIs available too. For convenience, properties sum and count are exposed to facilitate validating that data was recorded correctly.

Continuing the heapAllocated example above, at this point the metric should have a sum == 11 and a count == 2:

@testable import Glean

// Was anything recorded?
XCTAssert(Memory.heapAllocated.testHasValue())

// Get snapshot
let snapshot = try! Memory.heapAllocated.testGetValue()

// Does the sum have the expected value?
XCTAssertEqual(11, snapshot.sum)

// Usually you don't know the exact memory values, but how many should have been recorded.
XCTAssertEqual(2, snapshot.count)

// Did this record a negative value?
XCTAssertEqual(1, Memory.heapAllocated.testGetNumRecordedErrors(.invalidValue))

from glean import load_metrics

def allocate_memory(nbytes):
# ...
metrics.memory.heap_allocated.accumulate(nbytes / 1024)


There are test APIs available too. For convenience, properties sum and count are exposed to facilitate validating that data was recorded correctly.

Continuing the heapAllocated example above, at this point the metric should have a sum == 11 and a count == 2:

# Was anything recorded?

# Get snapshot
snapshot = metrics.memory.heap_allocated.test_get_value()

# Does the sum have the expected value?
assert 11 == snapshot.sum

# Usually you don't know the exact memory values, but how many should have been recorded.
assert 2 == snapshot.count

# Did this record a negative value?
assert 1 == metrics.memory.heap_allocated.test_get_num_recorded_errors(
ErrorType.INVALID_VALUE
)

using static Mozilla.YourApplication.GleanMetrics.Memory;

fun allocateMemory(ulong nbytes) {
// ...
Memory.heapAllocated.Accumulate(nbytes / 1024);
}


There are test APIs available too. For convenience, properties Sum and Count are exposed to facilitate validating that data was recorded correctly.

Continuing the heapAllocated example above, at this point the metric should have a Sum == 11 and a Count == 2:

using static Mozilla.YourApplication.GleanMetrics.Memory;

// Was anything recorded?
Assert.True(Memory.heapAllocated.TestHasValue());

// Get snapshot
var snapshot = Memory.heapAllocated.TestGetValue();

// Does the sum have the expected value?
Assert.Equal(11, snapshot.Sum);

// Usually you don't know the exact memory values, but how many should have been recorded.
Assert.Equal(2L, snapshot.Count);

// Did this record a negative value?
Assert.Equal(1, Memory.heapAllocated.TestGetNumRecordedErrors(ErrorType.InvalidValue));


#![allow(unused)]
fn main() {
use glean_metrics;

fn allocate_memory(bytes: u64) {
memory::heap_allocated.accumulate(bytes / 1024);
}
}


There are test APIs available too:


#![allow(unused)]
fn main() {
use glean::{DistributionData, ErrorType};
use glean_metrics;

// Was anything recorded?
assert!(memory::heap_allocated.test_get_value(None).is_some());

// Is the sum as expected?
let data = memory::heap_allocated.test_get_value(None).unwrap();
assert_eq!(11, data.sum)
// The actual buckets and counts live in data.values.

// Were there any errors?
assert_eq!(1, memory::heap_allocated.test_get_num_recorded_errors(InvalidValue));

}


Note: C++ APIs are only available in Firefox Desktop.

#include "mozilla/glean/GleanMetrics.h"

mozilla::glean::memory::heap_allocated.Accumulate(bytes / 1024);


There are test APIs available too:

#include "mozilla/glean/GleanMetrics.h"

// Does it have the expected value?
ASSERT_EQ(11 * 1024, mozilla::glean::memory::heap_allocated.TestGetValue().unwrap().value().sum);


Note: JS APIs are only available in Firefox Desktop.

Glean.memory.heapAllocated.accumulate(bytes / 1024);


There are test APIs available too:

const data = Glean.memory.heapAllocated.testGetValue();
Assert.equal(11 * 1024, data.sum);
// Does it have the right number of samples?
Assert.equal(1, Object.entries(data.values).reduce(([bucket, count], sum) => count + sum, 0));


## Limits

• The maximum memory size that can be recorded is 1 Terabyte (240 bytes). Larger sizes will be truncated to 1 Terabyte.

## Examples

• What is the distribution of the size of heap allocations?

## Recorded errors

• invalid_value: If recording a negative memory size.
• invalid_value: If recording a size larger than 1TB.

## Simulator

Note The data provided, is assumed to be in the configured memory unit. The data recorded, on the other hand, is always in bytes. This means that, if the configured memory unit is not byte, the data will be transformed before being recorded. Notice this, by using the select field above to change the memory unit and see the mean of the data recorded changing.