glean_parser package

Submodules

glean_parser.csharp module

Outputter to generate C# code for metrics.

glean_parser.csharp.class_name(obj_type: str) → str

Returns the C# class name for a given metric or ping type.

glean_parser.csharp.csharp_datatypes_filter(value: Optional[Union[list, dict, str, int, float]]) → str

A Jinja2 filter that renders C# literals.

Based on Python’s JSONEncoder, but overrides:

• lists to use new string[] {} (only strings)

• dicts to use new Dictionary<string, string> { …} (string, string)

• sets to use new HashSet<string>() {} (only strings)

• enums to use the like-named C# enum

glean_parser.csharp.output_csharp(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) → None

Given a tree of objects, output C# code to output_dir.

Parameters

• objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.

• output_dir – Path to an output directory to write to.

• options –

  options dictionary, with the following optional keys:

  • namespace: The package namespace to declare at the top of the generated files. Defaults to GleanMetrics.

  • glean_namespace: The package namespace of the glean library itself. This is where glean objects will be imported from in the generated code.

glean_parser.csharp.type_name(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) → str

Returns the C# type to use for a given metric or ping object.

glean_parser.kotlin module

Outputter to generate Kotlin code for metrics.

glean_parser.kotlin.class_name(obj_type: str) → str

Returns the Kotlin class name for a given metric or ping type.

glean_parser.kotlin.kotlin_datatypes_filter(value: Optional[Union[list, dict, str, int, float]]) → str

A Jinja2 filter that renders Kotlin literals.

Based on Python’s JSONEncoder, but overrides:

• lists to use listOf

• dicts to use mapOf

• sets to use setOf

• enums to use the like-named Kotlin enum

glean_parser.kotlin.output_gecko_lookup(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) → None

Given a tree of objects, generate a Kotlin map between Gecko histograms and Glean SDK metric types.

Parameters

• objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.

• output_dir – Path to an output directory to write to.

• options –

  options dictionary, with the following optional keys:

  • namespace: The package namespace to declare at the top of the generated files. Defaults to GleanMetrics.

  • glean_namespace: The package namespace of the glean library itself. This is where glean objects will be imported from in the generated code.

glean_parser.kotlin.output_kotlin(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) → None

Given a tree of objects, output Kotlin code to output_dir.

Parameters

• objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.

• output_dir – Path to an output directory to write to.

• options –

  options dictionary, with the following optional keys:

  • namespace: The package namespace to declare at the top of the generated files. Defaults to GleanMetrics.

  • glean_namespace: The package namespace of the glean library itself. This is where glean objects will be imported from in the generated code.

glean_parser.kotlin.type_name(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) → str

Returns the Kotlin type to use for a given metric or ping object.

glean_parser.lint module

class glean_parser.lint.CheckType(value)

Bases: enum.Enum

An enumeration.

error = 1

warning = 0

class glean_parser.lint.GlinterNit(check_name: str, name: str, msg: str, check_type: glean_parser.lint.CheckType)

Bases: object

format()

glean_parser.lint.check_bug_number(metric: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping], parser_config: Dict[str, Any]) → Generator[str, None, None]

glean_parser.lint.check_category_generic(category_name: str, metrics: Iterable[glean_parser.metrics.Metric]) → Generator[str, None, None]

The category name is too generic.

glean_parser.lint.check_common_prefix(category_name: str, metrics: Iterable[glean_parser.metrics.Metric]) → Generator[str, None, None]

Check if all metrics begin with a common prefix.

glean_parser.lint.check_expired_date(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) → Generator[str, None, None]

glean_parser.lint.check_expired_metric(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) → Generator[str, None, None]

glean_parser.lint.check_misspelled_pings(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) → Generator[str, None, None]

glean_parser.lint.check_unit_in_name(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) → Generator[str, None, None]

The metric name ends in a unit.

glean_parser.lint.check_user_lifetime_expiration(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) → Generator[str, None, None]

glean_parser.lint.check_valid_in_baseline(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) → Generator[str, None, None]

glean_parser.lint.glinter(input_filepaths: Iterable[pathlib.Path], parser_config: Optional[Dict[str, Any]] = None, file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>) → int

Commandline helper for glinter.

Parameters

• input_filepaths – List of Path objects to load metrics from.

• parser_config – Parser configuration object, passed to parser.parse_objects.

• file – The stream to write the errors to.

Returns

Non-zero if there were any glinter errors.

glean_parser.lint.lint_metrics(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], parser_config: Optional[Dict[str, Any]] = None, file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>) → List[glean_parser.lint.GlinterNit]

Performs glinter checks on a set of metrics objects.

Parameters

• objs – Tree of metric objects, as returns by parser.parse_objects.

• file – The stream to write errors to.

Returns

List of nits.

glean_parser.lint.lint_yaml_files(input_filepaths: Iterable[pathlib.Path], file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>, parser_config: Optional[Dict[str, Any]] = None) → List

Performs glinter YAML lint on a set of files.

Parameters

• input_filepaths – List of input files to lint.

• file – The stream to write errors to.

Returns

List of nits.

glean_parser.markdown module

Outputter to generate Markdown documentation for metrics.

glean_parser.markdown.data_sensitivity_numbers(data_sensitivity: Optional[List[glean_parser.metrics.DataSensitivity]]) → str

glean_parser.markdown.extra_info(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) → List[Tuple[str, str]]

Returns a list of string to string tuples with extra information for the type (e.g. extra keys for events) or an empty list if nothing is available.

glean_parser.markdown.if_empty(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) → bool

glean_parser.markdown.metrics_docs(obj_name: str) → str

Return a link to the documentation entry for the Glean SDK metric of the requested type.

glean_parser.markdown.output_markdown(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) → None

Given a tree of objects, output Markdown docs to output_dir.

This produces a single metrics.md. The file contains a table of contents and a section for each ping metrics are collected for.

Parameters

• objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.

• output_dir – Path to an output directory to write to.

• options – options dictionary, with the following optional key: - project_title: The projects title.

glean_parser.markdown.ping_bugs(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) → Optional[List[str]]

glean_parser.markdown.ping_data_reviews(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) → Optional[List[str]]

glean_parser.markdown.ping_desc(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) → str

Return a text description of the ping. If a custom_pings_cache is available, look in there for non-reserved ping names description.

glean_parser.markdown.ping_docs(ping_name: str) → str

Return a link to the documentation entry for the requested Glean SDK built-in ping.

glean_parser.markdown.ping_include_client_id(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) → bool

glean_parser.markdown.ping_reasons(ping_name: str, custom_pings_cache: Dict[str, glean_parser.pings.Ping]) → Dict[str, str]

Returns the reasons dictionary for the ping.

glean_parser.markdown.ping_review_title(data_url: str, index: int) → str

Return a title for a data review in human readable form.

Parameters

• data_url – A url for data review.

• index – Position of the data review on list (e.g: 1, 2, 3…).

glean_parser.metrics module

Classes for each of the high-level metric types.

class glean_parser.metrics.Boolean(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: glean_parser.metrics.Metric

typename: str = 'boolean'

class glean_parser.metrics.Counter(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: glean_parser.metrics.Metric

typename: str = 'counter'

class glean_parser.metrics.CustomDistribution(*args, **kwargs)

Bases: glean_parser.metrics.Metric

typename: str = 'custom_distribution'

class glean_parser.metrics.DataSensitivity(value)

Bases: enum.Enum

An enumeration.

highly_sensitive = 4

interaction = 2

technical = 1

web_activity = 3

class glean_parser.metrics.Datetime(*args, **kwargs)

Bases: glean_parser.metrics.TimeBase

typename: str = 'datetime'

class glean_parser.metrics.Event(*args, **kwargs)

Bases: glean_parser.metrics.Metric

property allowed_extra_keys

default_store_names: List[str] = ['events']

typename: str = 'event'

static validate_extra_keys(extra_keys: Dict[str, str], config: Dict[str, Any]) → None

class glean_parser.metrics.HistogramType(value)

Bases: enum.Enum

An enumeration.

exponential = 1

linear = 0

class glean_parser.metrics.Jwe(*args, **kwargs)

Bases: glean_parser.metrics.Metric

typename: str = 'jwe'

class glean_parser.metrics.Labeled(*args, **kwargs)

Bases: glean_parser.metrics.Metric

labeled = True

serialize() → Dict[str, Optional[Union[list, dict, str, int, float]]]

Serialize the metric back to JSON object model.

class glean_parser.metrics.LabeledBoolean(*args, **kwargs)

Bases: glean_parser.metrics.Labeled, glean_parser.metrics.Boolean

typename: str = 'labeled_boolean'

class glean_parser.metrics.LabeledCounter(*args, **kwargs)

Bases: glean_parser.metrics.Labeled, glean_parser.metrics.Counter

typename: str = 'labeled_counter'

class glean_parser.metrics.LabeledString(*args, **kwargs)

Bases: glean_parser.metrics.Labeled, glean_parser.metrics.String

typename: str = 'labeled_string'

class glean_parser.metrics.Lifetime(value)

Bases: enum.Enum

An enumeration.

application = 1

ping = 0

user = 2

class glean_parser.metrics.MemoryDistribution(*args, **kwargs)

Bases: glean_parser.metrics.Metric

typename: str = 'memory_distribution'

class glean_parser.metrics.MemoryUnit(value)

Bases: enum.Enum

An enumeration.

byte = 0

gigabyte = 3

kilobyte = 1

megabyte = 2

class glean_parser.metrics.Metric(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: object

default_store_names: List[str] = ['metrics']

glean_internal_metric_cat: str = 'glean.internal.metrics'

identifier() → str

Create an identifier unique for this metric. Generally, category.name; however, Glean internal metrics only use name.

is_disabled() → bool

is_expired() → bool

is_internal_metric() → bool

classmethod make_metric(category: str, name: str, metric_info: Dict[str, Optional[Union[list, dict, str, int, float]]], config: Optional[Dict[str, Any]] = None, validated: bool = False)

Given a metric_info dictionary from metrics.yaml, return a metric instance.

Param

category The category the metric lives in

Param

name The name of the metric

Param

metric_info A dictionary of the remaining metric parameters

Param

config A dictionary containing commandline configuration parameters

Param

validated True if the metric has already gone through jsonschema validation

Returns

A new Metric instance.

metric_types: Dict[str, Any] = {'ERROR': <class 'glean_parser.metrics.Labeled'>, 'boolean': <class 'glean_parser.metrics.Boolean'>, 'counter': <class 'glean_parser.metrics.Counter'>, 'custom_distribution': <class 'glean_parser.metrics.CustomDistribution'>, 'datetime': <class 'glean_parser.metrics.Datetime'>, 'event': <class 'glean_parser.metrics.Event'>, 'jwe': <class 'glean_parser.metrics.Jwe'>, 'labeled_boolean': <class 'glean_parser.metrics.LabeledBoolean'>, 'labeled_counter': <class 'glean_parser.metrics.LabeledCounter'>, 'labeled_string': <class 'glean_parser.metrics.LabeledString'>, 'memory_distribution': <class 'glean_parser.metrics.MemoryDistribution'>, 'quantity': <class 'glean_parser.metrics.Quantity'>, 'string': <class 'glean_parser.metrics.String'>, 'string_list': <class 'glean_parser.metrics.StringList'>, 'timespan': <class 'glean_parser.metrics.Timespan'>, 'timing_distribution': <class 'glean_parser.metrics.TimingDistribution'>, 'uuid': <class 'glean_parser.metrics.Uuid'>}

serialize() → Dict[str, Optional[Union[list, dict, str, int, float]]]

Serialize the metric back to JSON object model.

typename: str = 'ERROR'

validate_expires()

class glean_parser.metrics.Quantity(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: glean_parser.metrics.Metric

typename: str = 'quantity'

class glean_parser.metrics.String(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: glean_parser.metrics.Metric

typename: str = 'string'

class glean_parser.metrics.StringList(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: glean_parser.metrics.Metric

typename: str = 'string_list'

class glean_parser.metrics.TimeBase(*args, **kwargs)

Bases: glean_parser.metrics.Metric

class glean_parser.metrics.TimeUnit(value)

Bases: enum.Enum

An enumeration.

day = 6

hour = 5

microsecond = 1

millisecond = 2

minute = 4

nanosecond = 0

second = 3

class glean_parser.metrics.Timespan(*args, **kwargs)

Bases: glean_parser.metrics.TimeBase

typename: str = 'timespan'

class glean_parser.metrics.TimingDistribution(*args, **kwargs)

Bases: glean_parser.metrics.TimeBase

typename: str = 'timing_distribution'

class glean_parser.metrics.Uuid(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)

Bases: glean_parser.metrics.Metric

typename: str = 'uuid'

glean_parser.parser module

Code for parsing metrics.yaml files.

glean_parser.parser.get_parameter_doc(key: str) → str

Returns documentation about a specific metric parameter.

glean_parser.parser.get_ping_parameter_doc(key: str) → str

Returns documentation about a specific ping parameter.

glean_parser.parser.parse_objects(filepaths: Iterable[pathlib.Path], config: Optional[Dict[str, Any]] = None) → Generator[str, None, Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]]]

Parse one or more metrics.yaml and/or pings.yaml files, returning a tree of metrics.Metric and pings.Ping instances.

The result is a generator over any errors. If there are no errors, the actual metrics can be obtained from result.value. For example:

result = metrics.parse_metrics(filepaths)
for err in result:
    print(err)
all_metrics = result.value

The result value is a dictionary of category names to categories, where each category is a dictionary from metric name to metrics.Metric instances. There is also the special category pings containing all of the pings.Ping instances.

Parameters

• filepaths – list of Path objects to metrics.yaml and/or pings.yaml files

• config –

  A dictionary of options that change parsing behavior. Supported keys are:

  • allow_reserved: Allow values reserved for internal Glean use.

  • do_not_disable_expired: Don’t mark expired metrics as disabled. This is useful when you want to retain the original “disabled” value from the metrics.yaml, rather than having it overridden when the metric expires.

  • allow_missing_files: Do not raise a FileNotFoundError if any of the input filepaths do not exist.

glean_parser.parser.validate(content: Dict[str, Optional[Union[list, dict, str, int, float]]], filepath: Union[str, pathlib.Path] = '<input>') → Generator[str, None, None]

Validate the given content against the appropriate schema.

glean_parser.pings module

Classes for managing the description of pings.

class glean_parser.pings.Ping(name: str, description: str, bugs: List[str], notification_emails: List[str], data_reviews: Optional[List[str]] = None, include_client_id: bool = False, send_if_empty: bool = False, reasons: Optional[Dict[str, str]] = None, defined_in: Optional[Dict] = None, no_lint: Optional[List[str]] = None, _validated: bool = False)

Bases: object

identifier() → str

Used for the “generated from …” comment in the output.

property reason_codes

serialize() → Dict[str, Optional[Union[list, dict, str, int, float]]]

Serialize the metric back to JSON object model.

property type

glean_parser.swift module

Outputter to generate Swift code for metrics.

class glean_parser.swift.Category

Bases: object

Data struct holding information about a metric to be used in the template.

contains_pings: bool

name: str

objs: Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]

glean_parser.swift.class_name(obj_type: str) → str

Returns the Swift class name for a given metric or ping type.

glean_parser.swift.output_swift(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) → None

Given a tree of objects, output Swift code to output_dir.

Parameters

• objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.

• output_dir – Path to an output directory to write to.

• options – options dictionary, with the following optional keys: - namespace: The namespace to generate metrics in - glean_namespace: The namespace to import Glean from - allow_reserved: When True, this is a Glean-internal build

glean_parser.swift.swift_datatypes_filter(value: Optional[Union[list, dict, str, int, float]]) → str

A Jinja2 filter that renders Swift literals.

Based on Python’s JSONEncoder, but overrides:

• dicts to use [key: value]

• sets to use […]

• enums to use the like-named Swift enum

glean_parser.swift.type_name(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) → str

Returns the Swift type to use for a given metric or ping object.

glean_parser.swift.variable_name(var: str) → str

Returns a valid Swift variable name, escaping keywords if necessary.

glean_parser.translate module

High-level interface for translating metrics.yaml into other formats.

class glean_parser.translate.Outputter(output_func: Callable[[Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], pathlib.Path, Dict[str, Any]], None], clear_patterns: Optional[List[str]] = None)

Bases: object

Class to define an output format.

Each outputter in the table has the following member values:

• output_func: the main function of the outputter, the one which does the actual translation.

• clear_patterns: A list of glob patterns to clear in the directory before writing new results to it.

glean_parser.translate.translate(input_filepaths: Iterable[pathlib.Path], output_format: str, output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None, parser_config: Optional[Dict[str, Any]] = None)

Translate the files in input_filepaths to the given output_format and put the results in output_dir.

Parameters

• input_filepaths – list of paths to input metrics.yaml files

• output_format – the name of the output format

• output_dir – the path to the output directory

• options – dictionary of options. The available options are backend format specific.

• parser_config – A dictionary of options that change parsing behavior. See parser.parse_metrics for more info.

glean_parser.translate.translate_metrics(input_filepaths: Iterable[pathlib.Path], output_dir: pathlib.Path, translation_func: Callable[[Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]]], pathlib.Path, Dict[str, Any]], None], clear_patterns: Optional[List[str]] = None, options: Optional[Dict[str, Any]] = None, parser_config: Optional[Dict[str, Any]] = None)

Translate the files in input_filepaths by running the metrics through a translation function and writing the results in output_dir.

Parameters

• input_filepaths – list of paths to input metrics.yaml files

• output_dir – the path to the output directory

• translation_func –

  the function that actually performs the translation. It is passed the following arguments:

  • metrics_objects: The tree of metrics as pings as returned by parser.parse_objects.

  • output_dir: The path to the output directory.

  • options: A dictionary of output format-specific options.

  Examples of translation functions are in kotlin.py and swift.py.

• clear_patterns – a list of glob patterns of files to clear before generating the output files. By default, no files will be cleared (i.e. the directory should be left alone).

• options – dictionary of options. The available options are backend format specific. These are passed unchanged to translation_func.

• parser_config – A dictionary of options that change parsing behavior. See parser.parse_metrics for more info.

glean_parser.util module

glean_parser.util.Camelize(value: str) → str

Convert the value to CamelCase (with an upper case first letter).

This is a thin wrapper around inflection.camelize that handles dots in addition to underscores.

glean_parser.util.JSONType

The types supported by JSON.

This is only an approximation – this should really be a recursive type.

alias of Optional[Union[list, dict, str, int, float]]

glean_parser.util.camelize(value: str) → str

Convert the value to camelCase (with a lower case first letter).

This is a thin wrapper around inflection.camelize that handles dots in addition to underscores.

glean_parser.util.ensure_list(value: Any) → Sequence[Any]

Ensures that the value is a list. If it is anything but a list or tuple, a list with a single element containing only value is returned.

glean_parser.util.fetch_remote_url(url: str, cache: bool = True)

Fetches the contents from an HTTP url or local file path, and optionally caches it to disk.

glean_parser.util.format_error(filepath: Union[str, pathlib.Path], header: str, content: str) → str

Format a jsonshema validation error.

glean_parser.util.get_jinja2_template(template_name: str, filters: Iterable[Tuple[str, Callable]] = ())

Get a Jinja2 template that ships with glean_parser.

The template has extra filters for camel-casing identifiers.

Parameters

• template_name – Name of a file in glean_parser/templates

• filters – tuple of 2-tuple. A tuple of (name, func) pairs defining additional filters.

glean_parser.util.get_null_resolver(schema)

Returns a JSON Pointer resolver that does nothing.

This lets us handle the moz: URLs in our schemas.

glean_parser.util.is_expired(expires: str) → bool

Parses the expires field in a metric or ping and returns whether the object should be considered expired.

glean_parser.util.keep_value(f)

Wrap a generator so the value it returns (rather than yields), will be accessible on the .value attribute when the generator is exhausted.

glean_parser.util.load_yaml_or_json(path: pathlib.Path)

Load the content from either a .json or .yaml file, based on the filename extension.

Parameters

path – pathlib.Path object

Rtype object

The tree of objects as a result of parsing the file.

Raises

• ValueError – The file is neither a .json, .yml or .yaml file.

• FileNotFoundError – The file does not exist.

glean_parser.util.ordered_yaml_dump(data, **kwargs)

glean_parser.util.parse_expires(expires: str) → datetime.date

Parses the expired field date (yyyy-mm-dd) as a date. Raises a ValueError in case the string is not properly formatted.

glean_parser.util.pprint_validation_error(error) → str

A version of jsonschema’s ValidationError __str__ method that doesn’t include the schema fragment that failed. This makes the error messages much more succinct.

It also shows any subschemas of anyOf/allOf that failed, if any (what jsonschema calls “context”).

glean_parser.util.remove_output_params(d, output_params)

Remove output-only params, such as “defined_in”, in order to validate the output against the input schema.

glean_parser.util.report_validation_errors(all_objects)

Report any validation errors found to the console.

glean_parser.util.to_camel_case(input: str, capitalize_first_letter: bool) → str

Convert the value to camelCase.

This additionally replaces any ‘.’ with ‘_’. The first letter is capitalized depending on capitalize_first_letter.

glean_parser.util.validate_expires(expires: str) → None

Raises a ValueError in case the expires is not ISO8601 parseable, or in case the date is more than 730 days (~2 years) in the future.

glean_parser.util.yaml_load(stream)

Map line number to yaml nodes, and preserve the order of metrics as they appear in the metrics.yaml file.

glean_parser.validate_ping module

Validates the contents of a Glean ping against the schema.

glean_parser.validate_ping.validate_ping(ins, outs=None, schema_url=None)

Validates the contents of a Glean ping.

Parameters

• ins – Input stream or file path to the ping contents to validate

• outs – Output stream to write errors to. (Defaults to stdout)

• schema_url – HTTP URL or local filesystem path to Glean ping schema. Defaults to the current version of the schema in mozilla-pipeline-schemas.

Return type

int 1 if any errors occurred, otherwise 0.

Module contents

Top-level package for Glean parser.