glean_parser package¶
Submodules¶
glean_parser.coverage module¶
Produce coverage reports from the raw information produced by the GLEAN_TEST_COVERAGE feature.
- glean_parser.coverage.coverage(coverage_reports: List[pathlib.Path], metrics_files: Sequence[pathlib.Path], output_format: str, output_file: pathlib.Path, parser_config: Optional[Dict[str, Any]] = None, file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>) int [source]¶
Commandline helper for coverage.
- Parameters
coverage_reports – List of coverage report files, output from the Glean SDK when the GLEAN_TEST_COVERAGE environment variable is set.
metrics_files – List of Path objects to load metrics from.
output_format – The coverage output format to produce. Must be one of OUTPUTTERS.keys().
output_file – Path to output coverage report to.
parser_config – Parser configuration object, passed to parser.parse_objects.
- Returns
Non-zero if there were any errors.
glean_parser.data_review module¶
Produce skeleton Data Review Requests.
- glean_parser.data_review.generate(bug: str, metrics_files: Sequence[pathlib.Path]) int [source]¶
Commandline helper for Data Review Request template generation.
- Parameters
bug – pattern to match in metrics’ bug_numbers lists.
metrics_files – List of Path objects to load metrics from.
- Returns
Non-zero if there were any errors.
glean_parser.javascript module¶
Outputter to generate Javascript code for metrics.
- glean_parser.javascript.args(obj_type: str) Dict[str, object] [source]¶
Returns the list of arguments for each object type.
- glean_parser.javascript.class_name_factory(platform: str) Callable[[str], str] [source]¶
Returns a function that receives an obj_type and returns the correct class name for that type in the current platform.
- glean_parser.javascript.extra_type_name(extra_type: str) str [source]¶
Returns the equivalent TypeScript type to an extra type.
- glean_parser.javascript.generate_build_date(date: Optional[str]) str [source]¶
Generate the build Date object.
- glean_parser.javascript.import_path(obj_type: str) str [source]¶
Returns the import path of the given object inside the @mozilla/glean package.
- glean_parser.javascript.javascript_datatypes_filter(value: Optional[Union[list, dict, str, int, float]]) str [source]¶
A Jinja2 filter that renders Javascript literals.
- glean_parser.javascript.output(lang: str, objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Javascript or Typescript code to output_dir.
- Parameters
lang – Either “javascript” or “typescript”;
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options –
options dictionary, with the following optional keys: - platform: Which platform are we building for. Options are webext and qt.
Default is webext.
- version: The version of the Glean.js Qt library being used.
This option is mandatory when targeting Qt. Note that the version string must only contain the major and minor version i.e. 0.14.
- with_buildinfo: If “true” a gleanBuildInfo.(js|ts) file is generated.
Otherwise generation of that file is skipped. Defaults to “false”.
- build_date: If set to 0 a static unix epoch time will be used.
If set to a ISO8601 datetime string (e.g. 2022-01-03T17:30:00) it will use that date. Other values will throw an error. If not set it will use the current date & time.
- glean_parser.javascript.output_javascript(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Javascript code to output_dir.
- Parameters
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options –
options dictionary, with the following optional keys:
- namespace: The identifier of the global variable to assign to.
This will only have and effect for Qt and static web sites. Default is Glean.
- platform: Which platform are we building for. Options are webext and qt.
Default is webext.
- glean_parser.javascript.output_typescript(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Typescript code to output_dir.
# Note
The only difference between the typescript and javascript templates, currently is the file extension.
- Parameters
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options –
options dictionary, with the following optional keys:
- namespace: The identifier of the global variable to assign to.
This will only have and effect for Qt and static web sites. Default is Glean.
- platform: Which platform are we building for. Options are webext and qt.
Default is webext.
glean_parser.kotlin module¶
Outputter to generate Kotlin code for metrics.
- glean_parser.kotlin.class_name(obj_type: str) str [source]¶
Returns the Kotlin class name for a given metric or ping type.
- glean_parser.kotlin.extra_type_name(typ: str) str [source]¶
Returns the corresponding Kotlin type for event’s extra key types.
- glean_parser.kotlin.generate_build_date(date: Optional[str]) str [source]¶
Generate the build timestamp.
- glean_parser.kotlin.kotlin_datatypes_filter(value: Optional[Union[list, dict, str, int, float]]) str [source]¶
A Jinja2 filter that renders Kotlin literals.
- Based on Python’s JSONEncoder, but overrides:
lists to use listOf
dicts to use mapOf
sets to use setOf
enums to use the like-named Kotlin enum
- glean_parser.kotlin.output_gecko_lookup(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, generate a Kotlin map between Gecko histograms and Glean SDK metric types.
- Parameters
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options –
options dictionary, with the following optional keys:
namespace: The package namespace to declare at the top of the generated files. Defaults to GleanMetrics.
glean_namespace: The package namespace of the glean library itself. This is where glean objects will be imported from in the generated code.
- glean_parser.kotlin.output_kotlin(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Kotlin code to output_dir.
- Parameters
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options –
options dictionary, with the following optional keys:
namespace: The package namespace to declare at the top of the generated files. Defaults to GleanMetrics.
glean_namespace: The package namespace of the glean library itself. This is where glean objects will be imported from in the generated code.
with_buildinfo: If “true” a GleanBuildInfo.kt file is generated. Otherwise generation of that file is skipped. Defaults to “true”.
- build_date: If set to 0 a static unix epoch time will be used.
If set to a ISO8601 datetime string (e.g. 2022-01-03T17:30:00) it will use that date. Other values will throw an error. If not set it will use the current date & time.
- glean_parser.kotlin.type_name(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) str [source]¶
Returns the Kotlin type to use for a given metric or ping object.
glean_parser.lint module¶
- class glean_parser.lint.CheckType(value)[source]¶
Bases:
enum.Enum
An enumeration.
- error = 1¶
- warning = 0¶
- class glean_parser.lint.GlinterNit(check_name: str, name: str, msg: str, check_type: glean_parser.lint.CheckType)[source]¶
Bases:
object
- glean_parser.lint.check_bug_number(metric: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping], parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.check_category_generic(category_name: str, metrics: Iterable[glean_parser.metrics.Metric]) Generator[str, None, None] [source]¶
The category name is too generic.
- glean_parser.lint.check_common_prefix(category_name: str, metrics: Iterable[glean_parser.metrics.Metric]) Generator[str, None, None] [source]¶
Check if all metrics begin with a common prefix.
- glean_parser.lint.check_expired_date(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.check_expired_metric(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.check_misspelled_pings(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.check_redundant_ping(pings: glean_parser.pings.Ping, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
Check if the pings contains ‘ping’ as the prefix or suffix, or ‘ping’ or ‘custom’
- glean_parser.lint.check_tags_required(metric_or_ping: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping], parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.check_unit_in_name(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
The metric name ends in a unit.
- glean_parser.lint.check_user_lifetime_expiration(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.check_valid_in_baseline(metric: glean_parser.metrics.Metric, parser_config: Dict[str, Any]) Generator[str, None, None] [source]¶
- glean_parser.lint.glinter(input_filepaths: Iterable[pathlib.Path], parser_config: Optional[Dict[str, Any]] = None, file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>) int [source]¶
Commandline helper for glinter.
- Parameters
input_filepaths – List of Path objects to load metrics from.
parser_config – Parser configuration object, passed to parser.parse_objects.
file – The stream to write the errors to.
- Returns
Non-zero if there were any glinter errors.
- glean_parser.lint.lint_metrics(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], parser_config: Optional[Dict[str, Any]] = None, file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>) List[glean_parser.lint.GlinterNit] [source]¶
Performs glinter checks on a set of metrics objects.
- Parameters
objs – Tree of metric objects, as returns by parser.parse_objects.
file – The stream to write errors to.
- Returns
List of nits.
- glean_parser.lint.lint_yaml_files(input_filepaths: Iterable[pathlib.Path], file=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>, parser_config: Optional[Dict[str, Any]] = None) List [source]¶
Performs glinter YAML lint on a set of files.
- Parameters
input_filepaths – List of input files to lint.
file – The stream to write errors to.
- Returns
List of nits.
glean_parser.markdown module¶
Outputter to generate Markdown documentation for metrics.
- glean_parser.markdown.data_sensitivity_numbers(data_sensitivity: Optional[List[glean_parser.metrics.DataSensitivity]]) str [source]¶
- glean_parser.markdown.extra_info(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) List[Tuple[str, str]] [source]¶
Returns a list of string to string tuples with extra information for the type (e.g. extra keys for events) or an empty list if nothing is available.
- glean_parser.markdown.if_empty(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) bool [source]¶
- glean_parser.markdown.metrics_docs(obj_name: str) str [source]¶
Return a link to the documentation entry for the Glean SDK metric of the requested type.
- glean_parser.markdown.output_markdown(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Markdown docs to output_dir.
This produces a single metrics.md. The file contains a table of contents and a section for each ping metrics are collected for.
- Parameters
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options – options dictionary, with the following optional key: - project_title: The projects title.
- glean_parser.markdown.ping_bugs(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) Optional[List[str]] [source]¶
- glean_parser.markdown.ping_data_reviews(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) Optional[List[str]] [source]¶
- glean_parser.markdown.ping_desc(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) str [source]¶
Return a text description of the ping. If a custom_pings_cache is available, look in there for non-reserved ping names description.
- glean_parser.markdown.ping_docs(ping_name: str) str [source]¶
Return a link to the documentation entry for the requested Glean SDK built-in ping.
- glean_parser.markdown.ping_include_client_id(ping_name: str, custom_pings_cache: Optional[Dict[str, glean_parser.pings.Ping]] = None) bool [source]¶
- glean_parser.markdown.ping_reasons(ping_name: str, custom_pings_cache: Dict[str, glean_parser.pings.Ping]) Dict[str, str] [source]¶
Returns the reasons dictionary for the ping.
glean_parser.metrics module¶
Classes for each of the high-level metric types.
- class glean_parser.metrics.Boolean(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'boolean'¶
- class glean_parser.metrics.Counter(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'counter'¶
- class glean_parser.metrics.CustomDistribution(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'custom_distribution'¶
- class glean_parser.metrics.DataSensitivity(value)[source]¶
Bases:
enum.Enum
An enumeration.
- highly_sensitive = 4¶
- interaction = 2¶
- technical = 1¶
- web_activity = 3¶
- class glean_parser.metrics.Datetime(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.TimeBase
- typename: str = 'datetime'¶
- class glean_parser.metrics.Event(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- property allowed_extra_keys¶
- property allowed_extra_keys_with_types¶
- default_store_names: List[str] = ['events']¶
- property has_extra_types¶
If any extra key has a type specified, we generate the new struct/object-based API.
- typename: str = 'event'¶
- class glean_parser.metrics.HistogramType(value)[source]¶
Bases:
enum.Enum
An enumeration.
- exponential = 1¶
- linear = 0¶
- class glean_parser.metrics.Jwe(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'jwe'¶
- class glean_parser.metrics.Labeled(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- labeled = True¶
- class glean_parser.metrics.LabeledBoolean(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Labeled
,glean_parser.metrics.Boolean
- typename: str = 'labeled_boolean'¶
- class glean_parser.metrics.LabeledCounter(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Labeled
,glean_parser.metrics.Counter
- typename: str = 'labeled_counter'¶
- class glean_parser.metrics.LabeledString(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Labeled
,glean_parser.metrics.String
- typename: str = 'labeled_string'¶
- class glean_parser.metrics.Lifetime(value)[source]¶
Bases:
enum.Enum
An enumeration.
- application = 1¶
- ping = 0¶
- user = 2¶
- class glean_parser.metrics.MemoryDistribution(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'memory_distribution'¶
- class glean_parser.metrics.MemoryUnit(value)[source]¶
Bases:
enum.Enum
An enumeration.
- byte = 0¶
- gigabyte = 3¶
- kilobyte = 1¶
- megabyte = 2¶
- class glean_parser.metrics.Metric(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
object
- default_store_names: List[str] = ['metrics']¶
- glean_internal_metric_cat: str = 'glean.internal.metrics'¶
- identifier() str [source]¶
Create an identifier unique for this metric. Generally, category.name; however, Glean internal metrics only use name.
- classmethod make_metric(category: str, name: str, metric_info: Dict[str, Optional[Union[list, dict, str, int, float]]], config: Optional[Dict[str, Any]] = None, validated: bool = False)[source]¶
Given a metric_info dictionary from metrics.yaml, return a metric instance.
- Param
category The category the metric lives in
- Param
name The name of the metric
- Param
metric_info A dictionary of the remaining metric parameters
- Param
config A dictionary containing commandline configuration parameters
- Param
validated True if the metric has already gone through jsonschema validation
- Returns
A new Metric instance.
- metric_types: Dict[str, Any] = {'ERROR': <class 'glean_parser.metrics.Labeled'>, 'boolean': <class 'glean_parser.metrics.Boolean'>, 'counter': <class 'glean_parser.metrics.Counter'>, 'custom_distribution': <class 'glean_parser.metrics.CustomDistribution'>, 'datetime': <class 'glean_parser.metrics.Datetime'>, 'event': <class 'glean_parser.metrics.Event'>, 'jwe': <class 'glean_parser.metrics.Jwe'>, 'labeled_boolean': <class 'glean_parser.metrics.LabeledBoolean'>, 'labeled_counter': <class 'glean_parser.metrics.LabeledCounter'>, 'labeled_string': <class 'glean_parser.metrics.LabeledString'>, 'memory_distribution': <class 'glean_parser.metrics.MemoryDistribution'>, 'quantity': <class 'glean_parser.metrics.Quantity'>, 'rate': <class 'glean_parser.metrics.Rate'>, 'string': <class 'glean_parser.metrics.String'>, 'string_list': <class 'glean_parser.metrics.StringList'>, 'text': <class 'glean_parser.metrics.Text'>, 'timespan': <class 'glean_parser.metrics.Timespan'>, 'timing_distribution': <class 'glean_parser.metrics.TimingDistribution'>, 'url': <class 'glean_parser.metrics.Url'>, 'uuid': <class 'glean_parser.metrics.Uuid'>}¶
- serialize() Dict[str, Optional[Union[list, dict, str, int, float]]] [source]¶
Serialize the metric back to JSON object model.
- typename: str = 'ERROR'¶
- class glean_parser.metrics.Quantity(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'quantity'¶
- class glean_parser.metrics.Rate(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'rate'¶
- class glean_parser.metrics.String(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'string'¶
- class glean_parser.metrics.StringList(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'string_list'¶
- class glean_parser.metrics.Text(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'text'¶
- class glean_parser.metrics.TimeBase(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.Metric
- class glean_parser.metrics.TimeUnit(value)[source]¶
Bases:
enum.Enum
An enumeration.
- day = 6¶
- hour = 5¶
- microsecond = 1¶
- millisecond = 2¶
- minute = 4¶
- nanosecond = 0¶
- second = 3¶
- class glean_parser.metrics.Timespan(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.TimeBase
- typename: str = 'timespan'¶
- class glean_parser.metrics.TimingDistribution(*args, **kwargs)[source]¶
Bases:
glean_parser.metrics.TimeBase
- typename: str = 'timing_distribution'¶
- class glean_parser.metrics.Url(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'url'¶
- class glean_parser.metrics.Uuid(type: str, category: str, name: str, bugs: List[str], description: str, notification_emails: List[str], expires: Any, metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, version: int = 0, disabled: bool = False, lifetime: str = 'ping', send_in_pings: Optional[List[str]] = None, unit: Optional[str] = None, gecko_datapoint: str = '', no_lint: Optional[List[str]] = None, data_sensitivity: Optional[List[str]] = None, defined_in: Optional[Dict] = None, telemetry_mirror: Optional[str] = None, _config: Optional[Dict[str, Any]] = None, _validated: bool = False)[source]¶
Bases:
glean_parser.metrics.Metric
- typename: str = 'uuid'¶
glean_parser.parser module¶
Code for parsing metrics.yaml files.
- glean_parser.parser.get_parameter_doc(key: str) str [source]¶
Returns documentation about a specific metric parameter.
- glean_parser.parser.get_ping_parameter_doc(key: str) str [source]¶
Returns documentation about a specific ping parameter.
- glean_parser.parser.parse_objects(filepaths: Iterable[pathlib.Path], config: Optional[Dict[str, Any]] = None) Generator[str, None, Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]]] [source]¶
Parse one or more metrics.yaml and/or pings.yaml files, returning a tree of metrics.Metric, pings.Ping, and tags.Tag instances.
The result is a generator over any errors. If there are no errors, the actual metrics can be obtained from result.value. For example:
result = metrics.parse_metrics(filepaths) for err in result: print(err) all_metrics = result.value
The result value is a dictionary of category names to categories, where each category is a dictionary from metric name to metrics.Metric instances. There are also the special categories pings and tags containing all of the pings.Ping and tags.Tag instances, respectively.
- Parameters
filepaths – list of Path objects to metrics.yaml, pings.yaml, and/or tags.yaml files
config –
A dictionary of options that change parsing behavior. Supported keys are:
allow_reserved: Allow values reserved for internal Glean use.
do_not_disable_expired: Don’t mark expired metrics as disabled. This is useful when you want to retain the original “disabled” value from the metrics.yaml, rather than having it overridden when the metric expires.
allow_missing_files: Do not raise a FileNotFoundError if any of the input filepaths do not exist.
glean_parser.pings module¶
Classes for managing the description of pings.
- class glean_parser.pings.Ping(name: str, description: str, bugs: List[str], notification_emails: List[str], metadata: Optional[Dict] = None, data_reviews: Optional[List[str]] = None, include_client_id: bool = False, send_if_empty: bool = False, reasons: Optional[Dict[str, str]] = None, defined_in: Optional[Dict] = None, no_lint: Optional[List[str]] = None, _validated: bool = False)[source]¶
Bases:
object
- property reason_codes: List[str]¶
- serialize() Dict[str, Optional[Union[list, dict, str, int, float]]] [source]¶
Serialize the metric back to JSON object model.
- property type: str¶
glean_parser.rust module¶
Outputter to generate Rust code for metrics.
- class glean_parser.rust.Category(name: str, objs: Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]], contains_pings: bool)[source]¶
Bases:
object
Data struct holding information about a metric to be used in the template.
- glean_parser.rust.class_name(obj_type)[source]¶
Returns the Rust class name for a given metric or ping type.
- glean_parser.rust.ctor(obj)[source]¶
Returns the scope and name of the constructor to use for a metric object. Necessary because LabeledMetric<T> is constructed using LabeledMetric::new not LabeledMetric<T>::new
- glean_parser.rust.extra_keys(allowed_extra_keys)[source]¶
Returns the &’static [&’static str] ALLOWED_EXTRA_KEYS for impl ExtraKeys
- glean_parser.rust.extra_type_name(typ: str) str [source]¶
Returns the corresponding Rust type for event’s extra key types.
- glean_parser.rust.output_rust(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Rust code to output_dir.
- Parameters
objs – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options – options dictionary, not currently used for Rust
- glean_parser.rust.rust_datatypes_filter(value)[source]¶
A Jinja2 filter that renders Rust literals.
- Based on Python’s JSONEncoder, but overrides:
dicts and sets to raise an error
sets to vec![] (used in labels)
enums to become Class::Value
lists to vec![] (used in send_in_pings)
null to None
strings to “value”.into()
Rate objects to a CommonMetricData initializer (for external Denominators’ Numerators lists)
glean_parser.swift module¶
Outputter to generate Swift code for metrics.
- class glean_parser.swift.Category[source]¶
Bases:
object
Data struct holding information about a metric to be used in the template.
- contains_pings: bool¶
- name: str¶
- objs: Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]¶
- glean_parser.swift.class_name(obj_type: str) str [source]¶
Returns the Swift class name for a given metric or ping type.
- glean_parser.swift.extra_type_name(typ: str) str [source]¶
Returns the corresponding Kotlin type for event’s extra key types.
- glean_parser.swift.generate_build_date(date: Optional[str]) str [source]¶
Generate the build timestamp.
- glean_parser.swift.output_swift(objs: Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None) None [source]¶
Given a tree of objects, output Swift code to output_dir.
- Parameters
objects – A tree of objects (metrics and pings) as returned from parser.parse_objects.
output_dir – Path to an output directory to write to.
options –
options dictionary, with the following optional keys: - namespace: The namespace to generate metrics in - glean_namespace: The namespace to import Glean from - allow_reserved: When True, this is a Glean-internal build - with_buildinfo: If “true” the GleanBuildInfo is generated.
Otherwise generation of that file is skipped. Defaults to “true”.
- build_date: If set to 0 a static unix epoch time will be used.
If set to a ISO8601 datetime string (e.g. 2022-01-03T17:30:00) it will use that date. Other values will throw an error. If not set it will use the current date & time.
- glean_parser.swift.swift_datatypes_filter(value: Optional[Union[list, dict, str, int, float]]) str [source]¶
A Jinja2 filter that renders Swift literals.
- Based on Python’s JSONEncoder, but overrides:
dicts to use [key: value]
sets to use […]
enums to use the like-named Swift enum
- glean_parser.swift.type_name(obj: Union[glean_parser.metrics.Metric, glean_parser.pings.Ping]) str [source]¶
Returns the Swift type to use for a given metric or ping object.
glean_parser.translate module¶
High-level interface for translating metrics.yaml into other formats.
- class glean_parser.translate.Outputter(output_func: Callable[[Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], pathlib.Path, Dict[str, Any]], None], clear_patterns: Optional[List[str]] = None)[source]¶
Bases:
object
Class to define an output format.
Each outputter in the table has the following member values:
output_func: the main function of the outputter, the one which does the actual translation.
clear_patterns: A list of glob patterns to clear in the directory before writing new results to it.
- glean_parser.translate.transform_metrics(objects)[source]¶
Transform the object model from one that represents the YAML definitions to one that reflects the type specifics needed by code generators.
e.g. This will transform a rate to be a numerator if its denominator is external.
- glean_parser.translate.translate(input_filepaths: Iterable[pathlib.Path], output_format: str, output_dir: pathlib.Path, options: Optional[Dict[str, Any]] = None, parser_config: Optional[Dict[str, Any]] = None)[source]¶
Translate the files in input_filepaths to the given output_format and put the results in output_dir.
- Parameters
input_filepaths – list of paths to input metrics.yaml files
output_format – the name of the output format
output_dir – the path to the output directory
options – dictionary of options. The available options are backend format specific.
parser_config – A dictionary of options that change parsing behavior. See parser.parse_metrics for more info.
- glean_parser.translate.translate_metrics(input_filepaths: Iterable[pathlib.Path], output_dir: pathlib.Path, translation_func: Callable[[Dict[str, Dict[str, Union[glean_parser.metrics.Metric, glean_parser.pings.Ping, glean_parser.tags.Tag]]], pathlib.Path, Dict[str, Any]], None], clear_patterns: Optional[List[str]] = None, options: Optional[Dict[str, Any]] = None, parser_config: Optional[Dict[str, Any]] = None)[source]¶
Translate the files in input_filepaths by running the metrics through a translation function and writing the results in output_dir.
- Parameters
input_filepaths – list of paths to input metrics.yaml files
output_dir – the path to the output directory
translation_func –
the function that actually performs the translation. It is passed the following arguments:
metrics_objects: The tree of metrics as pings as returned by parser.parse_objects.
output_dir: The path to the output directory.
options: A dictionary of output format-specific options.
Examples of translation functions are in kotlin.py and swift.py.
clear_patterns – a list of glob patterns of files to clear before generating the output files. By default, no files will be cleared (i.e. the directory should be left alone).
options – dictionary of options. The available options are backend format specific. These are passed unchanged to translation_func.
parser_config – A dictionary of options that change parsing behavior. See parser.parse_metrics for more info.
glean_parser.translation_options module¶
glean_parser.util module¶
- glean_parser.util.Camelize(value: str) str [source]¶
Convert the value to CamelCase (with an upper case first letter).
This is a thin wrapper around inflection.camelize that handles dots in addition to underscores.
- glean_parser.util.JSONType¶
The types supported by JSON.
This is only an approximation – this should really be a recursive type.
alias of
Optional
[Union
[list
,dict
,str
,int
,float
]]
- glean_parser.util.build_date(date: Optional[str]) datetime.datetime [source]¶
Generate the build timestamp.
If date is set to 0 a static unix epoch time will be used. If date it is set to a ISO8601 datetime string (e.g. 2022-01-03T17:30:00) it will use that date. Note that any timezone offset will be ignored and UTC will be used. Otherwise it will throw an error.
If date is None it will use the current date & time.
- glean_parser.util.camelize(value: str) str [source]¶
Convert the value to camelCase (with a lower case first letter).
This is a thin wrapper around inflection.camelize that handles dots in addition to underscores.
- glean_parser.util.ensure_list(value: Any) Sequence[Any] [source]¶
Ensures that the value is a list. If it is anything but a list or tuple, a list with a single element containing only value is returned.
- glean_parser.util.fetch_remote_url(url: str, cache: bool = True)[source]¶
Fetches the contents from an HTTP url or local file path, and optionally caches it to disk.
- glean_parser.util.format_error(filepath: Union[str, pathlib.Path], header: str, content: str, lineno: Optional[int] = None) str [source]¶
Format a jsonshema validation error.
- glean_parser.util.get_jinja2_template(template_name: str, filters: Iterable[Tuple[str, Callable]] = ())[source]¶
Get a Jinja2 template that ships with glean_parser.
The template has extra filters for camel-casing identifiers.
- Parameters
template_name – Name of a file in
glean_parser/templates
filters – tuple of 2-tuple. A tuple of (name, func) pairs defining additional filters.
- glean_parser.util.get_null_resolver(schema)[source]¶
Returns a JSON Pointer resolver that does nothing.
This lets us handle the moz: URLs in our schemas.
- glean_parser.util.is_expired(expires: str, major_version: Optional[int] = None) bool [source]¶
Parses the expires field in a metric or ping and returns whether the object should be considered expired.
- glean_parser.util.keep_value(f)[source]¶
Wrap a generator so the value it returns (rather than yields), will be accessible on the .value attribute when the generator is exhausted.
- glean_parser.util.load_yaml_or_json(path: pathlib.Path)[source]¶
Load the content from either a .json or .yaml file, based on the filename extension.
- Parameters
path – pathlib.Path object
- Rtype object
The tree of objects as a result of parsing the file.
- Raises
ValueError – The file is neither a .json, .yml or .yaml file.
FileNotFoundError – The file does not exist.
- glean_parser.util.parse_expiration_date(expires: str) datetime.date [source]¶
Parses the expired field date (yyyy-mm-dd) as a date. Raises a ValueError in case the string is not properly formatted.
- glean_parser.util.parse_expiration_version(expires: str) int [source]¶
Parses the expired field version string as an integer. Raises a ValueError in case the string does not contain a valid positive integer.
- glean_parser.util.pprint_validation_error(error) str [source]¶
A version of jsonschema’s ValidationError __str__ method that doesn’t include the schema fragment that failed. This makes the error messages much more succinct.
It also shows any subschemas of anyOf/allOf that failed, if any (what jsonschema calls “context”).
- glean_parser.util.remove_output_params(d, output_params)[source]¶
Remove output-only params, such as “defined_in”, in order to validate the output against the input schema.
- glean_parser.util.report_validation_errors(all_objects)[source]¶
Report any validation errors found to the console.
Returns the number of errors reported.
- glean_parser.util.to_camel_case(input: str, capitalize_first_letter: bool) str [source]¶
Convert the value to camelCase.
This additionally replaces any ‘.’ with ‘_’. The first letter is capitalized depending on capitalize_first_letter.
- glean_parser.util.validate_expires(expires: str, major_version: Optional[int] = None) None [source]¶
If expiration by major version is enabled, raises a ValueError in case expires is not a positive integer. Otherwise raises a ValueError in case the expires is not ISO8601 parseable, or in case the date is more than 730 days (~2 years) in the future.
glean_parser.validate_ping module¶
Validates the contents of a Glean ping against the schema.
- glean_parser.validate_ping.validate_ping(ins, outs=None, schema_url=None)[source]¶
Validates the contents of a Glean ping.
- Parameters
ins – Input stream or file path to the ping contents to validate
outs – Output stream to write errors to. (Defaults to stdout)
schema_url – HTTP URL or local filesystem path to Glean ping schema. Defaults to the current version of the schema in mozilla-pipeline-schemas.
- Return type
int 1 if any errors occurred, otherwise 0.
Module contents¶
Top-level package for Glean parser.