exemptions module

exemptions.load(rules)

Marks tests as xfail based on test name and ID.

Parses the exemptions section of the conf file and returns a two level dict with format:

{<test_name>:

{<test_id>: (<exemption expiration>, <exemption reason>) … }

… }

Examples:

>>> load([
... {
... 'test_name': 'test_foo',
... 'test_param_id': 'foo-id',
... 'expiration_day': date(2050, 1, 1),
... 'reason': 'in prod never allow foo'
... }
... ]) == {'test_foo': {'foo-id': ('2050-01-01', 'in prod never allow foo')}}
True

>>> load([
... {
... 'test_name': '*test_foo',
... 'test_param_id': 'foo-id',
... 'expiration_day': date(2050, 1, 1),
... 'reason': 'in prod never allow foo'
... }
... ]) == {'*test_foo': {'foo-id': ('2050-01-01', 'in prod never allow foo')}}
True

>>> load([
... {
... 'test_name': 'test_foo',
... 'test_param_id': 'foo-id',
... 'expiration_day': date(2050, 1, 1),
... 'reason': 'in prod never allow foo'
... },
... {
... 'test_name': 'test_foo',
... 'test_param_id': 'bar-id',
... 'expiration_day': date(2050, 1, 1),
... 'reason': 'in prod never allow bar'
... }
... ]) == {
... 'test_foo': {
... 'foo-id': ('2050-01-01', 'in prod never allow foo'),
... 'bar-id': ('2050-01-01', 'in prod never allow bar')}}
True

Duplicate test name and IDs are ignored with a warning:

>>> load([
... {
... 'test_name': 'test_foo',
... 'test_param_id': 'foo-id',
... 'expiration_day': date(2050, 1, 1),
... 'reason': 'in prod never allow foo'
... },
... {
... 'test_name': 'test_foo',
... 'test_param_id': 'foo-id',
... 'expiration_day': date(2051, 1, 1),
... 'reason': 'in prod never allow foo another'
... }
... ]) == {'test_foo': {'foo-id': ('2050-01-01', 'in prod never allow foo')}}
True
>>> # UserWarning: Exemptions: test_name: test_foo | test_id: foo-id | Skipping duplicate test name and ID

Does not check that test name and IDs exist (since names might not be collected and IDs can require an HTTP call).

exemptions.add_xfail_marker(item)

Adds xfail markers for test names and ids specified in the exemptions conf.