fathom test

Evaluate how well a trained ruleset does.

TESTING_SET_FOLDER is a directory of labeled testing pages. It can also be, for backward compatibility, a JSON file of vectors from FathomFox’s Vectorizer.

WEIGHTS should be a JSON-formatted object, as follows. You can paste it directly from the output of trainer.

{“coeffs”: [[“nextAnchorIsJavaScript”, 1.1627885103225708],

[“nextButtonTypeSubmit”, 4.613410949707031],

[“nextInputTypeSubmit”, 4.374269008636475]],

“bias”: -8.645608901977539}

fathom test [OPTIONS] TESTING_SET_FOLDER WEIGHTS

Options

-t, --confidence-threshold <confidence_threshold>

Threshold at which a sample is considered positive. Higher values decrease false positives and increase false negatives. [default: 0.5]

-r, --ruleset <ruleset>

The rulesets.js file containing your rules. The file must have no imports except from fathom-web, so pre-bundle if necessary.

--trainee <trainee>

The trainee ID of the ruleset you are testing. Usually, this is the same as the type you are testing.

--testing-cache <testing_cache>

Where to cache testing vectors to speed future testing runs. Any existing file will be overwritten. [default: vectors/testing_yourTraineeId.json next to your ruleset]

--delay <delay>

Number of seconds to wait for a page to load before vectorizing it [default: 5]

--tabs <tabs>

Number of concurrent browser tabs to use while vectorizing [default: 16]

--show-browser

Show browser window while vectorizing. (Browser runs in headless mode by default.)

-v, --verbose

Show per-tag diagnostics, even though that could ruin blinding for the test set.

Arguments

TESTING_SET_FOLDER

Required argument

WEIGHTS

Required argument