Topic review ›

Topic review

Review pages

There is one review page. Labels stay in your browser until you submit.

Explore existing rule performance

How it works

1. One pass over every PDF counted every candidate keyword, per section, with and without the shark-or-ray proximity filter. That table is the whole input: a rule is only a way of adding the counts up.
2. So a change to a threshold, a section weight, a keyword, or the proximity filter re-scores every paper in the browser, immediately. No corpus re-run.
3. The reviewer judges papers against a written definition of the concept. Papers well above the line and well below it are spot-checked at random; the rest of the reading time goes on the papers near it.
4. Once a topic has labels, the search in the Optimiser tab proposes the rule that reproduces them best and reports accuracy on held-out labels, so reviewers never need to tune numbers by hand.

Is the instrument right?

Every rule in the topic, with its reviewing workload

"Margin" is the number of papers whose score lies within 0.5 of the line. Minutes assume four papers a minute on the margin plus 80 spot checks; that rate is a guess until someone has done a topic. Click a header to sort, a rule to open it.