The CQWW contest committee, specifically, the log checking sub-group, uses many tools to scrutinize all logs. One of the tools is used to establish Benchmarks across various parameters. Benchmarking only establishes expected “norms” by comparing logs to each other. One potential issue with Benchmarking is that if everybody cheats, then everybody still “looks normal.” Fortunately, the vast majority do not cheat.
CQWW has decided to pilot making one of our benchmarking tools public. The expectation is that more reviewers will result in wider and faster identification of potential issues (flags).
Continue reading “Crowd-sourcing for identifying outliers in CQWW logs”