If you are able to understand Shog's graph you will notice that NAAs handled by the moderators are twice as much as those handled by "community", which considering that the community is more strict handling NAAs (there's a stat saying that community tend to reject more flags on the low quality review queue than moderators, can't find it) it doesn't make sense that the "exception handlers" handle something so mundane like NAA flags where the community could, arguably, do this job.
Taking into account the above, I think we are incurring an opportunity cost by not allowing the community to handle more flags (since there are many more users than moderators, moderators' time is more valuable). But for that we need the data to identify if this hypothesis holds any water or if there is something else at play.
For that I would need the following:
- Number of NAA/VLQ flags on answers that had to be elevated to the moderator queue (not being handled on the LQRQ), the reason was:
- The answer was score >0
- The answer was accepted
- The timeout was reached (one hour after the flag is raised, it moves to the moderator queue)
- What was the LQRQ leaning towards, and it agreed with the action taken by the moderator?
- Break down of number of flags handled by the LQRQ in 5 minute brackets and its outcome. (I think average, std and median should be given as general measure too)
That would help us to identify whenever the LQRQ needs tuning and what exactly needs to be tuned.
