Managing Reviews

Before a library can be scheduled for formal review, an active Boost member (not connected with the library submission) must volunteer to be the Review Manager for the library. Members may contact a library author on- or off-list to express interest in managing the review. The library author has to accept a person as a review manager.

Before submitting a library, it will help to understand the role of the review manager.

The Role of Review Manager

The review manager works through the following process:

  1. Checks the submission to make sure it really is complete enough to warrant formal review. For full requirements, refer to the Library Requirements. If necessary, work with the submitter to verify the code compiles and runs correctly on several compilers and platforms.

  2. Finalizes the schedule with the Review Wizards and the submitter.

  3. Posts a notice of the review schedule on both the Boost developers' mailing list and the Boost-announce mailing list.

    1. The notice should include a brief description of the library and what it does, to let readers know if the library is one they are interested in reviewing.

    2. If the library is known to fail with certain compilers, mention them in the review notice so reviewers with those compilers won’t waste time diagnosing known problems.

    3. It is advised to send the notice to each mailing list in a separate e-mail, otherwise online e-mail to news gateways could get confused.

  4. Inspects the Boost library catalogue for libraries which may interact with the new submission. These potential interactions should be pointed out in the review announcement, and the authors of these libraries should be privately notified and urged to participate in the review.

  5. Urges people to do reviews if they aren’t forthcoming.

  6. Follows review discussions regarding the library, moderating or answering questions as needed. It is the review manager’s job to give reviewers the benefit of the doubt and to try to coax a serious review out of participants, by asking pertinent questions.

  7. Asks the Review Wizards for permission to extend the review schedule if it appears that too few reviews will be submitted during the review period.

  8. Decides if there is consensus to accept the library and if there are any conditions attached. Consensus is not the same as a vote. The review manager has discretion to weigh opinions based on authority or thoughtfulness.

    Reality Check

    The review manager is not supposed to impartially reflect the community opinion, as expressed in the form of formal reviews. Instead, the review manager’s role is to decide whether the library should be accepted, and the reviews help them with this, rather than decide for them. Simply put, the review manager does not impartially tally votes - reviews are not votes.

    A review manager can write a review themselves, though this process is independent of their role of review manager.

    The review process is more like a court - with the judge (the review manager) asking "Does the library meet the required standards for acceptance as a Boost library?". This question is resolved by various advocates presenting their cases, criticizing other advocates cases, presenting facts, logical arguments, their own experiences, and so on. The judge’s job is to weigh all this and reach a decision. And they may add conditions on acceptance, or not. If concensus between judge and authors cannot be reached on meeting any conditions - the process ends without a resolution (the mistrial).

    The review manager is - for better or worse - personally responsible for making the decision, defending the results, and dealing with future criticism. Their name will be public. This is not a job for everyone.

    Sometimes the judging process is going to produce irreconcilable differences, and continued discussions about the "verdict" are a fact of life. With any process there will be "winners" and "losers". So it’s not about everyone being happy at the end of the day.

    If, years from now, someone would like to know why a library was accepted or rejected, they only needs to read the court documents (the review posts and the review summary), and should come away with the correct understanding of what happened and why.

  9. Posts a notice of the review results on the Boost users mailing list as well as the Boost developers' mailing list and Boost-announce mailing list. A rationale is also helpful, but its extent is up to the review manager. If there are suggestions, or conditions that must be met before final inclusion, they should be stated. Concerns about the timeliness or quality of the review report should be brought to the Review Wizards off-list.

    Ideally, the review summary should contain all the information the manager has taken into account when coming to a decision. If there were discussions out of band, they need to be summarized. If there were discussions on the list that haven’t made their way into the formal reviews, they should be summarized too. Refer to the Best Practices section on Writing Reviews for some issues to look out for.

Becoming a Review Manager

To manage a review, you should have experience with the review process and ideally expert knowledge of the library’s domain. To volunteer to become a review manager, contact the current Review Wizards.

Review Wizards

Currently the review wizards are Mateusz Łoskot (mateusz@loskot.net) and John Phillips (johnphillipsithaca@gmail.com).

See Also

Edit this Page