Writing Reviews
The goal of a Boost library review is to improve a candidate library through constructive criticism. At the end a decision must be made: is the candidate library good enough at this point to accept into Boost? If not, we hope to have provided enough constructive criticism for it to be improved and accepted at a later time. The Boost.Serialization library is a good example of how constructive criticism resulted in revisions resulting in an excellent library that was accepted in its second review.
Your comments may be brief or lengthy. The review manager needs your evaluation of the library. If you identify problems in your evaluation, try to categorize them as minor, serious, or showstoppers.
Ask questions for the library authors, on the Boost Developers Mailing List, so all interested parties can see it. Authors are interested in defending their library. If you don’t get a response to your question quickly, be patient; if it takes too long or you don’t get an answer you feel is sufficient, ask again or try to rephrase the question. Clarity is important, English is not the native language of many Boost developers. Try to get your questions answered before you submit your review.
Here’s a structured approach to writing a review of a library, covering all the bases:
Introduction
Introduce yourself, and your knowledge of the problem domain. Remember that input from a newcomer to a domain is valuable. Briefly introduce the library and its purpose as you see it.
Scope
Describe the scope of your review, such as a focus on any particular area of the API, or any particular type of problem within the domain. Be clear on the level of commitment you made to the evaluation (a quick reading, in-depth testing of the API or a subset of it, focus on any particular area such as documentation, tutorials, portability, or similar).
Design and Architecture
Evaluate the design principles and architectural decisions of the library. Discuss the modularity, extensibility, and flexibility of the library’s design. Assess how well the design aligns with Boost’s guidelines and best practices.
Implementation
Provide an overview of the key functionality and features offered by the library. Evaluate the completeness and robustness of the feature set. Discuss any unique or innovative features that differentiate this library from existing alternatives.
Evaluate the usability of the library from your developer’s perspective. Discuss the ease of integration, API usability, and critically the learning curve.
Performance
If you wrote some test code, which is highly recommended, assess the performance characteristics of the library, including not only runtime efficiency but resource usage. Include code snippets from your test code in your evaluation, in particular highlighting the code you wrote to prepare for an API call into the library, and the code following the call to handle the result/data/errors appropriately.
If possible, benchmark the library against similar solutions or existing libraries in terms of speed, memory footprint, and scalability.
Testing and Reliability
Assess the reliability and stability of the library, including error handling and corner case handling. Discuss any known issues, limitations, or areas for improvement in terms of reliability. It is key to address the clarity and usefulness of error messages, and how they helped or hindered you in locating any particular problem.
Portability and Compatibility
If you are able, evaluate the portability of the library across different platforms, compilers, and environments. Give precise details on the tools or environment you used. Assess the library’s compatibility with relevant C++ standards and other Boost libraries. Discuss any platform-specific issues, considerations or dependencies that you encountered.
Documentation
Assess the quality and comprehensiveness of the documentation, including tutorials, examples, and API reference materials. Include comments on the ease of navigation within the documentation, the structure of the documentation, and how easy or difficult it was to find what you were looking for.
Conclusion
Briefly summarize the strengths and weaknesses of the candidate library. Critically, provide a recommendation regarding the suitability of the library for inclusion in Boost. If the library falls short in some way, offer suggestions for improvements or areas of focus that would bring the library up to the standard for a positive recommendation for inclusion.
Review Template
If it’s helpful, cut and paste the following template into your email text when you have completed your review:
1. Introduce yourself
* Are you knowledgeable about the problem domain?
* How much effort did you put into your evaluation? A glance? A quick reading? In-depth study?
* Do you have an affiliation to disclose?
2. What is your evaluation of the design and architecture?
* What is your evaluation of the potential usefulness of the library?
* Is the feature set, or subset, of the library available in another Boost or Standard library?
3. What is your evaluation of the implementation?
* Did you try to use the library? With what compiler? Did you have any problems (minor, series, showstoppers)?
* Can you highlight any performance issues?
* Any feedback on testing and reliability?
* Did you investigate portability and compatibility?
4. What is your evaluation of the documentation?
* Any thoughts on the introduction, tutorials, API reference, completeness, helpfulness?
5. My conclusion
* How do you think the community will react?
* What is your verdict: Yes, No, or Yes with changes?Best Practices
From experience, there a number of more common pitfalls when writing reviews and engaging in review feedback discussions, so keep the following in mind:
-
Literally, keep on the same page. Submissions, discussions, comment should always be on the Boost Developers Mailing List and not on Slack, Reddit, in personal email, or any other media.
-
Elaborate in full. If you have series technical feedback on a library then explain yourself in detail. Try to avoid "teaser" type comments where other developers might feel you are onto something, but they are just not sure what.
-
Reviews needs to be self-contained. It’s not a starting point for a discussion. Nobody is obligated to ask you clarifying questions, and there should be no missing parts that you fill in later in subsequent posts. This means that if you have questions about the library that you feel need to be answered by the author or review manager, you should ask these questions before you submit your review.
-
Do not expect complete agreement. Too much compromise, consensus, in engineering endeavors leads to poorer design.
-
Refer to the Reality Check section in the topic on Review Managers for clarity on the role of your review.