Skip to content

Production Reviews

This documentation is for anyone reviewing another person's content.

The purpose of a review is to reinforce or improve aspects of a person's behavior or performance. Review comments should be made from this perspective. When making review comments, think about what behavior you are trying to either reinforce or improve.

Setting Expectations

Prior to beginning the production of content items, set expectations with authors by communicating:

  • Who will be conducting the review (a CDev, an SME, both, etc..)
  • Who has the final say on making revisions (typically the CDev leading the project)
  • How feedback will be provided (via Slack, Github, in a Google doc, in a workshop, etc…)
  • How to prioritize feedback (ex: suggestions vs. required changes)
  • How much time the author can expect the reviewer to take to complete a review (typically 1-2 days)
  • How the feedback should be resolved (ex: in place vs. in the next deliverable)
  • Reference the specific curriculum standards that will be used to review the content item.

Provide clear and actionable feedback

When reviewing content, feedback should be clear and actionable. Providing clear and actionable feedback is important because it:

  • Minimizes miscommunication
  • Helps the reviewee learn from their mistakes
  • Saves the reviewee time to figure out how to resolve issues
  • Decreases number of reviews and time needed for reviews (which saves $$$)

To ensure our feedback is clear and actionable, we encourage the usage of the following system:

  • When identifying an issue, avoid simply stating that something is "wrong". Describe clearly what the problem is and what the potential impact of that problem is to our learners.
  • When possible, provide alternatives to problematic content. If you are struggling to come up with an alternative, say so and ask the reviewee to brainstorm other ideas.
  • Provide links to documentation if possible.

Comment Labels

When leaving comments, follow the formatting provided by Conventional Comments (open the link for more details!). This strategy formats each comment by prepending a label and/or decoration. For example:

Nitpick (editorial): Make sure to use the royal "we" rather than "you" when addressing the learner (unless you are providing direct instruction).

See the table below for descriptions of each label and decoration:

Label Description
praise Praises highlight something positive. Try to leave at least one of these comments per review. Do not leave false praise (which can actually be damaging). Do look for something to sincerely praise.
nitpick Nitpicks are small, trivial, but necessary changes. Distinguishing nitpick comments significantly helps direct the reader's attention to comments requiring more involvement. Often, making nitpick edits using suggestion mode in Google Docs can be sufficient without making a new comment.
suggestion Suggestions propose improvements to the current subject. It's important to be explicit and clear on what is being suggested and why it is an improvement. Consider offering a valid solution and decorations (explained below) to further communicate your intent.
issue Issues highlight specific problems with the subject under review. These problems can be user-facing or behind the scenes. It is strongly recommended to pair this comment with a suggestion. If you are not sure if a problem exists or not, consider leaving a question.
question Questions are appropriate if you have a potential concern but are not quite sure if it's relevant or not. Asking the author for clarification or investigation can lead to a quick resolution.
thought Thoughts represent an idea that popped up from reviewing. These comments are non-blocking by nature, but they are extremely valuable and can lead to more focused initiatives and mentoring opportunities.
chore Chores are simple tasks that must be done before the subject can be “officially” accepted. Usually, these comments reference some common process. Try to leave a link to the process description so that the reader knows how to resolve the chore.

Comment Decorations

There are occasions in which it would beneficial to further clarify the type of comment you are leaving. For these situations, we can use decorations. Decorations could include the level of importance (i.e. non-blocking), the perspective you're coming from (i.e. editorial), or any other additional "tag" you'd like to give your comment.

Decoration Description
(non-blocking) A comment with this decoration should not prevent the subject under review from being accepted. This is helpful for organizations that consider comments blocking by default.
(blocking) A comment with this decoration should prevent the subject under review from being accepted, until it is resolved. This is helpful for organizations that consider comments non-blocking by default.
(if-minor) This decoration gives some freedom to the author that they should resolve the comment only if the changes ends up being minor or trivial.
(editorial) A comment with this decoration indicates an issue with grammar, spelling, typos, or clarity of sentences.
(pedagogical) A comment with this decoration indicates an issue with the pedagogical structure, scope, and/or logical flow of the content item
(technical) A comment with this decoration indicates an issue with the technical accuracy of the content. The SME may be tagged to help resolve this issue.

Comment Examples

  • Praise: Great hook! I love how you put the learner into a real world scenario. This makes it clear what problem this technology solves while adding context for the lesson.
  • Nitpick (editorial): Make sure to use the royal "we" rather than "you" when addressing the learner (unless you are providing direct instruction).
  • Suggestion (editorial): This sentence may be difficult to understand since it has a lot of technical language in one sentence. Consider breaking it up into two sentences and simplifying some of the language like so: "suggestion…"
  • Issue (pedagogical, blocking): This exercise task asks the learner to practice a skill that they haven't encountered in the lesson yet. They will learn about this skill later on. Please take some time to think about how we can either rearrange the exercises or remove this extra step from this exercise.
  • Question (pedagogical): Do you plan on providing a link to the documentation for this topic? Here is the link for reference:
  • Thought (if-minor, non-blocking): It could be interesting to add a real-world example of a famous cyber attack here! Only do this if you can easily find an example as it isn't essential but could be a fun extra piece of detail.
  • Chore: Start reasons with a one- or two-word affirmation like 'Well done!' or 'Correct!'. Incorrect reasons can begin with 'Not quite.' Here's the link to the docs