Skip to content

AI-Integrated Production Process

A description of specific stages of the production process and how AI could be utilized, along with the pros and cons of using it.

Syllabus

A syllabus is an outline of topics covered in a course, shown through various lesson, article, project, and quiz content items.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate syllabi, check out the Syllabus section in Curriculum Research Findings.
  • Generate an outline of a course.
  • Specify a target audience and any prerequisite knowledge we expect learners to have before taking this course.
  • Specify the scope of the course. Should the course be an introduction course, an intermediate course, or an advanced course?

Pros and Cons of Using AI

✅ It might be useful to generate bullet point lists of topics that could be covered in a course.

✅ It might be easier to use for courses on well-defined technical concepts.

❌ It might not be great for courses on niche or new topics.

Design Doc

Lesson Outline

A lesson outline is a skeleton that describes at a high level, including what learners will learn throughout the lesson. The lesson outline clarifies the scope of the prospective lesson, including anything that might be out of scope. They also define learning outcomes.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate lesson outlines, check out the Content Item Outlines section in Curriculum Research Findings.
  • Generate a starting point
  • Generate a situation/metaphor for the lesson to follow to help demonstrate concepts.
    • I.e., picking where to live using nested if statements.

Pros and Cons of Using AI

✅ AI could assist in writing a first draft of these documents so that CDevs have a starting place to edit.

✅ It could also help generate ideas for situations that lesson exercises can follow in their examples/code.

❌ AI is missing context on what Codecademy does and does not cover and where. Making decisions about scope is unlikely to be an area in which AI is helpful.

Learning Standards

Learning Standards are short, simple passages between 1 and 4 sentences that describe the conceptual and technical facts that will be introduced throughout the lesson. Learning standards should avoid pronouns and can be written in passive voice. Learning standards should always be prefaced with the topic they’re describing, such as “In Python…” or “In UX Design…”.

Notes on Using AI

Notes on using AI

  • Likely will need to provide examples to GPT so it can continue in this style.
  • Generate learning standards given a lesson outline/learning outcomes

Pros and Cons of Using AI

✅ AI seems to be skilled at this short generation, so this is likely to be a good use-case.

❌ We need to watch out for “sounds correct” issues here, as the lesson content is based on these.

Lesson Breakdown

The Lesson breakdown is a list of suggested titles for the different exercises (or, pages) of a lesson. A lesson is typically made up of between 6-10 exercises.

Notes on Using AI

Notes on using AI

  • Generate a lesson outline. Use the output as a reference to draft list of suggested titles for the lesson.

Pros and Cons of Using AI

❌ It might be difficult to construct a prompt to get a desired output that meets Codecademy style and standards of a lesson breakdown.

Lesson Draft

Narrative Drafts

Lesson exercises can be grouped into three categories: - An introduction exercise - Instructional exercises - A review exercise

Introduction exercise

An exercise is a section of a lesson. An introduction exercise is the first exercise of the lesson that provides an overview of what learners will learn.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate an introduction exercise, check out the Introduction and Review Sections section in Curriculum Research Findings.
  • Generate a draft of an introduction to the main topic of the lesson.
    • An overarching learning outcome for the lesson could be provided to capture the main topic of the lesson.
  • The AI draft will need to be modified to follow review guidelines but also to make sure that:
    • the contents of the introduction consider learners’ prerequisite knowledge
    • the content is within the intended scope of the lesson

Pros and Cons of Using AI

✅ GPT could be great at generating introductory content for a general audience. It could be a great use case for the first lesson of a Learn X course that does not have any prerequisites.

❌ It might be more time-consuming to review and edit the draft if we are creating a lesson on very specific content that needs to consider the context of learner’s learning journey (i.e., there is a lot of prerequisite content and the lesson appears in a specific context)

Instructional exercise

An exercise is a section of a lesson. An instructional exercise covers one or more learning objectives through a narrative with examples.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate exercises, check out the Lessons section in Curriculum Research Findings.
  • Generate a draft based on the learning standards covered in the exercise.

Pros and Cons of Using AI

✅ An AI-generated draft could be a great starting point for lessons that introduce a new syntax.

❌ It might be more time-consuming to get an exact prompt to generate an exercise on a desired topic of the exercise. This might be more true for exercises for intermediate and advanced lessons.

Review exercise

An exercise is a section of a lesson. A review exercise is the last exercise of the lesson that provides a short summary of what was covered in the lesson and a bullet point list of learning objectives covered in the lesson.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate a review exercise, check out the Introduction and Review Sections section in Curriculum Research Findings.
  • Generate a bullet point list of topics that summarizes what was covered in the lesson based on either 1) a list of learning standards or 2) lesson narratives

Pros and Cons of Using AI

✅ AI is generally good at summarizing text.

❌ It might be easier/faster to write the review exercise rather than generate it using GPT. Usually, the review exercise contains a list of learning outcomes that exist at this point in the production process.

Workspace Code

A workspace code is a collection of one or more files provided to learners in a lesson exercise. The workspace code exemplifies how the syntax and concepts covered in the exercise are used.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate workspace code, check out the Code Snippets section in Curriculum Research Findings.
  • Generate code that uses syntax that was introduced in the exercise.

Pros and Cons of Using AI

✅ AI-generated code would be a great first draft containing the necessary overall structure.

❌ We will need to experiment further on whether AI can generate separate starting and solution codes.

❌ The generated code may not be original or an engaging example.

Accompanying Art

An artwork can appear either in the workspace or within the narrative to visually illustrate a concept or syntax described in the exercise narrative.

Notes on Using AI

Notes on using AI

  • Generate an idea for an artwork that illustrates a concept covered in the exercise. This could be used in an art request ticket.

Pros and Cons of Using AI

✅ It could be worthwhile to generate different ideas for creative artwork.

❌ If we are looking to create a simple diagram, it will not be necessary to involve AI.

Instructions

Instructions describe activities learners should complete to practice what they learned in the exercise. Instructions can be: - Short or long answer questions to check learner’s understanding - Description of activities learners should complete in a specific platform

Notes on Using AI

Notes on using AI

  • Generate ideas for learner activities to help learners practice a particular concept.
  • Generate short or long answer questions to check learner’s understanding

Pros and Cons of Using AI

✅ It could be worthwhile to generate different ideas for learner activities.

✅ It might be great to generate short and long answer questions

❌ We might have something very specific in mind already for learner activities.

Checkpoints

Checkpoints provide instructions for tasks learners should complete to practice what they learned in the exercise. Checkpoints ask learners to write code in the workspace and evaluate learner code with tests.

Notes on Using AI

Notes on using AI

  • More research needs to be done on generating checkpoint instructions.

Checkpoint Test Code

Checkpoint tests evaluate the code learners have written in the workspace by following checkpoint instructions. The checkpoint test should use a specific test suite.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate workspace code, check out the Code Snippets section in Curriculum Research Findings.
  • Use GPT to write regular expressions to include in your component tests.
  • Specify a testing suite to use to test the outcome of a code snippet.

Pros and Cons of Using AI

✅ After some testing, it seems that GPT can do regular expressions pretty well, and does a good job of accounting for the scenarios we try to account for.

❌ All test code should be thoroughly checked and tested.

❌ For consistency, we should keep our templates, and allow GPT to make things (like Regex) that we fill them in with.

### Examples
Component Test Example

Sure! Here's a modified regular expression that matches print("Hello World") with any number of whitespace characters (including spaces, tabs, and newlines) before and/or after the string:

/[\\t\\n\\r\\f\\v\\s]*print\\s*\\(\\s*"Hello World"\\s*\\)[\\t\\n\\r\\f\\v\\s]*/

Here's a breakdown of what each part of the regular expression does:

  • [\\t\\n\\r\\f\\v\\s]* matches zero or more whitespace characters (including tabs, newlines, carriage returns, form feeds, vertical tabs, and spaces). The square brackets define a character class, and the `` indicates that the class can appear zero or more times.
  • print matches the string "print" literally.
  • \\s* matches zero or more whitespace characters.
  • \\( matches a left parenthesis character literally.
  • \\s* matches zero or more whitespace characters.
  • " matches a double quotation mark character literally.
  • Hello World matches the string "Hello World" literally.
  • " matches a double quotation mark character literally.
  • \\s* matches zero or more whitespace characters.
  • \\) matches a right parenthesis character literally.
  • [\\t\\n\\r\\f\\v\\s]* matches zero or more whitespace characters (including tabs, newlines, carriage returns, form feeds, vertical tabs, and spaces).

Note that the [\\t\\n\\r\\f\\v\\s] character class at the beginning and end of the regular expression is not strictly necessary if you only want to match the code in a single line of Python code. However, if you want to match the code in a multi-line block of Python code, you might need to include these characters to account for whitespace at the beginning or end of lines.

Jest Test Example Sure! Here's an example of a React app with an `App` component that renders an `h1` element with the text "Hello World": 
import React from "react";

function App() {
  return (
    <div>
      <h1>Hello World</h1>
    </div>
  );
}

export default App;
In this example, the `App` component returns a `div` element that contains an `h1` element with the text "Hello World". The `export default` statement at the bottom of the file makes the `App` component available for use in other parts of the app. Sure! Here's an example test using React Testing Library to check if an `h1` element with the text "Hello World" is rendered to the DOM in the `App` component: 
import React from "react";
import { render, screen } from "@testing-library/react";
import App from "./App";

describe("App component", () => {
  it("renders an h1 element with the text 'Hello World'", () => {
    render(<App />);
    const h1Element = screen.getByText("Hello World");
    expect(h1Element).toBeInTheDocument();
  });
});
In this test, we use the `render` function from React Testing Library to render the `App` component. We then use the `getByText` function from the `screen` object to find an `h1` element with the text "Hello World". Finally, we use Jest's `toBeInTheDocument` matcher to check if the `h1` element is present in the DOM. If the `h1` element with the text "Hello World" is found, the test will pass.

Practice Content

Review Cards

Review cards are similar to learning standards, and usually borrow language directly from learning standards. They are learner-facing reminders of the essential facts of lesson. Unlike learning standards, they can also feature image or code examples that demonstrate the concepts.

Notes on Using AI

Notes on using AI

  • Generate summary text based on a learning standard.
  • Use AI to add a code snippet or image description that succinctly demonstrates a concept.

Pros and Cons of Using AI

✅ AI tends to be good at generating code snippets.

❌ Review cards tend to be close to learning standards and are often copy-pasted. The actual text may not be necessary.

Assessments

Multiple Choice: A multiple-choice assessment contains a question stem and 2 or more answer choices, of which only one can be correct. It optionally may also include an image or a code block to accompany the question stem. These assessments are aligned to learning standard(s) and test learner's understanding of those learning standard(s).

Fill in the BlankA Fill in the Code assessment contains a question stem and a partially completed code or paragraph block, along with a variety of correct and incorrect code snippets that the learner must select to complete the code block correctly. These assessments are aligned to a learning standard and test a learner's understanding of that learning standard.

Notes on Using AI

Notes on using AI

  • For research notes on using AI to generate workspace code, check out the Quizzes/Assessments section in Curriculum Research Findings.
  • It’s been said that multiple-choice assessments have been a strong suit for GPT. FITB, on the other hand, has been described as too troublesome.

Pros and Cons of Using AI

✅ AI has been proven to be great for generating multiple-choice assessments.

❌ For other assessment types, like Fill-in-the-blank, it might not be worthwhile to generate using GPT.