Skip to content

Bats (passive) Testing

These are the same as Bats tests (Active) but they do not penalize the user for failing the test by showing them errors in the Learning Environment UI. The main use case for Bats Test (Passive) is for testing learner actions when they are in an interactive shell program (such as nano). Bats tests are supported by all workspace types.

Test Structure

Bats tests generally follow this format:

  1. @test label with the feedback text (In Bats tests, code ticks must be escaped: `)
  2. sqlite command followed by a hidden output filename (usually output.sqlite), the 'solution' SQL query piped into a temporary file (tmp.txt).
  3. A diff command run on the learner's output (output.txt) and the expected output (tmp.txt)
  4. [ "$status" -eq 0 ] to assert that the expected and actual results were identical.

We almost always test that learners are writing queries that will actually return results. It's helpful to start with a test case to catch when a learner's query returns no results, and provide them with feedback to that effect:

@test "The query didn't return any results." {
  touch empty.txt
  run diff output.txt empty.txt
  [ "$status" -eq 1 ]
}

Example Tests

Bats Test for Bash command

setup() {
  source /var/batslib/batslib.bash
}

@test "Did you use <code>echo</code> to create a greeting in the bash profile that begins with <code>Welcome</code>?" {
  run file_exists ~/.bash_profile
  [ "$status" -eq 0 ]
  run grep -q "echo\s\"Welcome" ~/.bash_profile
  [ "$status" -eq 0 ]
  run source ~/.bash_profile
  [ "$status" -eq 0 ]
}

In this example, we are testing if the ~/.bash_profile file exists and that the string "Welcome" exists in the ~/.bash_profile.

Bats Test for SQL command

This is an example of a Bats test for a SQL query. The basic "empty query" test case is usually followed by a test case to check a learner's results against a solution query:

@test "SELECT all columns from the \`table_name\` table." {
  sqlite3 output.sqlite "SELECT * FROM table_name;" > tmp.txt
  run diff output.txt tmp.txt
  [ "$status" -eq 0 ]
}