The YAML You Actually Need for CI/CD

Variations of YAML are commonly used to define CI/CD pipelines. At first glance, YAML can appear scary. The good news is that you don’t have to be a YAML guru to effectively use it to create your own CI/CD pipelines. There is a practical subset of the official YAML specification that is all that’s needed.

The goal here is to get you capable of reading YAML, not writing it quite yet. Writing YAML will be discussed separately.

Before we dive in, you must understand how indentation impacts YAML. In YAML, indentation is how structure is expressed. Anything indented under a key belongs to that key. This is how YAML communicates hierarchy and relationships.

It’s also important to understand what a string is in YAML (and elsewhere in the technical world). If you are not a Software Developer or Engineer, a string is simply text. In many programming languages, strings are enclosed in quotation marks. In YAML, however, strings are not usually enclosed in quotation marks unless absolutely necessary.

You’re now ready to dive in.

First, CI/CD pipelines need to store data. So how do we store data in YAML? Easy:

key: value

In this syntax, replace key with a variable (or storage location) name, and replace value with the data value you want to store.

Technically speaking, this structure is called a map, because you are mapping a value onto a key and using the key to retrieve that value later. Maps can store all kinds of data and are used heavily in pipeline definitions.

Next, you may see syntax like this:

set: |

This syntax looks scary, but it’s really not. All this says is that the lines following this one will be indented and treated as a collection of work to be accomplished. Technically speaking, this is called a literal block. In CI/CD pipelines, literal blocks are commonly used to group work that the pipeline should execute, such as a series of commands.

Speaking of grouping, here is an example of grouping related configuration using nested maps:

a_nested_map:
  key: value
  another_key: Another Value
  another_nested_map:
    hello: hello

Notice that the value of the key a_nested_map is itself a collection of maps. We can even nest a map inside another map. This syntax is heavily used in pipeline definitions to group configuration that belongs together.

Sometimes in a pipeline, we need to execute work in a specific order. This is where sequences come in:

a_sequence:
  - Step 1
  - Step 2

A sequence is simply an ordered list. In CI/CD pipelines, sequences are commonly used to represent steps that are executed in order. An important thing to understand is that maps and sequences are very often combined in real pipeline definitions.

Before we finish, you may notice lines that begin with a # character. This character indicates a comment. The # can appear at the beginning of a line or later in the line. Everything after the # character is ignored when the pipeline is executed. Comments exist solely to help humans understand what the pipeline is doing.

There is still more to learn about practical YAML for QA, but this is enough to get you reading and understanding CI/CD pipeline definitions–no matter the CI/CD platform.

Here is a fully-commented example CI/CD pipeline definition that demonstrates these concepts.

# Example CI/CD pipeline using GitHub Actions.
# GitHub Actions will be explained later. Right
# now, all we're focused on is the YAML syntax.

# Simple key/value map or variable
name: API Tests (pytest)

# A nested map
on:
  workflow_dispatch:
    inputs:
      base_url:
        description: "Base URL of the deployed API"
        required: true
        type: string

# Another nested map with a few surprises!
jobs:
  test:
    runs-on: ubuntu-latest
    
    # A map containing a sequence
    steps:
      # A sequence is strictly the set of lines
      # beginning with a - character and does
      # not include the label above it
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      # Each - introduces one item in a list,
      # and that item can be a map with several
      # keys
      - name: Install dependencies
        # A literal block that executes
        # some Python code
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run pytest against deployed API
        env:
          BASE_URL: ${{ inputs.base_url }}
        # Another literal block
        run: |
          pytest -v