Save lines in tests with Payload Helper functions

I’ve recently been looking into how I can clean up some of my test code and make it smaller. I feel I’ve found a wonderful solution that anybody can implement with ease.

Normal Test

Imagine we've got an API endpoint that responds to requests for information about employees at an organization. The API lives at https://some.address and returns a JSON body that looks like

GET https://some.address/employee/1/

{
    "uuid"        : uuid.uuid4(),
    "name"        : "Joshua Cloward",
    "work"        : "Authentise",
    "title"       : "Junior Programmer",
    "office"      : "SLC",
    "description" : "super cool",
}

Not only can we perform a GET on a single resource, we can also perform a POST to create a new resource and do a GET against https://some.address/employee/ and get a list of all employee resources. We'll call this a LIST. I know it's not a real HTTP method, but work with me here. We use it when we mean a GET against a REST endpoint that will return a collection of resources.

Because we are responsible engineers, we want to test this endpoint so we can etch its behavior in stone for future generations. Let's start with a basic test against the GET

def test_get_request():
    employee = {
        'uuid'        : uuid.uuid4(),
        'name'        : 'Joshua Cloward',
        'work'        : 'Authentise',
        'title'       : 'Junior Programmer',
        'office'      : 'SLC',
        'description' : 'super cool',
    }
    create_employee(employee)

    response = requests.get('https://some.address/{}/'.format(employee['uuid']))

    assert response.json() == employee

I'm going to use py.test which expects to find test functions named test_* which it will run. We use assert statements to test expected conditions against what our code actually does.

This test relies on a few functions. create_employee will insert the employee into our database so our API has some data to provide. Then we use the excellent requests library to make the actual HTTP request. Finally, we check that the response data contains the employee we expected.

So far so good. Let's add a second test to see if we can LIST all of the employees in the system.

import uuid.uuid4
import create_employee

def test_list_request():
    first_employee = {
        'uuid'        : uuid.uuid4(),
        'name'        : 'Joshua Cloward',
        'work'        : 'Authentise',
        'title'       : 'Junior Programmer',
        'office'      : 'SLC',
        'description' : 'super cool',
    }
    second_employee = {
        'uuid'        : uuid.uuid4(),
        'name'        : 'Nick Cloward',
        'work'        : 'Authentise',
        'title'       : 'Senior Engineer',
        'office'      : 'SLC',
        'description' : 'Not as cool as his brother Josh',
    }
    create_employee(first_employee)
    create_employee(second_employee)

    response = requests.get('https://some.address/employee')
    expected = [first_employee, second_employee]

    assert response.json() == expected

You'll notice there's a fair amount of duplication here already. We have two different employees taking up most of the code in this test and they are mostly very similar. Let's live with it for now and finish creating our third test.

For the POST we will be creating 1 employee and then querying the DB to see if the employee we created is there.

def test_post_request():
    employee = {
        'uuid'        : uuid.uuid4(),
        'name'        : 'Joshua Cloward',
        'work'        : 'Authentise',
        'title'       : 'Junior Programmer',
        'office'      : 'SLC',
        'description' : 'super cool',
    }

    response = requests.post('https://some.address/', json=employee)
    assert response.status_code == 201

    created_employee = query_employee_table.by_uuid(employee['uuid'])
    assert created_employee == employee

I've introduced a new function in this test, query_employee_table.by_uuid. It does exactly what it sounds like - queries the database and returns an employee record that matches the given UUID. The return value will be a dict exactly like what we used when POST ing the resource.

Now that we've got all of our tests written we can take a look at just how long these 3 simple tests have become since they each do all of their own setup work independently.

import uuid.uuid4
import create_employee

def test_list_request():
    first_employee = {
            'uuid'        : uuid.uuid4(),
            'name'        : 'Joshua Cloward',
            'work'        : 'Authentise',
            'title'       : 'Junior Programmer',
            'office'      : 'SLC',
            'description' : 'super cool',
    }
    second_employee = {
            'uuid'        : uuid.uuid4(),
            'name'        : 'Nick Cloward',
            'work'        : 'Authentise',
            'title'       : 'Senior Engineer',
            'office'      : 'SLC',
            'description' : 'Not as cool as his brother Josh',
    }
    create_employee(first_employee)
    create_employee(second_employee)

    response = requests.get('https://some.address/', json=payload)
    expected = [first_employee, second_employee]

    assert response.json() == expected

def test_get_request():
    employee = {
            'uuid'        : uuid.uuid4(),
            'name'        : 'Joshua Cloward',
            'work'        : 'Authentise',
            'title'       : 'Junior Programmer',
            'office'      : 'SLC',
            'description' : 'super cool',
    }
    create_employee(first_employee)

    response = requests.get('https://some.address/{}'.format(employee['uuid']), json=payload)
    expected = [first_employee, second_employee]

    assert response.json() == expected

def test_post_request():
    employee = {
            'uuid'        : uuid.uuid4(),
            'name'        : 'Joshua Cloward',
            'work'        : 'Authentise',
            'title'       : 'Junior Programmer',
            'office'      : 'SLC',
            'description' : 'super cool',
    }

    response = requests.post('https://some.address/', json=employee)
    created_employee = query_employee_table.by_uuid(employee['uuid'])

    assert created_employee == employee
    assert response.status_code == 201

This is a pretty basic API endpoint test setup. It tests exactly what we need it to. But in reality the heart of our testing code is only 2 lines: Doing the request and then comparing the results. The rest is setting up data or moving it around. So what if we tried to reduce the number of lines by refactoring the code that is devilishly simple?

Meet our helper function the savior of lines

Lets try coming up with a helper function that can fix some of this duplication we have going through our tests.

def _employee_info(name, title, description):
    return {
        'uuid'        : uuid or uuid.uuid4(),
        'name'        : name,
        'work'        : 'Authentise',
        'title'       : title,
        'office'      : 'SLC',
        'description' : description,
    }

Right here we are setting up our helper function. The purpose is to save lines of code and help readability. We could make this helper smarter by also adding in the creation of the employee or even sending the request in here as well. If we were to do that it would do more on its own, but it would be used everywhere else less, making it less effective. That is not what we want. We want our helper function to be used in many places and often.

Cleaning up the list request test

So now that we have this helper function for our employee's info lets try implementing it into the test where it will be of most use the LIST.

import uuid.uuid4

def _employee_info(name, title, description):
    return {
        'uuid'        : uuid or uuid.uuid4(),
        'name'        : name,
        'work'        : 'Authentise',
        'title'       : title,
        'office'      : 'SLC',
        'description' : description,
    }

def test_response():
    first_employee = _employee_info(name='Joshua Cloward', title='Junior Programmer', description='super cool')
    second_employee = _employee_info(name='Nick Cloward', title='Senior Dev', description='not as cool as brother josh')
    create_employee(first_employee)
    create_employee(second_employee)

    response = requests.get('https://some.address/', json=payload)
    expected = [first_employee, second_employee]

    assert response.json() == expected

Now this test looks much better we were able to cut the code we used in setting up the values in half. We then replaced them with two lines that accomplish the exact same thing. This also allows us to put a little bit more focus on the values that matter like me being way cooler then my brother.

Implementing this in the rest of the tests

Now that we have one test looking good it shouldn't take much to get the other two tests up to par. So lets try implementing this into our 'GET' and 'POST' tests.

import uuid.uuid4

def _employee_to_dict(name, title, description, uuid=None):
    return {
        'uuid'        : uuid or uuid.uuid4(),
        'name'        : name,
        'work'        : 'Authentise',
        'title'       : title,
        'office'      : 'SLC',
        'description' : description,
    }

def test_list_request():
    first_employee = _employee_to_dict(name='Joshua Cloward', title='Junior Programmer', description='super cool')
    second_employee = _employee_to_dict(name='Nick Cloward', title='Senior Dev', description='not as cool as brother josh')
    create_employee(first_employee)
    create_employee(second_employee)

    response = requests.get('https://some.address/', json=payload)
    expected = [first_employee, second_employee]

    assert response.json() == expected

def test_get_request():
    employee = _employee_to_dict(name='Joshua Cloward', title='Junior Programmer', description='super cool')
    create_employee(employee)

    response = requests.get('https://some.address/', json=payload)

    assert response.json() == expected

def test_post_request():
    employee = _employee_to_dict(name='Joshua Cloward', title='Junior Programmer', description='super cool')
    response = requests.post('https://some.address/', json=employee)
    created_employee = query_employee_table.by_uuid(employee['uuid'])

    assert created_employee == employee
    assert response.status_code == 201

This is our final product and it looks good. We replaced all of the code that handles an employee's info with our new helper function removing tons of lines. Not only that but also improving readability drastically and we can easily point out the difference between each test. Also if we needed different titles or names for more employess it would be easy to change and it would put more emphasis cause theres less clutter with information that doesn't matter.

Drawbacks

Engineering is all about tradeoffs. So what do we tradeoff by doing our tests this way?

• Less emphasis on all of the expected values

Tests are supposed to assert things - these are our expected values. By writing our tests this way a casual reader isn't sure exactly what our tests expect. Fortunately with py.test we can get detailed statements about what the differences are in the assert statements. That makes this less painful. Still, as we build these tests to be more and more complex, it can get harder and harder to trace how you build up your expected output.

Honestly that's all I've got for the 'cons'. What are the pros?

Advantages

• Saves lines

Our final files is much shorter. Shorter code means less to maintain and fewer bugs. The most bug-free code is the code that didn't get written.

• Easier to read

This may seem like the opposite of my one con. But stay with me: The updated tests look way cleaner and it's easy to see the differences between the two tests. The updated tests have lots of these mostly-useless blocks of code tucked away at the top of the program, and not distracting you from whats important when reading this test.

• Easy to implement

There isn't a huge difference between both sets of tests. We really just copied the payload put into a helper function and then called that helper function with some values. So this is very easy addition that has the ability to be used in many places.

Conclusion

Pros

• Saves lines

• Easier to read

• Easy to implement

• Simple

Cons

• Less emphasis on expected values

With all this included I believe this is a easy first step into refactoring your tests and getting used to helper functions. As well as help your tests contain less lines, but also more readable in the aspect of pointing out what is important to the test. I hope you guys learned a bit about helper functions and cleaning up your tests. As always have a wonderful day and look forward to more blogs from your friends at Authentise.