Using E2E Tests as Documentation

Working on a codebase or using an application itself can sometimes be challenging without proper documentation. One effective way to bridge this gap is by using end-to-end (E2E) tests as a form of documentation. E2E tests not only verify that the application works as expected but also provide concrete examples of how different features and functionalities are intended to be used.

What is E2E Testing?

End-to-end (E2E) testing is a software testing methodology that involves testing an application from start to finish. The goal is to simulate real user scenarios and interactions to ensure that the entire system works together as intended.

The way these tests are executed often involves automating user actions such as clicking buttons, filling out forms, and navigating through the application.

Playwright for E2E Testing

Playwright is a popular framework for E2E testing that supports multiple browsers and platforms. Playwright supports video recording of test runs, which are primarily used for debugging failed tests. However, these videos can also serve as valuable documentation for understanding how the application behaves in various scenarios.

Writing E2E Tests as Documentation

Normally, E2E tests are written to validate specific functionalities, meaning they focus on just "getting the job done" as simply and quickly as possible.

For example:

In our case the intentions of the tests are to be clear, understandable and slow-paced, so that anyone watching the video can easily follow along and grasp how the application works.

The following sections outline strategies to achieve this goal.

Cursor Visibility

By default, the cursor is hidden and jumps instantly between actions. We can modify this behavior to make the cursor movement visible and smooth.

In order to have a visible cursor we need a custom CSS that renders a cursor element on the screen.

Good news that such libraries exist, for example mouse-helper

Note: The above library may not work for all applications mainly due to the Z-index of the cursor element conflicting with the application's own elements, so a custom implementation may be required.

Execution Speed and Cursor Movement

We add a cooldown period between actions to allow viewers to observe what is happening on the screen.

This can be easily achieved by using playwright's slowMo option.

Cursor movement can also be slowed down using the delay and steps options of the click methods.

Other Visual Aids

The same logic of injecting custom CSS can be applied to other visual aids, such as pointing arrows, highlights, or annotations to draw attention to specific parts of the UI during the test execution.

Video Collection and Presentation

After the tests are executed, the resulting videos should be collected and organized.

First we can map each video with a title and a short description of what the video demonstrates.

For example, have a JSON file like:

[

  {

    "title": "User Login",

    "description": "This video demonstrates how a user can log into the application using valid credentials.",

    "videoPath": "videos/user-login.mp4"

  },...

]

The web page can then dynamically load this JSON file and present the videos in a user-friendly manner, allowing users to easily navigate through different functionalities of the application.

Tests as Documentation in Mill

The Mill build tool - where our team contributes features such as Android and Spring Boot support - follows a similar approach by treating end-to-end tests as documentation.

In Mill’s case, these E2E tests are simply CLI commands paired with their expected output. Those tests are then collected and published directly on the documentation site, turning real, executable scenarios into documentation that users can follow along with confidence.

A good example is the Android Native Sample page which showcases the test commands exactly as users would run them, along with the output they should expect.

About the Author: Stylianos Anastasiou is a Software Engineer at Vaslabs LTD and the main contributor behind the internal tools Vaslabs uses for e2e documentation.