Creating End‐to‐End Tests

This guide will walk you through how to create end-to-end (e2e) tests using the Playwright framework. The examples are based on the existing installer.spec.js test file. Adjust as needed for your tests.

1. Choose a test file

Test files are created by feature. You can add tests to an existing test file or create a new one.

If there is no test file for the feature you want to test, create it. Make sure the name is descriptive of the feature, and try to keep it short and sweet. The name should follow this format: <feature-name>.spec.js

2. Initialize Dependencies

At the start of your test file, import all the necessary dependencies.

const { _electron: electron } = require( 'playwright' );
const { test, expect } = require( '@playwright/test' );
const { findLatestBuild, parseElectronApp, stubDialog } = require( 'electron-playwright-helpers' );
const { promisify } = require( 'util' );

const path = require( 'path' );
const exec = promisify( require( 'child_process' ).exec );
const fs = require( 'fs-extra' );

3. Define Global Variables

Declare your global variables that will be used across multiple tests.

/** @type {import('playwright').Page} */
let window;

/** @type {import('playwright').ElectronApplication} */
let electronApp;

/** @type {string} */
let latestBuild;

/** @type {import('electron-playwright-helpers').ElectronAppInfo} */
let appInfo;

/** @type {string} */
let exeDir;

/** @type {string} */
let appData;

/** @type {string} */
let installDir;

4. Set Up Test Hooks

The beforeAll hook runs once before all tests. Here you can initialize global setup operations.

test.beforeAll( () => {
  // Add setup logic here - check out existing tests and use their setup logic as a base
} );

The beforeEach hook runs before each test. It's typically used to set up test conditions and initialize variables.

test.beforeEach( async () => {
  // Add setup logic here - check out existing tests and use their setup logic as a base
} );

The afterEach hook is used for cleanup operations that need to be executed after each test.

test.afterEach( async () => {
  // Add cleanup logic here - check out existing tests and use their cleanup logic as a base
} );

5. Write Tests

Write your tests within a test() function call. Define a test name and an asynchronous function that contains the test code.

test( 'lowercase description of what is being tested', async () => {
  // Add test logic here
} );

Remember to put your tests inside a test.describe( 'test suite name', () => { // ... } ); block. This should encompass all your tests, global variables, pre-hooks and post-hooks related to that test suite/group.

6. Test Actions

Use the following actions in your tests to interact with the app. Consult the Playwright docs for more ways to interact with the app via tests. These use CSS selectors to identify HTML elements.

Navigation:

await window.locator( '[page-trigger="installer"]' ).click(); // Click a button or link
await window.locator( '#full-install-button' ).waitFor(); // Wait for an element to be visible

Validate conditions:

expect( await window.locator( '.modal-error' ) ).toBeFalsy(); // Assert a condition

Read contents of HTML elements:

const dlTitle = await window.locator( '#downloads-page .download .progress-title' ).textContent();
const dlId = await window.locator( '#downloads-page .download' ).getAttribute( 'id' );

expect( dlTitle.toLowerCase() ).toContain( 'simitone' );

Stub native Electron dialogs:

await stubDialog( electronApp, 'showOpenDialog', { filePaths: [ 'C:\\example' ] } );

↑ Native dialogs always have to be stubbed, because they are blocking and pause the execution of tests. Since we are not real users running the app, we cannot click on the native dialog and it cannot be simulated.

Fill in form fields:

Text fields (inputs, textareas, contenteditables)

await window.locator( 'input' ).fill( 'value' );

Dropdowns (select options)

await window.locator( '[option-id="Launcher.Theme"]' ).selectOption( 'indigo' );

7. Handling Different Platforms

You can handle platform-specific code using the process.platform variable.

if ( process.platform === 'win32' ) {
  // Windows-specific code
} else {
  // macOS-specific code
}

8. Performing Cleanup Actions

Use an afterEach hook to perform cleanup actions after each test, such as closing the app.

test.afterEach( async () => {
  await electronApp.close();
} );

9. Changing Test Timeouts

By default, Playwright establishes a 30-second timeout per test. Sometimes, more or less time may be needed for particular tests.

Set a timeout for the whole test

You can set the timeout for a whole test function:

test('some test', async ({ page }) => {
  // at the beginning of the test, change the timeout
  test.setTimeout(15 * 60 * 1000); // 15 minutes in milliseconds
});

window.waitForSelector timeout

window.locator().waitFor(), which waits for an HTML element to be visible, has its own timeout, which is not tied to any of the previous methods. To change the ms timeout for this particular action:

await window.locator( `#${dlId}.stopped` ).waitFor( { timeout: 15 * 60 * 1000 } ); // 15 minutes in milliseconds

10. Try Your Tests on Your Machine

Before committing your new tests to the repo, you should try them locally first.

To execute your tests, you can use the npm script included in the package.json file. In the command line, navigate to the root directory of the project and enter the command npm run test.

This script should execute the test suite, providing you with instant feedback about the performance and stability of each test.

Finishing Up

Remember to save your test file with a .spec.js extension, and place it in the src/tests/e2e folder. Playwright's test runner will automatically recognize and run these test files.

Keep adding more tests to your file to cover all the various features and functionalities of the launcher. This way, you'll ensure that everything works as expected and any breaking changes are quickly identified before a release is sent out to end users.

Recommended Visual Studio Code extension

Name: Playwright Test for VSCode
Id: ms-playwright.playwright
Description: Run Playwright Test tests in Visual Studio Code.
Version: 1.0.15
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright