ci: add UI tests for improved release stability

[damianstasik]

New Pull Request Checklist

• I am not disclosing a vulnerability.
• I am creating this PR in reference to an issue.

Issue Description

This PR sets up a foundation for E2E testing along with two tests to have a starting point and verify whether CI jobs works correctly.

Related issue: #2007

Approach

TODOs before merging

• Remove old electron tests and deps
• Add commands for testing during development
• Include a few tests
• Setup one server, but handle single app and multi-app configs
• Add changes to documentation (guides, repository pages, in-code descriptions)
• A changelog entry is created automatically using the pull request title (do not manually add a changelog entry)

[parse-github-assistant]

Thanks for opening this pull request!

• 🎉 We are excited about your hands-on contribution!

[mtrezza]

Wow, I'm super excited about this! This would be a major milestone for dashboard tests and release stability.

[damianstasik]

@mtrezza I think the core is ready, please take a look when you will have some time. I am not sure about developer guide for testing, is it clear enough?

[mtrezza]

This is amazing!

How could I run cypress using a locally running Parse Server deployment for debugging, instead of using docker?

[damianstasik]

@mtrezza Make sure it is started on the default port, has app ID set to hello and master key to world (default values in the Parse-Dashboard/parse-dashboard-config.json config).

[mtrezza]

Cypress is listening on port 4040, so I think there are some more steps necessary. Could you write a quick 1-2-3 instruction on how to run it with local Parse Server? I think we'd also need that in the contribution docs.

[Moumouls]

Hi @visualfanatic,
First of all thanks for this PR ! 🙂

I have set up Cypress in my company and we use it extensively to cover our apps , and our apps mainly use parse-server as the backend framework.

On my team monorepo, I made the choice to start with a completely native structure:

• Our backend is started by npm script natively, here we are also lucky we can start parse server by CLI, thanks to the parse-server command, if we install parse-server as a devDep
• Also parse-server plays very well with mongodb runner (here "pretest" command: https://github.com/parse-community/parse-server/blob/43cc97d99fa7346739f9e376a22afe3d95439b4c/package.json#L123 )

Here i can suggest may be remove docker need for local development and directly work in native env with parse-server CLI and mongodb runner.

I can also suggest to add 2 commands here:

• "test:watch": which start parse+ mongo and then open cypress in UI mode
• "test": which start parse + mongo and then run cypress (headless)

Then a developer will be able to just clone, npm i and npm test:watch without docker or manually launching/waiting services to be up.

I can also suggest here this popular package to ensure parse server startup before lauching cypress: https://www.npmjs.com/package/start-server-and-test

[Moumouls]

With my team I am very sensitive to the developer experience, i think a native usage with a single command usage (all in one) could be more accessible for futur contributors and will facilitate new cypress tests contributions.

[damianstasik]

I would not want to ditch the docker environment completely, it's just another option for people that prefer it. If we allow using a local backend for E2E tests we need to make sure all features required by tests are enabled and set up properly. It also changes mindset when you write a test, because you have to think about what other data there may be (or not) and how it can affect the test result. What if a flaky test overwrites some important data? I can think of a lot of similar concerns, that is why I would propose to allow both methods. Docker for those who prefer it and local server with a cautionary note about possible side effects. We could provide instructions for both methods with convenient all-in-one commands as you suggested.

@mtrezza @Moumouls what do you think?

[mtrezza]

I'd suggest to focus on the native usage at first. Docker would be an additional obstacle for contributors and could be a challenge especially for those who are just beginning to contribute. On GitHub it may slow down the CI significantly to the point where it's not feasible to run the CI for every PR commit. CI resources are not free, so there is also an economic aspect to it. Maybe docker env could be a complementary PR later on when we have more insight into the behavior of native testing and can tell more specifically whether/why docker is needed.

[mtrezza]

@damianstasik sorry for the long wait time, it slipped through my notifications that you were waiting for a response here. Do you think you could pick this PR up? We just made a release with an obvious bug today that could have been easily prevented with these UI tests. So I'm all eager to see this merged. I've also increased the bounty on the issue because I see you are working on this quite a bit.

[mtrezza]

@visualfanatic just a friendly ping, it would be great if you could pick up this PR, it would be a long-awaited feature.