chore(dashboard): user journey smoke tests #7124

LetItRock · 2024-11-25T13:04:27Z

What changed? Why was the change needed?

Dashboard - the core user journeys smoke tests that cover:

dashboard-created workflow
code-created workflow

TODO (in the future):

we have one drawback: we need to establish independent clerk and api sessions for the tests, I left a comment about it in the code

Screenshots

linear · 2024-11-25T13:04:30Z

NV-4795 Write two E2E user journey as a smoke test for new Dashboard

netlify · 2024-11-25T13:04:46Z

✅ Deploy Preview for novu-stg-vite-dashboard-poc ready!

Name	Link
🔨 Latest commit	`fa860f3`
🔍 Latest deploy log	https://app.netlify.com/sites/novu-stg-vite-dashboard-poc/deploys/674d7a4f4008e900085d165d
😎 Deploy Preview	https://deploy-preview-7124.dashboard-v2.novu-staging.co
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

LetItRock · 2024-11-25T13:06:12Z

apps/api/.gitignore

@@ -121,3 +121,4 @@ backups/

 # Nest.js auto-generated metadata (https://docs.nestjs.com/recipes/swc#monorepo-and-cli-plugins)
 src/metadata.ts
+src/.env.test


removed the .env.test file from git, because the local file that I used for the tests had sensitive clerk credentials

LetItRock · 2024-11-25T13:06:45Z

apps/dashboard/.gitignore

@@ -28,4 +28,6 @@ tsconfig.node.tsbuildinfo
 /test-results/
 /playwright-report/
 /blob-report/
-/playwright/.cache/
+/playwright/
+.env.test


.env.test is the file for the Dashboard env variables

LetItRock · 2024-11-25T13:07:21Z

apps/dashboard/.gitignore

-/playwright/.cache/
+/playwright/
+.env.test
+.env.playwright


.env.playwright is the file for the playwright that has all env variables to setup the clerk and api session

LetItRock · 2024-11-25T13:07:48Z

apps/dashboard/package.json

@@ -5,6 +5,7 @@
  "type": "module",
  "scripts": {
    "start": "vite",
+    "start:test": "vite --mode test",


run dashboard and use .env.test file

LetItRock · 2024-11-25T13:08:21Z

apps/dashboard/package.json

+    "@novu/dal": "workspace:*",
+    "@novu/testing": "workspace:*",


used to initialize the api session

LetItRock · 2024-11-25T13:22:45Z

apps/dashboard/tests/code-created-flow.spec.ts

+  bridgeServer = new BridgeServer({ secretKey: session.environment.apiKeys[0].key, apiUrl: process.env.API_URL });
+
+  const newWorkflow = workflow(workflowId, async ({ step }) => {
+    await step.inApp(
+      inAppStepId,
+      async (controls) => {
+        return {
+          subject: `Hi ${controls.name}! You've been invited to join the Novu project`,
+          body,
+        };
+      },
+      {
+        controlSchema: {
+          type: 'object',
+          properties: {
+            name: { type: 'string', default: 'John' },
+          },
+        } as const,
+      }
+    );
+  });
+  await bridgeServer.start({ workflows: [newWorkflow] });
+  await session.testAgent.post(`/v1/bridge/sync`).send({
+    bridgeUrl: bridgeServer.serverPath,
+  });


Run the bridge server and sync the workflow

LetItRock · 2024-11-25T13:23:04Z

apps/dashboard/tests/code-created-flow.spec.ts

+  await session.testAgent.post(`/v1/bridge/sync`).send({
+    bridgeUrl: bridgeServer.serverPath,
+  });
+  await page.waitForTimeout(2000);


should be removed

LetItRock · 2024-11-25T13:23:44Z

apps/dashboard/tests/global.setup.ts

+  /**
+   * TODO: Currently we are running tests with a single Clerk user and organization.
+   * This approach has a drawback that the tests are not independent from each other and write the data to the same organization.
+   * We should consider creating a new Clerk user and organization for each test using the @clerk/backend package.
+   * Then initialize the BE session with the new Clerk user and organization and update Clerk user metadata from the newly created session.
+   */


hopefully, it explains the problem well

Correct! We should leverage Clerk Backend API, create a user before every run, sign in with that user, and then delete the user at the end.

Regarding the need for Clerk webhooks during user and organization creation, I suggest connecting to Mongo directly from the test code using the response of the Clerk API to create the user and organization record the application needs to function.

On user deletion, I don't think we need to do anything in Mongo as we run in a dockerized environment.

LetItRock · 2024-11-25T13:26:31Z

apps/dashboard/tests/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "commonjs"


The dal and testing packages are CJS modules which can't be imported by the ESM modules.
So this is the trick (together with tsconfig.json) to say "please allow me to import" in my test files :D

LetItRock · 2024-11-25T13:30:28Z

libs/testing/src/ee/ee.organization.service.ts

-  private memberRepository = getEERepository<MemberRepository>('MemberRepository');
+  private memberRepository: MemberRepository;
+
+  constructor({ mockClerkClient = true }: { mockClerkClient?: boolean } = {}) {


For the Dashboard E2E tests we don't want to mock the Clerk client, because we initialize the Clerk session with a real user that has the org and metadata that is used by the Dashboard code.

pkg-pr-new · 2024-11-25T13:41:28Z

Open in Stackblitz

@novu/client

npm i https://pkg.pr.new/novuhq/novu/@novu/client@7124

@novu/framework

npm i https://pkg.pr.new/novuhq/novu/@novu/framework@7124

@novu/js

npm i https://pkg.pr.new/novuhq/novu/@novu/js@7124

@novu/nest

npm i https://pkg.pr.new/novuhq/novu/@novu/nest@7124

@novu/headless

npm i https://pkg.pr.new/novuhq/novu/@novu/headless@7124

@novu/nextjs

npm i https://pkg.pr.new/novuhq/novu/@novu/nextjs@7124

@novu/node

npm i https://pkg.pr.new/novuhq/novu/@novu/node@7124

@novu/notification-center

npm i https://pkg.pr.new/novuhq/novu/@novu/notification-center@7124

novu

npm i https://pkg.pr.new/novuhq/novu@7124

@novu/providers

npm i https://pkg.pr.new/novuhq/novu/@novu/providers@7124

@novu/react

npm i https://pkg.pr.new/novuhq/novu/@novu/react@7124

@novu/react-native

npm i https://pkg.pr.new/novuhq/novu/@novu/react-native@7124

@novu/shared

npm i https://pkg.pr.new/novuhq/novu/@novu/shared@7124

@novu/stateless

npm i https://pkg.pr.new/novuhq/novu/@novu/stateless@7124

commit: 5d0de32

SokratisVidros · 2024-12-10T09:39:06Z

apps/dashboard/tests/dashboard-created-flow.spec.ts

@@ -0,0 +1,115 @@
+import { StepTypeEnum } from '@novu/shared';


How about naming it from the user journey perspective? For example: manage-workflows.e2e.ts.

I added the e2e extension as we have the convention with e2e testing file naming on the API.

SokratisVidros · 2024-12-10T09:39:43Z

apps/dashboard/tests/code-created-flow.spec.ts

@@ -0,0 +1,90 @@
+import { test } from '@playwright/test';


How about naming it from the user journey perspective? For example: sync-workflow.e2e.ts.

SokratisVidros · 2024-12-10T09:39:57Z

apps/dashboard/tests/code-created-flow.spec.ts

+  await bridgeServer.stop();
+});
+
+test('code defined workflow user journey', async ({ page }) => {


Suggested change

test('code defined workflow user journey', async ({ page }) => {

test('sync workflow', async ({ page }) => {

SokratisVidros · 2024-12-10T09:41:01Z

apps/dashboard/tests/dashboard-created-flow.spec.ts

+  await initializeSession({ page });
+});
+
+test('dashboard defined workflow user journey', async ({ page }) => {


Suggested change

test('dashboard defined workflow user journey', async ({ page }) => {

test('manage workflows', async ({ page }) => {

SokratisVidros · 2024-12-10T09:44:29Z

apps/dashboard/tests/global.setup.ts

+  /**
+   * TODO: Currently we are running tests with a single Clerk user and organization.
+   * This approach has a drawback that the tests are not independent from each other and write the data to the same organization.
+   * We should consider creating a new Clerk user and organization for each test using the @clerk/backend package.
+   * Then initialize the BE session with the new Clerk user and organization and update Clerk user metadata from the newly created session.
+   */


Correct! We should leverage Clerk Backend API, create a user before every run, sign in with that user, and then delete the user at the end.

Regarding the need for Clerk webhooks during user and organization creation, I suggest connecting to Mongo directly from the test code using the response of the Clerk API to create the user and organization record the application needs to function.

On user deletion, I don't think we need to do anything in Mongo as we run in a dockerized environment.

SokratisVidros · 2024-12-10T09:49:33Z

apps/dashboard/tests/page-models/create-workflow-sidebar.ts

+  }
+
+  async expectNameValidationError(): Promise<void> {
+    await expect(await this.page.getByText('Name is required')).toBeVisible();


Page object models are usually selectors and request helpers that keep the e2e code dry. To me, expectations belong to the test. Moreover, E2E tests have to be readable and self-explanatory. Dryness is not the most critical requirement.

So, I suggest adding all the expectations in the e2e file and using the page object models for selectors, event listeners and waiting logic. This ensures that page object models are highly reusable and can work with any expectation the e2e will require.

Most importantly, when the e2e is read by the future maintainer, they don't have to jump into every function to see what it does.

SokratisVidros · 2024-12-10T09:50:39Z

apps/dashboard/tests/page-models/create-workflow-sidebar.ts

+    const workflowIdInput = await this.page.locator('input[name="workflowId"]');
+
+    // check the workflow id
+    await expect(await workflowIdInput.inputValue()).toEqual(workflowId);


Following on my previous comment, this is a good example of overloading a page object function with expectation logic that belongs to the test.

SokratisVidros · 2024-12-10T09:51:39Z

apps/dashboard/tests/tsconfig.json

@@ -0,0 +1,10 @@
+{


Is the extra tsconfig.json required?

SokratisVidros · 2024-12-10T09:52:23Z

apps/dashboard/tests/utils/session.ts

+  await page.addInitScript((currentSession) => {
+    window.addEventListener('DOMContentLoaded', () => {
+      localStorage.setItem('nv_auth_token', currentSession.token);
+      localStorage.setItem('nv_last_environment_id', currentSession.environment._id);


This is no longer required in the new dashboard. We can safely remove it.

SokratisVidros · 2024-12-10T09:54:38Z

libs/testing/src/ee/ee.repository.factory.ts


-  return new EEMemberRepository(new CommunityOrganizationRepository(), new ClerkClientMock());
+  if (mockClerkClient) {


Please refer to my previous comment about creating a Clerk user before each test run and doing an actual sign-in so that we can test the actual interaction between Clerk and the application, not a mocked version of it.

LetItRock added 2 commits November 25, 2024 13:48

chore(dashboard): user journey smoke tests

e12abe9

chore(dashboard): fix imports in page models

9f22ad5

LetItRock self-assigned this Nov 25, 2024

LetItRock commented Nov 25, 2024

View reviewed changes

LetItRock requested review from SokratisVidros, BiswaViraj, ChmaraX and desiprisg November 25, 2024 13:31

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

d0a3333

LetItRock temporarily deployed to Linting November 25, 2024 13:33 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 13:33 — with GitHub Actions Failure

chore(dashboard): remove timeout

ceabff2

LetItRock temporarily deployed to Linting November 25, 2024 13:37 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 13:37 — with GitHub Actions Failure

chore(api): revert .env.test

8ad2e31

LetItRock temporarily deployed to Linting November 25, 2024 14:23 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 14:23 — with GitHub Actions Failure

chore(dashboard): fix spell check and adjust pipelines

461275c

LetItRock requested a review from a team as a code owner November 25, 2024 15:52

LetItRock requested review from scopsy and removed request for a team November 25, 2024 15:52

LetItRock temporarily deployed to Linting November 25, 2024 15:52 — with GitHub Actions Inactive

LetItRock temporarily deployed to Linting December 5, 2024 08:32 — with GitHub Actions Inactive

chore(api): fix e2e tests

8a70a51

LetItRock temporarily deployed to Linting December 5, 2024 08:58 — with GitHub Actions Inactive

chore(api): fix e2e tests

76c80c0

LetItRock temporarily deployed to Linting December 5, 2024 17:37 — with GitHub Actions Inactive

chore(api): fix e2e tests

a1a5a6b

LetItRock temporarily deployed to Linting December 5, 2024 17:58 — with GitHub Actions Inactive

LetItRock added 2 commits December 6, 2024 00:46

chore(api): fix e2e tests

9c8f28c

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

674861a

LetItRock temporarily deployed to Linting December 5, 2024 23:47 — with GitHub Actions Inactive

github-actions bot removed the @novu/api label Dec 5, 2024

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

5727282

LetItRock temporarily deployed to Linting December 6, 2024 09:14 — with GitHub Actions Inactive

chore(dashboard): polishing the e2e pipeline

5d0de32

LetItRock temporarily deployed to Linting December 6, 2024 09:42 — with GitHub Actions Inactive

SokratisVidros requested changes Dec 10, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(dashboard): user journey smoke tests #7124

chore(dashboard): user journey smoke tests #7124

LetItRock commented Nov 25, 2024

linear bot commented Nov 25, 2024

netlify bot commented Nov 25, 2024 •

edited

Loading

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

SokratisVidros Dec 10, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

pkg-pr-new bot commented Nov 25, 2024 •

edited

Loading

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

		@@ -0,0 +1,115 @@
		import { StepTypeEnum } from '@novu/shared';

	test('code defined workflow user journey', async ({ page }) => {
	test('sync workflow', async ({ page }) => {

	test('dashboard defined workflow user journey', async ({ page }) => {
	test('manage workflows', async ({ page }) => {


		return new EEMemberRepository(new CommunityOrganizationRepository(), new ClerkClientMock());
		if (mockClerkClient) {

chore(dashboard): user journey smoke tests #7124

Are you sure you want to change the base?

chore(dashboard): user journey smoke tests #7124

Conversation

LetItRock commented Nov 25, 2024

What changed? Why was the change needed?

Screenshots

linear bot commented Nov 25, 2024

netlify bot commented Nov 25, 2024 • edited Loading

✅ Deploy Preview for novu-stg-vite-dashboard-poc ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkg-pr-new bot commented Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify bot commented Nov 25, 2024 •

edited

Loading

pkg-pr-new bot commented Nov 25, 2024 •

edited

Loading