Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Load Testing] gherkin/go implementation of test plan #1 #432

Merged
merged 113 commits into from
May 15, 2024

Conversation

bryanchriswhite
Copy link
Contributor

@bryanchriswhite bryanchriswhite commented Mar 15, 2024

Summary

Implementing load test plan #1 using gherkin and go tests.

How to run:

  • Make sure to have the appropriate gateways and relayminers in localnet_config.yaml (max number is 3 for each)
  • Edit the relays_stress.feature to match the testing plan to execute
    And a rate of "1" relay requests per second is sent per application
    And the following initial actors are staked:
      | actor       | count |
      | gateway     | 1     |
      | application | 1     |
      | supplier    | 1     |
    And more actors are staked as follows:
      | actor       | actor inc rate | blocks per inc | max actors |
      | gateway     | 1              | 4              | 3          |
      | application | 2              | 4              | 5          |
      | supplier    | 1              | 4              | 3          |

Run the test with the following command:

go test -v -run TestLoadRelays ./load-testing/tests/...

Issue

Type of change

Select one or more:

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Documentation
  • Other (specify)

Testing

  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR. THIS IS VERY EXPENSIVE, so only do it after all the reviews are complete.
  • Documentation changes: make docusaurus_start

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and referenced any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

red-0ne and others added 27 commits March 1, 2024 15:08
…roof-integration

* pokt/feat/proof-validation:
  chore: Add godoc comments
  chore: Update keeper query clients in-code documentation
  chore: Address review change requests
  [PubKeyClient] Implement PubKeyClient for on/off-chain usage (#413)
…roof-integration

* pokt/feat/proof-validation:
  fix: test error msg assertion
(cherry picked from commit eab6d984e65a9a07014c9a9839e7b55d125147a1)
@bryanchriswhite bryanchriswhite added the loadtest Work related to load testing label Mar 15, 2024
@bryanchriswhite bryanchriswhite added this to the Shannon TestNet milestone Mar 15, 2024
@bryanchriswhite bryanchriswhite self-assigned this Mar 15, 2024
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryanchriswhite Overall, awesome job!

I reviewed everything with the exception of load-testing/tests/relays_stress_helpers_test.go, which is partially reviewed.

As you go through my comments, take note that I was building context / understanding throughout. If something feels "completely off" or redundant, feel free to just resolve.

app/app_config.go Outdated Show resolved Hide resolved
loadtest_manifest.yaml Show resolved Hide resolved
loadtest_manifest.yaml Show resolved Hide resolved
load-testing/tests/anvil.feature Outdated Show resolved Hide resolved
load-testing/tests/anvil.feature Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_test.go Show resolved Hide resolved
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryanchriswhite I stopped reviewing at ~ line 800 in load-testing/tests/relays_stress_helpers_test.go but getting close.

Overall:

  1. I understand your bias towards "DO Repeat Yourself" and aligned with the approach.
  2. I think there's an opportunity to improve some var naming by making it more explicit so please take that into consideration as you go through my comments and/or do a self review.
  3. This PR was pretty big, so if there's a comment that you believe is non-relevant, bias to just resolving it without an explanation than trying to figure it out.

Overall, it's really work through. Undoubted.

I'll pick this up again once all the comments are tended to.

load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
* pokt/main:
  [Code Health] chore: cleanup localnet testutils (#515)
  Zero retryLimit Support in ReplayClient (#442)
  [LocalNet] Add infrastructure to run LLM inference (#508)
  [LocalNet] Documentation for MVT/LocalNet (#488)
  [GATEWAY] Makefile target added to send relays to grove gateway (#487)
  Update README
  [CI] Add GATEWAY_URL envar for e2e tests (#506)
  [Tooling] Add gateway stake/unstake/ logs (#503)
@bryanchriswhite
Copy link
Contributor Author

@Olshansk I think I covered everything except for a few comments:

image

image

I think it's safe to re-review already.

// blocksToMaxAmountIncrementEnd returns the number of blocks that will have
// elapsed when one increment duration **after** the maxActorCount for the given
// actor has been committed.
func (plan *actorLoadTestIncrementPlan) blocksToMaxAmountIncrementEnd() int64 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@red-0ne 👀

Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few more nits/asks, but otherwise g2g!

pkg/relayer/session/proof.go Outdated Show resolved Hide resolved
pkg/relayer/session/claim.go Outdated Show resolved Hide resolved
pkg/relayer/session/claim.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress.feature Show resolved Hide resolved
// Set content type to application/json
req.Header.Set("Content-Type", "application/json")

// Create an HTTP client and send the request
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Understood, but going to use this as an opportunity to call out that we have to get better of reusing (updating / refactoring) each other's helpers rather than creating new ones every time.

load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_helpers_test.go Outdated Show resolved Hide resolved
load-testing/tests/relays_stress_test.go Outdated Show resolved Hide resolved
bryanchriswhite and others added 6 commits May 13, 2024 11:48
* pokt/main:
  [CI] Release artifacts job: binaries and containers (#523)
* pokt/main:
  A couple comment improvements
  Update google drive video to youtube video
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@bryanchriswhite bryanchriswhite merged commit 2f910bc into main May 15, 2024
9 checks passed
@bryanchriswhite
Copy link
Contributor Author

🚨 Do not delete this branch until #521 has changed its base to main 🚨

@Olshansk Olshansk deleted the test/gherkin-load branch May 29, 2024 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
loadtest Work related to load testing
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

[Testing] Load Testing Relays Round #1
4 participants