[Testing, Tooling] feat: implement in-memory networks #290

bryanchriswhite · 2023-12-21T18:53:20Z

Summary

Human Summary

Adds implementations for the InMemoryCosmosNetwork interface from #289:

BaseInMemoryCosmosNetwork
"Abstract" implementation intended to be embedded in "concrete" implementations. It implements the following interface members:
- #GetClientCtx()
- #GetNetworkConfig()
- #GetNetwork()
Additionally, it includes the following convenience methods:
- #InitializeDefaults()
- #GetLastAccountSequenceNumber()
- #NextAccountSequenceNumber()
inMemoryNetworkWithSessions()
Implementation where #Start() creates InMemoryNetworkConfig#NumSuppliers number of genesis suppliers and #NumSuppliers * #AppToSupplierRatio number of applications where #AppToSupplierRatio number of applications will match one service with each supplier (in distinct pairings). It also creates corresponding on-chain accounts (auth module) for each genesis actor. It exposes #CreateClaims() and #CreateProofs() methods for each session * InMemoryNetworkConfig#NumSession number of session.
inMemoryNetworkWithGateways()
Implementation where #Start() creates InMemoryNetworkConfig#NumApplications and #NumGateways number of genesis applications and gateways, respectively. It also creates corresponding on-chain accounts (auth module) for each genesis actor.

Issue

Type of change

Select one or more:

Testing

Run all unit tests: make go_develop_and_test
Run E2E tests locally: make test_e2e
Run E2E tests on DevNet: Add the devnet-test-e2e label to the PR. This is VERY expensive, only do it after all the reviews are complete.

Sanity Checklist

I have tested my changes using the available tooling
I have performed a self-review of my own code
I have commented my code, updated documentation and left TODOs throughout the codebase

testutil/network/sessionnet/accounts.go

Co-authored-by: harry <[email protected]> Co-authored-by: Daniel Olshansky <[email protected]>

Co-authored-by: Daniel Olshansky <[email protected]> Co-authored-by: harry <[email protected]>

…im-proof * pokt/main: Fix bug introduced by #252 where genesis file was no longer being copied to the right location [Docs] Introduce Docusaurus documentation (#252) [Cleanup] Centralzie websocket url -> endpoint changes (#272) refactor: `NewMinedRelay` to shared testutil (#262) fix: PR template typo 2 (#269) [Testing] Unit tests, E2E tests, logging and other Fixes / "Touchups" (#253) [SDK] feat: Have distinct JSON-RPC and gRPC urls (#261)

…or/supplier-keys * issues/141/refactor/claim-proof: chore: review feedback improvements chore: review feedback improvements Fix bug introduced by #252 where genesis file was no longer being copied to the right location [Docs] Introduce Docusaurus documentation (#252) [Cleanup] Centralzie websocket url -> endpoint changes (#272) refactor: `NewMinedRelay` to shared testutil (#262) fix: PR template typo 2 (#269) [Testing] Unit tests, E2E tests, logging and other Fixes / "Touchups" (#253) [SDK] feat: Have distinct JSON-RPC and gRPC urls (#261)

Co-authored-by: Daniel Olshansky <[email protected]>

…ctor/supplier-errors * issues/141/refactor/supplier-keys: chore: review feedback improvements chore: review feedback improvements Fix bug introduced by #252 where genesis file was no longer being copied to the right location [Docs] Introduce Docusaurus documentation (#252) [Cleanup] Centralzie websocket url -> endpoint changes (#272) refactor: `NewMinedRelay` to shared testutil (#262) fix: PR template typo 2 (#269) [Testing] Unit tests, E2E tests, logging and other Fixes / "Touchups" (#253) [SDK] feat: Have distinct JSON-RPC and gRPC urls (#261)

This reverts commit af531b0.

h5law

Overall this is such an amazing PR! Love the attention to detail, bumped a few things (I think) got overlooked but really 11/10 this is great work.

testutil/network/basenet/accounts.go

testutil/network/basenet/network.go

testutil/network/gatewaynet/network.go

testutil/network/sessionnet/network.go

testutil/network/genesis.go

Olshansk · 2024-01-11T17:10:55Z

testutil/network/sessionnet/sessions.go

+
+// cliEncodeSessionHeader encodes the given session header as a base64-encoded
+// string.
+func cliEncodeSessionHeader(t *testing.T, sessionHeader *sessiontypes.SessionHeader) string {


Both optional

Do we need cli in the method header?

Thoughts on moving this into the session module types with the sessionHeader as a receiver?

This is only used for CLI testing purposes. I don't think it would be reused anywhere.

testutil/network/sessionnet/sessions.go

testutil/network/sessionnet/claims.go

Olshansk · 2024-01-12T15:50:13Z

testutil/network/sessionnet/claims.go

+ require.NoError(t, err)
+ require.Equal(t, float64(0), responseJson["code"], "code is not 0 in the response: %v", responseJson)
+
+ // TODO_TECHDEBT: Forward the actual claim in the response once the response is updated to return it.


I don't fully understand this TODO given that we're returning it below. Can you expand?

Updated:

// TODO_TECHDEBT: Forward the claim in the response, as opposed to constructing // an equivalent one here once the response and respective message handler is // updated to include it.

testutil/network/basenet/accounts.go

h5law

Great work, amazing - love to see how its coming together. Left a few comments, once they are done I think this one is good to go, lets get it in!

testutil/network/basenet/accounts.go

testutil/network/sessionnet/genesis.go

Olshansk · 2024-01-19T22:37:15Z

@h5law Just posting here for posterity, but let's leave this until after the v0.50 migration.

We may be leverage some of the newer patterns (example) as we integrate it with our specific business logic.

Also spoke about it with @bryanchriswhite as well

Co-authored-by: h5law <[email protected]>

testutil/network/basenet/accounts.go

Co-authored-by: h5law <[email protected]> Co-authored-by: Daniel Olshansky <[email protected]>

Co-authored-by: h5law <[email protected]>

Co-authored-by: Daniel Olshansky <[email protected]>

bryanchriswhite · 2024-01-22T10:40:38Z

@h5law Just posting here for posterity, but let's leave this until after the v0.50 migration.

We may be leverage some of the newer patterns (example) as we integrate it with our specific business logic.

Also spoke about it with @bryanchriswhite as well

@Olshansk I would definitely hold off on #307 but I think we can push this through as reviewers are able to allocate bandwidth to it.

I think it represents a significant improvement from the existing tooling and can start to be used independent of the refactoring of existing integration tests.

Olshansk · 2024-01-25T20:34:07Z

Discussed offline with @bryanchriswhite. Holding off on this until after the migration

bryanchriswhite · 2024-04-02T11:43:06Z

Closing, superseded by #412 & #417.

bryanchriswhite added 7 commits December 13, 2023 14:56

refactor: NewMinedRelay to shared testutil

4bcd8de

refactor: claim & proof protobuf types

673911c

refactor: rename supplier keeper UpsertClaim & UpsertProof

fd55896

refactor: misc. claim-side improvements

43cbf0e

chore: add TODOs

ef3cd99

refactor: supplier module keys

f509fbf

refactor: supplier module errors

6817364

bryanchriswhite added testing Test (or test utils) additions, fixes, improvements or other tooling Tooling - CLI, scripts, helpers, off-chain, etc... labels Dec 21, 2023

bryanchriswhite added this to the Shannon TestNet milestone Dec 21, 2023

bryanchriswhite self-assigned this Dec 21, 2023

bryanchriswhite mentioned this pull request Dec 21, 2023

[Testing, Tooling] refactor: in-memory network usage #285

Merged

11 tasks

bryanchriswhite commented Dec 21, 2023

View reviewed changes

testutil/network/sessionnet/accounts.go Outdated Show resolved Hide resolved

bryanchriswhite force-pushed the issues/141/prep/in-memory-network branch from 5414460 to e3c3f8c Compare December 21, 2023 20:10

bryanchriswhite force-pushed the issues/141/feat/in-memory-network branch 2 times, most recently from 11c019d to f03177e Compare December 22, 2023 12:28

bryanchriswhite and others added 14 commits December 22, 2023 14:36

chore: review feedback improvements

6e60952

Co-authored-by: harry <[email protected]> Co-authored-by: Daniel Olshansky <[email protected]>

chore: review feedback improvements

edc4e41

Co-authored-by: Daniel Olshansky <[email protected]> Co-authored-by: harry <[email protected]>

chore: review feedback improvements

dfb067d

Co-authored-by: Daniel Olshansky <[email protected]>

fix: pre-generated keyring accounts

b82e49a

fix: session hydrator

2d52838

feat: add InMemoryCosmosNetwork interface

6a9d6ec

chore: add InMemoryNetworkConfig

5566e5c

chore: add TODO comment

923907b

Revert "fix: pre-generated keyring accounts"

8b57b22

This reverts commit af531b0.

chore: add BaseInMemoryCosmosNetwork abstract impl.

eee5dcf

feat: add inMemoryNetworkWithSessions impl.

4af4132

bryanchriswhite changed the base branch from issues/141/prep/in-memory-network to main January 10, 2024 14:09

bryanchriswhite added 2 commits January 10, 2024 15:14

fixup! chore: review feedback improvements

dc07730

fix: linter errors

a906d45

bryanchriswhite requested a review from Olshansk January 10, 2024 14:19

Merge branch 'main' into issues/141/feat/in-memory-network

cd19c7e

h5law suggested changes Jan 11, 2024

View reviewed changes

h5law reviewed Jan 11, 2024

View reviewed changes

testutil/network/genesis.go Outdated Show resolved Hide resolved

chore: review feedback improvements

12dbc57

bryanchriswhite requested a review from h5law January 12, 2024 11:39

Olshansk requested changes Jan 12, 2024

View reviewed changes

bryanchriswhite mentioned this pull request Jan 18, 2024

[Tokenomics] Post-scaffold cleanup & parameter updates foundation #287

Merged

11 tasks

h5law suggested changes Jan 19, 2024

View reviewed changes

bryanchriswhite and others added 2 commits January 22, 2024 11:24

chore: review feedback improvements

c1774bf

Co-authored-by: h5law <[email protected]>

chore: review feedback improvements

985a817

Co-authored-by: h5law <[email protected]>

bryanchriswhite commented Jan 22, 2024

View reviewed changes

testutil/network/basenet/accounts.go Outdated Show resolved Hide resolved

bryanchriswhite and others added 5 commits January 22, 2024 11:27

chore: review feedback improvements

e905b12

Co-authored-by: h5law <[email protected]> Co-authored-by: Daniel Olshansky <[email protected]>

ch

413f80e

Co-authored-by: h5law <[email protected]>

Merge branch 'main' into issues/141/feat/in-memory-network

911c423

chore: review feedback improvements

3f03074

Co-authored-by: Daniel Olshansky <[email protected]>

chore: review feedback improvements

4516ff5

bryanchriswhite requested review from h5law and Olshansk January 22, 2024 10:38

fix: linter errors

86c7133

bryanchriswhite marked this pull request as draft January 26, 2024 17:39

bryanchriswhite mentioned this pull request Feb 29, 2024

[Testing] Investigate cost/benefit of in-memory network vs. E2E tests #404

Closed

8 tasks

bryanchriswhite closed this Apr 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Testing, Tooling] feat: implement in-memory networks #290

[Testing, Tooling] feat: implement in-memory networks #290

bryanchriswhite commented Dec 21, 2023 •

edited

Loading

h5law left a comment

Olshansk Jan 11, 2024

bryanchriswhite Jan 22, 2024

Olshansk Jan 12, 2024

bryanchriswhite Jan 22, 2024

h5law left a comment

Olshansk commented Jan 19, 2024

bryanchriswhite commented Jan 22, 2024 •

edited

Loading

Olshansk commented Jan 25, 2024

bryanchriswhite commented Apr 2, 2024

[Testing, Tooling] feat: implement in-memory networks #290

[Testing, Tooling] feat: implement in-memory networks #290

Conversation

bryanchriswhite commented Dec 21, 2023 • edited Loading

Summary

Human Summary

Issue

Type of change

Testing

Sanity Checklist

h5law left a comment

Choose a reason for hiding this comment

Olshansk Jan 11, 2024

Choose a reason for hiding this comment

bryanchriswhite Jan 22, 2024

Choose a reason for hiding this comment

Olshansk Jan 12, 2024

Choose a reason for hiding this comment

bryanchriswhite Jan 22, 2024

Choose a reason for hiding this comment

h5law left a comment

Choose a reason for hiding this comment

Olshansk commented Jan 19, 2024

bryanchriswhite commented Jan 22, 2024 • edited Loading

Olshansk commented Jan 25, 2024

bryanchriswhite commented Apr 2, 2024

bryanchriswhite commented Dec 21, 2023 •

edited

Loading

bryanchriswhite commented Jan 22, 2024 •

edited

Loading