Feat/lp v2 gateway queue #1930

cdamian · 2024-07-29T08:20:34Z

Description

Moved queue logic from LP gateway to a new pallet.

Checklist:

I have added Rust doc comments to structs, enums, traits and functions
I have made corresponding changes to the documentation
I have performed a self-review of my code
I have added tests that prove my fix is effective or that my feature works

lemunozm · 2024-07-29T10:50:42Z

Is this ready to review @cdamian?

cdamian · 2024-07-29T10:54:55Z

Is this ready to review @cdamian?

@lemunozm yes, I don't think I want to add anything else here for now.

lemunozm

Really love how clean and organize is this!!

Some questions below, but it has a ver good look!

lemunozm · 2024-07-29T11:03:37Z

pallets/liquidity-pools-gateway/queue/src/benchmarking.rs

+
+#[benchmarks(
+	where
+		T: Config<Message = LPTestMessage>,


This will probably not pass for the real runtimes. My suggestion is to add a Default bound for Message only when benchmarking it. (which would be an Invalid one)

Option2. Add a worst-case method under runtime-benchmarks in LPEncoding, which allows creation of the worst message possible.

Option3. Add a parameter to the benchmarks that allow to passing the Message, so each message is benchmarked for itself. This will be the most accurate/clean, but not sure if it is possible to pass non-integer parameters to benchmarking functions 🤔

I would go with option 2 since that makes the most sense for benchmarks IMO.

so each message is benchmarked for itself.

Would this be possible? More specifically, does the benchmark test suite execute for each variant of an enum?

The benchmark doesn't know about Message itself. But if it receives the Message as a parameter, it would compute the effect of that Message for that specific call

But not sure if it would work. Option 2 seems good

Something like this you mean?

Done, added a default weight ref time that's also used in the pallet instead of default. PTAL

Ok, just one thing, the issue is still there because the benchmark is tied to LPTestMessage

Been trying things out for a while now and I'm still a bit confused...

Option 2 doesn't really work since LPEncoding is not a trait used as a constraint for the Message type of this pallet, as per these constraints.

Ok, just one thing, the issue is still there because the benchmark is tied to LPTestMessage

~~This will remain true if we use the mocked runtime anyway. The message type that we use here is the LPTestMessage.~~ nvm, this should be fine as long as the LPTestMessage is not part of the actual benchmark tests.

Updated this btw, took me a while to figure out what was happening when running tests and then the check benchmark scripts.

pallets/liquidity-pools-gateway/queue/src/lib.rs

runtime/altair/src/lib.rs

runtime/centrifuge/src/lib.rs

runtime/development/src/lib.rs

runtime/integration-tests/src/cases/liquidity_pools_gateway_queue.rs

pallets/liquidity-pools-gateway/queue/src/lib.rs

…processing logic, use BaseArithmetic for nonce

pallets/liquidity-pools-gateway/queue/src/lib.rs

codecov · 2024-07-29T23:31:44Z

Codecov Report

Attention: Patch coverage is 47.50000% with 42 lines in your changes missing coverage. Please review.

Project coverage is 47.75%. Comparing base (020cf59) to head (c55d8c9).
Report is 5 commits behind head on feat/lp-v2.

Files	Patch %	Lines
pallets/liquidity-pools-gateway/queue/src/lib.rs	53.22%	29 Missing ⚠️
libs/mocks/src/liquidity_pools_gateway_queue.rs	0.00%	4 Missing ⚠️
runtime/development/src/lib.rs	0.00%	3 Missing ⚠️
pallets/liquidity-pools-gateway/src/lib.rs	0.00%	2 Missing ⚠️
runtime/altair/src/lib.rs	0.00%	2 Missing ⚠️
runtime/centrifuge/src/lib.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           feat/lp-v2    #1930   +/-   ##
===========================================
  Coverage       47.74%   47.75%           
===========================================
  Files             180      184    +4     
  Lines           12960    13036   +76     
===========================================
+ Hits             6188     6225   +37     
- Misses           6772     6811   +39

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lemunozm

Just a last thing about Default, but everything else looks great! 👏🏻

libs/traits/src/liquidity_pools.rs

lemunozm · 2024-07-30T07:12:03Z

pallets/liquidity-pools-gateway/queue/src/benchmarking.rs

+	#[benchmark]
+	fn process_message() -> Result<(), BenchmarkError> {
+		let caller: T::AccountId = account("acc_0", 0, 0);
+		let message = T::Message::default();


I am a bit worried about the PoV with batch messages, but I think we can go with it now 👍🏻

Maybe in the next steps we can have something like a MaxEncodedLenSelf trait or similar for cases like these.

Yeah, something like:

#[pallet::weight(T::WeightInfo::process_message().saturating_add(Weights::from_parts(0, T::Message::max_encoded_len())))]

pallets/liquidity-pools-gateway/src/lib.rs

pallets/liquidity-pools/src/message.rs

pallets/liquidity-pools-gateway/queue/src/lib.rs

lemunozm

Everything looks super good! thanks for tackling all my comments/suggestions/nits 🙌🏻

* feat: LPv2 message reorder (#1892) * wip: add new msgs + reorder * refactor: apply renaming * fix: ITs * feat: UpdateRestriction message * fix: cancel_unprocessed_investment IT * fix: fmt * fix: clippy * tests: reanchor solidity to reorder branch * feat: apply hook to AddTranche msg * tests: fix pool_management ITs * wip: fix lp investments * fmt * fix: Tranche namespace * refactor: remove fulfilled from FulfilledRedeemRequest * chore: update lp submodule * minor cleanup * chore: update lp submodule * chore: add missing cleanup * fixes + ignore failing tests * fmt * tests: fix message indices * ignore failing tests (#1910) * LPv2: ForeignInvestments changes (#1895) * add uts (#1905) * main changes for FI * fix correlation precission * minor renames * update investment UTs * update redemption UTs * miscelaneous UTs * minor renames * simplify correlation and embed to the only needed place * doc change * remove unused bound * swapping calls into entities.rs file * merge SwapDone methods into FulfilledSwapHook * fix events * working without pallet-swaps * remove boilerplate for removing entries * minor msg change * minor simplification * correct fulfilled sum after last collect * check OrderIdToSwapId storage * sending the message inside Info closure is not really a problem * send msgs from the entities * remove same currency check in impl.rs * unify hooks * remove pallet-swaps * minor fmt * add docs * add architecture diagram * return cancelled foreign amount from FI interface * update liquidity-pools * add messages to diagram * implement hooks * fix runtimes * adapt integration-tests * fix docs * fix clippy * fix clippy * fix tests after merge * fix foreign investment tests (#1918) * ignore failing tests (#1919) * fix previous merge * LP v2: fix integration tests (#1915) * chore: update submodule to latest `main` 6d7f242c0dd83b1b5a4f6d506370a1f3ecbef9ce * wip: fix ITs * chore: update submodule * fix: remove sender param from `Transfer*` messages * chore: cleanup * docs: setup evm * fix: msg tests * fix: more ITs * fix: missing refactor after rebase * chore: update submodule to 223a0f36edabc675f8c74c47b20e366178df7ca3 * chore: improvements * fmt * Apply suggestions from code review * chore: bump spec_version * fmt: taplo * LPv2: Batch Message serialization (#1920) * custo serialize/deserialize for batch * compiling solution * serialization/deserialization working for batch message * remove gmpf changes * add batch nested protection * faster serialization for submessages * correct termination * add tests for deserialize batch of batch * add into_iter * remove unused Box and add pack methods * fix non_std compilation * non_std * fix max_encoded_len recursion issue * fix submodules * reduce batch limit * feat: add domain hook storage (#1928) * refactor: add domain hook address storage * tests: add gateway lp admin account checks * refactor: use GetByKey * fix: outboundq mock * refactor: hook 20 bytes instead of 32 * fix cargo fmt * Feat/lp v2 gateway queue (#1930) * pallets: Add LP Gateway queue pallet * lp-gateway-queue: Add benchmarks * integration-tests: Add LP gateway tests * docs: Update LP gateway queue entry * lp-gateway-queue: Use default weight for PostDispatchInfo * lp-gateway-queue: Add DEFAULT_WEIGHT_REF_TIME const, extract message processing logic, use BaseArithmetic for nonce * runtime: Add defensive weights for LP gateway queue * lint: Obey * taplo: Obey * pallet: Use DispatchResult for extrinsics * runtime: Update benchmarks and weight info * benchmarks: Add default for Message type (wip) * pallet: Add Default bound to Message type * lp-v2: fix message fields (#1933) * fix: add remark call to borrow proxy * fix: add missing message fields * chore: bump to v0.13.3 * chore: update submodule * chore: enable fixed tests --------- Co-authored-by: Frederik Gartenmeister <[email protected]> * refactor: cleanup my leftovers (#1935) * LPv2: Bump-up foreign investment. Fix failing investment ITs (#1934) * increase version for foreign investments * fix investment issue * fix solidity call for transfers_tokens * fix tests * minimize required tranche investors for IT * fix docs * fix docs * docs: fix hyperlink * refactor: enable ITs for centrifuge + dev runtimes (#1938) * fix: enable ITs for centrifuge + dev runtimes * fmt * fix: revert some centrifuge ITs * fmt * revert centrifuge addition to ITs --------- Co-authored-by: William Freudenberger <[email protected]> --------- Co-authored-by: William Freudenberger <[email protected]> Co-authored-by: Cosmin Damian <[email protected]> Co-authored-by: Frederik Gartenmeister <[email protected]>

cdamian added 5 commits July 26, 2024 00:58

pallets: Add LP Gateway queue pallet

ab8fcd4

lp-gateway-queue: Add benchmarks

53b8472

integration-tests: Add LP gateway tests

705bc49

docs: Update LP gateway queue entry

a8969cb

lp-gateway-queue: Use default weight for PostDispatchInfo

66e3252

cdamian marked this pull request as ready for review July 29, 2024 10:54

cdamian requested review from mustermeiszer, wischli and hieronx as code owners July 29, 2024 10:54

lemunozm reviewed Jul 29, 2024

View reviewed changes

cdamian added 4 commits July 29, 2024 17:12

lp-gateway-queue: Add DEFAULT_WEIGHT_REF_TIME const, extract message …

de162fd

…processing logic, use BaseArithmetic for nonce

runtime: Add defensive weights for LP gateway queue

1699eda

lint: Obey

8e51711

taplo: Obey

02d0b3d

lemunozm reviewed Jul 29, 2024

View reviewed changes

pallets/liquidity-pools-gateway/queue/src/lib.rs Outdated Show resolved Hide resolved

lemunozm reviewed Jul 29, 2024

View reviewed changes

pallets/liquidity-pools-gateway/queue/src/lib.rs Show resolved Hide resolved

cdamian added 3 commits July 29, 2024 23:16

pallet: Use DispatchResult for extrinsics

9c44e90

runtime: Update benchmarks and weight info

bd5c49a

benchmarks: Add default for Message type (wip)

7be3549

cdamian force-pushed the feat/lp-v2-gateway-queue branch from 5bb5402 to 7be3549 Compare July 29, 2024 21:47

lemunozm reviewed Jul 30, 2024

View reviewed changes

pallet: Add Default bound to Message type

c55d8c9

lemunozm approved these changes Jul 30, 2024

View reviewed changes

cdamian merged commit 81d193b into feat/lp-v2 Jul 30, 2024
11 of 13 checks passed

cdamian deleted the feat/lp-v2-gateway-queue branch July 30, 2024 10:52

cdamian restored the feat/lp-v2-gateway-queue branch July 31, 2024 17:29

cdamian deleted the feat/lp-v2-gateway-queue branch July 31, 2024 18:21

cdamian restored the feat/lp-v2-gateway-queue branch August 2, 2024 09:30

cdamian deleted the feat/lp-v2-gateway-queue branch August 2, 2024 09:31

cdamian mentioned this pull request Aug 5, 2024

Add multi-router support in LP gateway. #1893

Draft

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/lp v2 gateway queue #1930

Feat/lp v2 gateway queue #1930

cdamian commented Jul 29, 2024 •

edited

Loading

lemunozm commented Jul 29, 2024

cdamian commented Jul 29, 2024

lemunozm left a comment

lemunozm Jul 29, 2024 •

edited

Loading

cdamian Jul 29, 2024

cdamian Jul 29, 2024 •

edited

Loading

lemunozm Jul 29, 2024 •

edited

Loading

lemunozm Jul 29, 2024

lemunozm Jul 29, 2024

cdamian Jul 29, 2024

lemunozm Jul 29, 2024

cdamian Jul 29, 2024 •

edited

Loading

cdamian Jul 30, 2024

codecov bot commented Jul 29, 2024 •

edited

Loading

lemunozm left a comment

lemunozm Jul 30, 2024

cdamian Jul 30, 2024

lemunozm Jul 30, 2024

lemunozm left a comment

Feat/lp v2 gateway queue #1930

Feat/lp v2 gateway queue #1930

Conversation

cdamian commented Jul 29, 2024 • edited Loading

Description

Checklist:

lemunozm commented Jul 29, 2024

cdamian commented Jul 29, 2024

lemunozm left a comment

Choose a reason for hiding this comment

lemunozm Jul 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cdamian Jul 29, 2024 • edited Loading

Choose a reason for hiding this comment

lemunozm Jul 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cdamian Jul 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jul 29, 2024 • edited Loading

Codecov Report

lemunozm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lemunozm left a comment

Choose a reason for hiding this comment

cdamian commented Jul 29, 2024 •

edited

Loading

lemunozm Jul 29, 2024 •

edited

Loading

cdamian Jul 29, 2024 •

edited

Loading

lemunozm Jul 29, 2024 •

edited

Loading

cdamian Jul 29, 2024 •

edited

Loading

codecov bot commented Jul 29, 2024 •

edited

Loading