chore: qol improvements and fixes #160
Conversation
📝 WalkthroughWalkthroughThis pull request introduces several new JSON metadata files for prerelease changes across various packages, along with dependency upgrades for the “consola” package in multiple package manifests. Configuration files, environment examples, and indexer implementations have been updated to streamline connection handling and runtime configuration management. The CLI tooling has been refactored to support an authenticated client creation flow with enhanced error checking and logging. Additionally, database schema definitions for the drizzle persistence layer have been restructured, and some legacy helper functions have been removed. Changes
Sequence Diagram(s)sequenceDiagram
participant U as User
participant S as startCommand (CLI)
participant A as createAuthenticatedClient
participant H as getProcessedRuntimeConfig
participant C as createClient (protocol)
U->>S: Invoke start command
S->>A: Call createAuthenticatedClient with config & streamUrl
A->>H: Process runtime configuration
H-->>A: Return merged runtimeConfig
A->>A: Check for DNA_TOKEN in env
A->>C: Call createClient with Authorization header
C-->>A: Return authenticated client instance
A-->>S: Return client instance to CLI
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (4)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (2)
🧰 Additional context used🧬 Code Definitions (1)packages/plugin-mongo/src/mongo.ts (1)
🔇 Additional comments (1)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
jaipaljadeja
force-pushed
the
feat/qol-improvements-fixes
branch
from
70188d0 to
657f08f
Compare
jaipaljadeja
force-pushed
the
feat/qol-improvements-fixes
branch
from
7e5af17 to
5d5407e
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
change/@apibara-plugin-drizzle-8378f0ae-ad6a-4f2d-bf17-777c9e20e4b1.json (1)
3-3: Minor typo in commentThere's a small typo in the comment.
- "comment": "plugin-drizzle: use seperate schema and default connectionString to env", + "comment": "plugin-drizzle: use separate schema and default connectionString to env",packages/cli/src/runtime/internal/helper.ts (1)
1-55: Well-structured configuration handler with clear priority logicThis new utility function properly merges runtime configurations from multiple sources with a clear priority order (env vars > presets > defaults). The error handling is robust and includes helpful error messages with proper cause chaining.
One suggestion to improve robustness:
export function getProcessedRuntimeConfig({ preset, presets, runtimeConfig, }: { preset?: string; presets?: Record<string, unknown>; runtimeConfig?: Record<string, unknown>; +}) { -}) { + // Ensure runtimeConfig is always an object + runtimeConfig = runtimeConfig || {}; let _runtimeConfig: Record<string, unknown> = { ...runtimeConfig }; const envRuntimeConfig = process.env.APIBARA_RUNTIME_CONFIG;
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (34)
change/@apibara-indexer-3f5398ba-95bd-4752-abed-e779ea496399.json(1 hunks)change/@apibara-plugin-drizzle-8378f0ae-ad6a-4f2d-bf17-777c9e20e4b1.json(1 hunks)change/@apibara-protocol-a188449c-6a23-4c55-8e4b-1eebcb28f947.json(1 hunks)change/apibara-d3198897-bb32-4e69-a1de-117791352c44.json(1 hunks)examples/beaconchain-client/package.json(1 hunks)examples/cli-drizzle/.env.example(1 hunks)examples/cli-drizzle/apibara.config.ts(1 hunks)examples/cli-drizzle/indexers/1-evm.indexer.ts(1 hunks)examples/cli-drizzle/indexers/2-starknet.indexer.ts(2 hunks)examples/cli-drizzle/package.json(1 hunks)examples/cli-drizzle/test/ethereum.test.ts(1 hunks)examples/cli-drizzle/test/starknet.test.ts(1 hunks)examples/evm-client/package.json(1 hunks)examples/starknet-client/package.json(1 hunks)packages/cli/package.json(1 hunks)packages/cli/src/cli/commands/dev.ts(1 hunks)packages/cli/src/cli/commands/start.ts(2 hunks)packages/cli/src/core/config/loader.ts(1 hunks)packages/cli/src/core/config/resolvers/preset.resolver.ts(1 hunks)packages/cli/src/core/config/resolvers/runtime-config.resolver.ts(0 hunks)packages/cli/src/hooks/useRuntimeConfig.ts(1 hunks)packages/cli/src/rolldown/config.ts(1 hunks)packages/cli/src/rolldown/plugins/config.ts(1 hunks)packages/cli/src/runtime/dev.ts(2 hunks)packages/cli/src/runtime/internal/app.ts(2 hunks)packages/cli/src/runtime/internal/helper.ts(1 hunks)packages/cli/src/runtime/start.ts(2 hunks)packages/cli/src/utils/helper.ts(0 hunks)packages/indexer/package.json(1 hunks)packages/plugin-drizzle/src/constants.ts(1 hunks)packages/plugin-drizzle/src/helper.ts(3 hunks)packages/plugin-drizzle/src/persistence.ts(4 hunks)packages/plugin-drizzle/src/storage.ts(9 hunks)packages/protocol/src/client.ts(1 hunks)
💤 Files with no reviewable changes (2)
- packages/cli/src/core/config/resolvers/runtime-config.resolver.ts
- packages/cli/src/utils/helper.ts
🧰 Additional context used
🧬 Code Definitions (5)
packages/cli/src/runtime/dev.ts (1)
packages/cli/src/runtime/internal/app.ts (1) (1)
createAuthenticatedClient(94-116)
packages/cli/src/runtime/start.ts (1)
packages/cli/src/runtime/internal/app.ts (1) (1)
createAuthenticatedClient(94-116)
packages/plugin-drizzle/src/storage.ts (1)
packages/plugin-drizzle/src/constants.ts (1) (1)
SCHEMA_NAME(3-3)
packages/cli/src/runtime/internal/app.ts (4)
packages/cli/src/types/virtual/indexers.d.ts (1) (1)
indexers(10-13)packages/cli/src/runtime/internal/helper.ts (1) (1)
getProcessedRuntimeConfig(10-55)packages/cli/src/types/virtual/config.d.ts (1) (1)
config(3-6)packages/protocol/src/client.ts (2) (2)
CreateClientOptions(62-68)createClient(71-89)
packages/plugin-drizzle/src/persistence.ts (1)
packages/plugin-drizzle/src/constants.ts (1) (1)
SCHEMA_NAME(3-3)
🔇 Additional comments (70)
examples/beaconchain-client/package.json (1)
16-16: Updated "consola" Dependency Version:
The "consola" dependency has been upgraded to "^3.4.2", which brings it in line with similar updates across the project. Please verify that this new version does not introduce any breaking changes for logging or error handling in the client.packages/indexer/package.json (1)
76-76: Ensure Consistency with "consola" Upgrade:
The "consola" dependency is now set to "^3.4.2". This update is consistent with other parts of the codebase. Please review the release notes to confirm that there are no incompatible changes that would affect the indexer functionality.change/@apibara-indexer-3f5398ba-95bd-4752-abed-e779ea496399.json (1)
1-7: Metadata File for Pre-release Update:
This new JSON file correctly documents the prerelease change related to the "consola" upgrade for the "@apibara/indexer" package. Please double-check that the contact email ("[email protected]") is accurate for release notifications.examples/evm-client/package.json (1)
16-16: Consistent "consola" Version Across Examples:
The "consola" dependency in the EVM client has been updated to "^3.4.2". This change aligns with other example packages. It is recommended to review documentation for any behavioral changes between versions "^3.2.3" and "^3.4.2".examples/starknet-client/package.json (1)
16-16: "consola" Version Upgrade Verification:
The update to "^3.4.2" for the "consola" dependency in the Starknet client ensures uniformity with the project's dependency versions. Please verify via the release notes that the new version meets all logging and error display expectations.packages/cli/package.json (1)
99-99: Dependency update for consola looks good.This update from version ^3.2.3 to ^3.4.2 aligns with the PR objectives to address issues related to proper console output of error cause traces.
packages/cli/src/cli/commands/start.ts (3)
4-4: Good addition of colors utility.Importing the colors utility from consola will help improve error message formatting.
59-62: Proper child process reference.Good change to capture the spawned process in a variable, which allows for proper event handling.
63-71: Improved error handling and exit code propagation.This enhancement properly handles process termination with improved user feedback by:
- Displaying a clear exit message
- Using color highlighting for error codes
- Ensuring the parent process exits with the same code as the child process
This change improves developer experience when running indexers.
packages/protocol/src/client.ts (1)
24-24: Added Metadata export supports authentication improvements.Exporting the
Metadataclass from nice-grpc enables the authentication token improvements mentioned in the PR objectives. This will allow passing authentication tokens and other metadata in gRPC calls.packages/plugin-drizzle/src/constants.ts (1)
3-3: Good addition of schema constant.Adding the
SCHEMA_NAMEconstant directly supports the PR objective of organizing indexer-related tables into a separate PostgreSQL schema. Using a named constant ensures consistency across the codebase and makes future changes easier.packages/cli/src/core/config/resolvers/preset.resolver.ts (1)
4-6: Added clarification comment is helpful.The added comment clearly documents that this resolver is not currently in use, which is valuable information for developers working with the codebase. This aligns with the PR objective of improving quality of life for developers.
change/@apibara-protocol-a188449c-6a23-4c55-8e4b-1eebcb28f947.json (1)
1-7: New prerelease metadata looks good.This new JSON file properly defines the prerelease metadata for exporting the
nice-grpcMetaDataclass from the@apibara/protocolpackage. The format is correct and includes all necessary fields for version management.change/apibara-d3198897-bb32-4e69-a1de-117791352c44.json (1)
1-7: New prerelease metadata correctly tracks CLI improvements.This JSON file properly documents the prerelease changes related to adding the
createAuthenticatedClientfunction and improvingruntimeConfig, which aligns with the PR objective of streamlining the authentication process mentioned in the PR summary.packages/cli/src/hooks/useRuntimeConfig.ts (1)
4-4: Simplified runtime config retrieval.The implementation has been improved by:
- Replacing a custom
deserializefunction with standardJSON.parse- Using the new environment variable
APIBARA_RUNTIME_CONFIG_HOOK_DATAwhich is set by the newcreateAuthenticatedClientfunctionThis change aligns with the PR objectives of enhancing runtime configuration and improving codebase functionality.
examples/cli-drizzle/test/starknet.test.ts (1)
14-15: Good change to standardize numeric representationRemoving the BigInt literals (
nsuffix) aligns with the broader refactoring effort to simplify configuration handling across the codebase. The change maintains consistency as the test still uses BigInt literals forfromBlockandtoBlockparameters.packages/cli/src/rolldown/plugins/config.ts (1)
14-14: Simplification of configuration parsingReplacing the custom
deserializefunction with standardJSON.parseimproves code maintainability and aligns with the runtime configuration enhancements mentioned in the PR objectives. This change standardizes configuration parsing across the codebase.Be aware that if any configurations previously contained strings with 'n' suffix to represent BigInt values, they would need to be updated as they will no longer be automatically converted to BigInt objects.
change/@apibara-plugin-drizzle-8378f0ae-ad6a-4f2d-bf17-777c9e20e4b1.json (1)
1-7: Well-structured change entry for plugin-drizzleThis change entry correctly documents the modifications to the plugin-drizzle package, which now uses a separate schema for indexer-related tables and sources the connection string from environment variables. This aligns with the PR objective of organizing internal indexer-related tables in a separate PostgreSQL schema.
examples/cli-drizzle/.env.example (2)
2-7: Clear and helpful configuration documentationThe comments provide excellent documentation on the configuration priority order, making it clear how the different configuration sources are merged. This enhances developer experience by making the system behavior more predictable.
7-8: Good implementation of runtime config and authenticationThese new environment variables align perfectly with the PR objectives:
APIBARA_RUNTIME_CONFIGsupports the runtime configuration enhancementsDNA_TOKENimplements the authentication token setting as mentioned in the PR descriptionThe JSON structure in
APIBARA_RUNTIME_CONFIGmatches the changes in the test files where BigInt notation was removed.packages/cli/src/runtime/dev.ts (2)
3-7: Authentication integration improves connection securityThe import statement has been updated to include
createAuthenticatedClientfrom the internal app module, which now handles authentication through environment variables.
47-50: Successfully implemented token-based authenticationThe client creation now uses
createAuthenticatedClientinstead of directly usingcreateClient. This change aligns with the PR objective of streamlining authentication by leveraging theDNA_TOKENenvironment variable.packages/cli/src/cli/commands/dev.ts (1)
121-129: Improved process exit handlingThe handler now properly captures both exit code and signal parameters, adds a visual separator with a blank line, and ensures the process exits with the appropriate status code. This is a good improvement for developer experience.
Good practice to use the nullish coalescing operator (
??) to default to exit code 0 when the code is null.examples/cli-drizzle/package.json (1)
11-11: Added convenient PostgreSQL configuration scriptThis new
start:pgscript provides an easy way to launch the application with a predefined PostgreSQL connection and runtime configuration. It aligns with the PR objective of organizing indexer-related tables in a separate PostgreSQL schema.The script effectively demonstrates how to configure both the database connection and runtime parameters via environment variables, which is valuable for documentation purposes.
packages/cli/src/rolldown/config.ts (2)
99-103: Simplified serialization of runtime configThe code now uses standard
JSON.stringifyinstead of a custom serialization function, which improves maintainability and reduces complexity.
105-108: Improved error message clarityThe error message has been simplified to clearly communicate that all values in the runtime configuration must be JSON serializable, making it more straightforward for developers to understand and fix serialization issues.
packages/cli/src/runtime/start.ts (2)
5-5: Good update to use authenticated clientThe change to import
createAuthenticatedClientaligns with the PR objective of setting authentication tokens based on theDNA_TOKENenvironment variable.
32-35: Improved security with authenticated client connectionReplacing
createClientwithcreateAuthenticatedClientensures proper authentication when connecting to the stream service. This implementation maintains backward compatibility by falling back to unauthenticated connections when no token is present.packages/cli/src/core/config/loader.ts (2)
11-11: Simplified configuration resolver listThe configuration resolver list has been streamlined to only include
resolvePathOptions. This simplification suggests that runtime configuration handling has been refactored elsewhere.
20-27: Good addition of runtime config validationAdding validation for serializable runtime configuration is a sensible enhancement. This check prevents potential issues that could arise when the configuration needs to be serialized later in the application lifecycle.
examples/cli-drizzle/indexers/1-evm.indexer.ts (2)
22-23: Improved database connection handlingThe code now relies on environment variables for database connections instead of explicit parameters. The added comment clearly explains the fallback behavior.
32-32: Enhanced type safety with explicit BigInt conversionExplicitly converting
startingBlockto BigInt ensures proper handling of block numbers, especially for large values that might exceed JavaScript's safe integer range.examples/cli-drizzle/test/ethereum.test.ts (1)
13-14: Consistent approach to handling block numbersUsing regular number literals for configuration while maintaining BigInt types for actual block handling in the test execution is a good approach. This makes the configuration more portable and easier to serialize/deserialize.
examples/cli-drizzle/apibara.config.ts (1)
6-6: Type consistency improvement for block numbersThe removal of the
nsuffix changes these values from BigInt literals to regular number literals. This is a positive change as it aligns with how these values are handled in the indexers, where they're explicitly converted to BigInt when needed (as seen in the starknet indexer).Also applies to: 9-9
examples/cli-drizzle/indexers/2-starknet.indexer.ts (3)
17-17: Clear documentation of default behaviorGood addition of a comment explaining the default connection string behavior, which helps developers understand where the database connection is configured.
27-27: Explicit type conversion ensures consistencyExplicitly converting the
startingBlocktoBigIntensures that the value is always treated as a BigInt regardless of its source format, which aligns with the changes in the configuration file where the BigInt literals were replaced with regular numbers.
69-76: Improved data validation for database operationsThe filter operation ensures that only valid entries with defined and non-NaN numbers are inserted into the database, which improves data integrity and prevents potential errors.
packages/plugin-drizzle/src/helper.ts (2)
61-61: Updated documentation to reflect new default behaviorThe JSDoc now correctly indicates that the default connection string comes from an environment variable with a fallback.
105-107: Enhanced function documentationGood addition of important usage information to the function documentation, making it clear how the connection string defaults are determined.
packages/plugin-drizzle/src/storage.ts (16)
14-14: ImportingpgSchemais appropriateThis new import is necessary to define a proper namespaced schema.
18-18: Shared schema nameReferencing
SCHEMA_NAMEfrom a constants file ensures consistency across the codebase.
21-21: IntroducingROLLBACK_TABLE_NAMEUsing a dedicated constant for the rollback table name is a clear, maintainable approach.
23-23: Defining the schema instanceCreating
schemaviapgSchema(SCHEMA_NAME)helps keep table definitions under the correct namespace.
31-32: Schema-scoped rollback tableReplacing direct
pgTablecalls withschema.tablekeeps tables neatly namespaced, simplifying future management.
51-54: Safeguard schema creationUsing “
IF NOT EXISTS” is good. However, concurrency might still pose edge cases (e.g., two transactions simultaneously creating the same schema). Usually this is safe, but consider verifying.Would you like a script to search for other points in the codebase where schema creation might conflict under parallel operations?
58-58: Qualified table creationEmploying fully qualified paths (
${SCHEMA_NAME}.${ROLLBACK_TABLE_NAME}) is a robust pattern for multi-schema setups.
72-73: Index creationUsing “
IF NOT EXISTS” prevents errors when the index is already present. This is a best practice for incremental deployments.
85-85: Creating the trigger function in the schema“CREATE OR REPLACE FUNCTION” within the schema neatly organizes custom triggers and avoids name collisions.
95-96: Capture DELETE operationsInserting old row data on deletion is essential for rollback. Implementation looks correct.
98-99: Handle UPDATE operationsProperly logging updated rows enables fine-grained rollback. No issues found.
102-103: Handle INSERT operationsOnly the newly inserted row’s ID is needed because there is no prior row state. This is consistent with reorg logic.
143-143: Trigger for each rowAssociating the row-level trigger with
reorg_checkpointensures robust reorg tracking.
195-197: Invalidate rollback entries past the target cursorHelps revert problematic blocks beyond the specified cursor. Implementation is logical.
318-321: Finalize rollback entriesCleaning out older rollback entries once the chain state is deemed final is standard reorg procedure.
351-353: Complete storage cleanupDropping leftover rollback data by
indexer_idprevents orphaned records and stale triggers.packages/cli/src/runtime/internal/app.ts (5)
11-16: Extended imports for client creationPulling
CreateClientOptions,Metadata,StreamConfig, andcreateClientsupports both standard and authenticated usage scenarios.
21-21: ImportinggetProcessedRuntimeConfigCentralizing config resolution in one helper function reduces code duplication and improves maintainability.
27-32: Consolidating runtime configMerging preset and environment-based configurations into
_runtimeConfigensures consistent settings. This approach is clear and robust.
34-34: Exposing runtime configStoring the serialized config in
process.env.APIBARA_RUNTIME_CONFIG_HOOK_DATAhelps unify config usage across modules.
94-116:createAuthenticatedClientfor token-based authThis addition seamlessly injects Bearer tokens into RPC metadata. Consider handling expired or invalid tokens gracefully, but for now, this is a straightforward approach.
packages/plugin-drizzle/src/persistence.ts (10)
2-2: Added imports for multi-conditional queriesPulling in
and,eq,gt,isNull,lt, andsqlis appropriate for advanced query logic.
8-8: ImportingpgSchema,primaryKey,text, andintegerThis allows transitioning to schema-based table definitions, ensuring flexible DB organization.
9-9: ReferencingSCHEMA_NAMEConsistently placing tables within the defined schema fosters better DB structure.
12-14: Renaming table name constantsUsing simpler “checkpoints”, “filters”, and “schema_version” clarifies their purpose and moves away from underscored prefixes.
16-16: Instantiate main schema
const schema = pgSchema(SCHEMA_NAME)organizes table definitions under the shared namespace.
18-19: Define thecheckpointstableMarking it as “not used for migrations” clarifies the table’s ephemeral or internal use case.
25-26: Define thefilterstableUsing the namespaced
schema.tableapproach for partial indexing or block-based filters is consistent with the new structure.
41-42: Define theschemaVersiontableSimilarly applying
schema.tableto track versioning. Good for aligning all internal tables under one schema.
62-66: Creating the schema if absentLaunching “
CREATE SCHEMA IF NOT EXISTS” ensures the code can run on fresh databases without manual setup.
70-76: Creating theschema_versiontableExplicitly scoping the table to
${SCHEMA_NAME}fosters clarity for version control.
jaipaljadeja
force-pushed
the
feat/qol-improvements-fixes
branch
from
380c272 to
625b1cd
Compare
|
Looks good now! |
consolapackage to fix proper console of error cause tracesDNA_TOKENenvironment variable