Skip to content

[DWH][DATA][FLUENTD] Enhancements and bug fixes to align PATH to DWH table structure #393

@fredteumer

Description

@fredteumer

Objective

Ensure as many columns as possible are populated in DWH when reported by PATH. Also clarify what the data fields are and how they are defined.

Origin Document

Data Dictionary:
https://www.notion.so/buildwithgrove/DWH-Data-Dictionary-247a36edfff68034bdc1d7e1772d90c8

Goals

  • Enumerate the possible options for error_type
  • Enumerate the possible options for error_source
  • Align PATH data reported via Fluentd to the existing RELAYS.R2 / RELAYS.D2 table structure

Deliverables

  • Finalized data dictionary
  • Enumerations for columns error_type and error_source
  • Alignment of PATH struct to D2 output
  • Full fluentd integration and data population in R2 / D2

Non-goals / Non-deliverables

  • Enhancements to DWH or RELAYS.D2

General deliverables

  • Comments: Add/update TODOs and comments alongside the source code so it is easier to follow.
  • Testing: Add new tests (unit and/or E2E) to the test suite.
  • Makefile: Add new targets to the Makefile to make the new functionality easier to use.
  • Documentation: Update architectural or development READMEs; use mermaid diagrams where appropriate.

Creator: [GitHub handle of issue owner]
Co-Owners: [OPTIONAL - GitHub handle of co-owner(s)]

Metadata

Metadata

Labels

bugSomething isn't workingdocumentationImprovements or additions to documentation

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions