[COST-4745] OCPGCP Network data processing SQL #5058
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira Ticket
COST-4745
Network unattributed
project in [COST-4744] Azure network data processing SQL #5056Description
This change will add ocp on azure network processing. This change does a few things:
Identifies Network records from the GCP bill that are associated with a specific Compute Instance that can be tied to an OCP Node
Separates the usage and cost for these records into a distinct row per day, one for inbound traffic, one for outbound traffic when we aggregate the gcp_openshift_daily records up
Filter out the networking records when we are grouping by namespace because these values cannot be attributed to a specific namespace/project (hence the Network unattributed project!)
Perform a new insert into the project daily summary table for the networking records grouped by OCP node
Back populate these records into the OCPUsage table adding a data transfer direction to the group by which has 3 options, IN, OUT, and NULL
NOTE: when GCP renamed Ingress to
Data Transfer in
, Egress was renamed toData Transfer
that sometimes has a conditional of out but sometimes does not. Based on my understanding of this GCP article, Ingress was simply renamed toData Transfer In
and any other data transfer is Egress/OutboundNise has been updated and the test customer yamls now include network in and out records.
Testing
docker exec -it trino trino --server localhost:8080 --catalog hive --schema org1234567 --user admin --debug
Inbound math:
Cost:
2400 = 50 (usage) * 2 (rate) * 24 hours
✅Quantity:
1288.488 = 50 (usage) * 24 hours * 1.07374 (gibibyte to gigabyte conversion)
✅Outbound math:
Cost:
7200 = 30 (usage) * 10 (rate) * 24 hours
✅Quantity:
257.6976 = 30 (usage) * 24 hours * 1.07374 (gibibyte to gigabyte conversion)
✅Release Notes