Skip to content
Build

Allow to pro-actively copy shuffle data to fallback storage #531

Sign in to view logs

Allow to pro-actively copy shuffle data to fallback storage

Allow to pro-actively copy shuffle data to fallback storage #531

Triggered via push November 3, 2025 09:54
@EnricoMiEnricoMi
pushed 8e1f6d3
fallback-storage-proactive-copy
Status Success
Total duration 1h 51m 48s
Artifacts 26

build_main.yml

on: push
Run  /  Check changes
38s
Run / Check changes
Run  /  Base image build
1m 26s
Run / Base image build
Run  /  Protobuf breaking change detection and Python CodeGen check
1m 12s
Run / Protobuf breaking change detection and Python CodeGen check
Run  /  Java 17 build with Maven
43m 57s
Run / Java 17 build with Maven
Run  /  Java 25 build with Maven
40m 38s
Run / Java 25 build with Maven
Run  /  Run TPC-DS queries with SF=1
1h 23m
Run / Run TPC-DS queries with SF=1
Run  /  Run Docker integration tests
1h 43m
Run / Run Docker integration tests
Run  /  Run Spark on Kubernetes Integration test
1h 4m
Run / Run Spark on Kubernetes Integration test
Run  /  Run Spark UI tests
28s
Run / Run Spark UI tests
Matrix: Run / build
Run  /  Build modules: sparkr
30m 12s
Run / Build modules: sparkr
Run  /  Linters, licenses, and dependencies
27m 20s
Run / Linters, licenses, and dependencies
Run  /  Documentation generation
38m 39s
Run / Documentation generation
Matrix: Run / pyspark
Fit to window
Zoom out
Zoom in

Annotations

8 errors and 3 warnings
Run / Run Spark on Kubernetes Integration test
Status(apiVersion=v1, code=404, details=StatusDetails(causes=[], group=null, kind=pods, name=spark-test-app-60fb393a997648aeb047fc19f3d1acc9-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "spark-test-app-60fb393a997648aeb047fc19f3d1acc9-driver" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=NotFound, status=Failure, additionalProperties={})..
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-6299619a495857d4-exec-1".
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-f5f2fd9a49549e3f-exec-1".
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-8dfd3a9a4953aa86-exec-1".
Run / Run Spark on Kubernetes Integration test
sleep interrupted
Run / Run Spark on Kubernetes Integration test
sleep interrupted
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-54839b9a493f8181-exec-1".
Run / Run Spark on Kubernetes Integration test
HashSet() did not contain "decomtest-feacf19a493e9091-exec-1".
Run / Build modules: pyspark-core, pyspark-errors, pyspark-streaming, pyspark-logger
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.
Run / Build modules: pyspark-core, pyspark-errors, pyspark-streaming, pyspark-logger
WARNING conda.cli.main_config:_set_key(451): Key auto_activate_base is an alias of auto_activate; setting value with latter
Run / Build modules: pyspark-core, pyspark-errors, pyspark-streaming, pyspark-logger
The 'defaults' channel might have been added implicitly. If this is intentional, add 'defaults' to the 'channels' list. Otherwise, consider setting 'conda-remove-defaults' to 'true'.

Artifacts

Produced during runtime
Name Size Digest
G-Research~spark~8L12PX.dockerbuild
28.1 KB
sha256:c864bdf41372e4690de7ac47dad0466bee5a07a35b0770b8cdd764b9923ae86f
G-Research~spark~XC9VMK.dockerbuild
29.3 KB
sha256:a722053c99004526601534012ac4683f1489c005b415e2e50ed163b7d326feed
G-Research~spark~XDOJBZ.dockerbuild
27.8 KB
sha256:47bef0644b04e89db7057ab79656bff29b7ed8ccd064f2eba4de05d156295a4f
G-Research~spark~Z4HHXQ.dockerbuild
34.8 KB
sha256:9797d67af123c180d2b401225294cbca37292b7a27906dbe766327c86b2aabbd
site Expired
63.5 MB
sha256:c42d4890635ef037997cf313475b667fdeb63461c1459afe6a495b149fa722e7
test-results-api, catalyst, hive-thriftserver--17-hadoop3-hive2.3
672 KB
sha256:4898d3dd92e01a19405b70c7f8a7c60d3b2530b399dc19c373a3b7cda183148c
test-results-core, unsafe, kvstore, avro, utils, utils-java, network-common, network-shuffle, repl, launcher, examples, sketch, variant--17-hadoop3-hive2.3
821 KB
sha256:3e107ffa390bd2f5f91a977493994721ff89a114199b9179e3e1b62697b83cad
test-results-docker-integration--17-hadoop3-hive2.3
52.3 KB
sha256:a8af502dc20ffc35f531faa7e1d6192c3624f78e938df671f7322cf823936228
test-results-hive-- other tests-17-hadoop3-hive2.3
234 KB
sha256:6736f24659187877f28fb5c772fb557e4568ef1a0288d95eaf39042b915f8a78
test-results-hive-- slow tests-17-hadoop3-hive2.3
224 KB
sha256:2312a6181eb520f31e1851c8923a72c6a0f2c53473d69f5719f1cc8687e77b41
test-results-mllib-local, mllib, graphx, profiler, pipelines--17-hadoop3-hive2.3
496 KB
sha256:4ce33e37b4f958b5d9a93fbba88a4b920cbe5d2eb1684de84fcfb0c571180980
test-results-pyspark-connect--17-hadoop3-hive2.3-python3.11
226 KB
sha256:98bbdd83fb27cbc39b77616eb9769e6f7340c1164bec12f0aa8dfe44578cc15c
test-results-pyspark-mllib, pyspark-ml, pyspark-ml-connect, pyspark-pipelines--17-hadoop3-hive2.3-python3.11
176 KB
sha256:cda33eae9d9200bcfaea3cecec412af522dd5a2e4f5c2538b0223dc75d2743cc
test-results-pyspark-pandas--17-hadoop3-hive2.3-python3.11
196 KB
sha256:ee8de306c0c64795c35bb5d22b923d3a4b2f2948361ed2a9d0a48e51c3730e60
test-results-pyspark-pandas-connect--17-hadoop3-hive2.3-python3.11
206 KB
sha256:9743fd3eded0df82d356859a931bac6a15b33d86a9b1458f760ea285fdda061e
test-results-pyspark-pandas-slow--17-hadoop3-hive2.3-python3.11
173 KB
sha256:2a4511a7cb125869a2b49229a3e70aba83cc5800ab1d479f4b5b466f3f5c6c1d
test-results-pyspark-pandas-slow-connect--17-hadoop3-hive2.3-python3.11
183 KB
sha256:504f0dbf512bcfc26b85df00bad5dd5fbf0fcf28e71fbc944b09887bcf0e27ac
test-results-pyspark-sql, pyspark-resource, pyspark-testing--17-hadoop3-hive2.3-python3.11
249 KB
sha256:7ae834570e0b94966e2bb7754f255e03401e00e808a39af7044f811ad08d38c1
test-results-pyspark-structured-streaming, pyspark-structured-streaming-connect--17-hadoop3-hive2.3-python3.11
41.3 KB
sha256:382c44d3081d881ccdb83311d9b74fe16360a541785fa539b59ffda37c69d895
test-results-sparkr--17-hadoop3-hive2.3
17.4 KB
sha256:9f9f22ab31b5bb43b6709c75a9428b13eacc11650ad69dd049a58db48536c68e
test-results-sql-- extended tests-17-hadoop3-hive2.3
1.24 MB
sha256:7a144f4dcd267f075690dd5318818038fe371f53e6af25cd30151b64588e9a62
test-results-sql-- other tests-17-hadoop3-hive2.3
1.47 MB
sha256:565808d568e0401f7528a4597af0c9d6aaf08b641c5dde1c99ffeae51e0432c9
test-results-sql-- slow tests-17-hadoop3-hive2.3
1.25 MB
sha256:b5e1ab640bd4e477d1e2eb13ddbc3716705acef48bf6bb9f16f4a79f3115764f
test-results-streaming, sql-kafka-0-10, streaming-kafka-0-10, streaming-kinesis-asl, kubernetes, hadoop-cloud, spark-ganglia-lgpl, protobuf, connect--17-hadoop3-hive2.3
424 KB
sha256:aee66c5006d7fb8c7509b88108ff048355fb10fc18f5b31da8ce171a5267a144
test-results-tpcds--17-hadoop3-hive2.3
4.88 KB
sha256:0b305863d64992f5ffc51003b7cbc355064d641c3e498a66720096630aaa4a15
test-results-yarn--17-hadoop3-hive2.3
42.4 KB
sha256:583133d10dcabfb61bd439e60c4e471c345b7b20228f5f79ba44fc29a14b2311