Blinded Block proposal fails with Prysm BN

[sugh01]

🐞 Bug Report

Proposals failed with Prysm BN

BN Args
"Args": [ "--accept-terms-of-use=true", "--datadir=/data/prysm/beacon-data/", "--execution-endpoint=http://172.16.0.11:8551", "--rpc-host=0.0.0.0", "--rpc-port=4000", "--grpc-gateway-host=0.0.0.0", "--grpc-gateway-corsdomain=*", "--grpc-gateway-port=3500", "--p2p-host-ip=172.16.0.14", "--p2p-tcp-port=13000", "--p2p-udp-port=12000", "--min-sync-peers=1", "--verbosity=info", "--slots-per-archive-point=32", "--suggested-fee-recipient=0x8943545177806ED17B9F23F0a21ee5948eCaa776", "--subscribe-all-subnets=true", "--jwt-secret=/jwt/jwtsecret", "--enable-debug-rpc-endpoints=true", "--disable-monitoring=false", "--monitoring-host=0.0.0.0", "--monitoring-port=8080", "--p2p-static-id=true", "--chain-config-file=/network-configs/network-configs/config.yaml", "--genesis-state=/network-configs/network-configs/genesis.ssz", "--http-mev-relay=http://mev-boost-1-prysm-geth:18550" ],

VC Flags
exec lighthouse validator \ --beacon-nodes ${LIGHTHOUSE_BEACON_NODE_ADDRESS} \ --suggested-fee-recipient "0x0000000000000000000000000000000000000000" \ --metrics \ --metrics-address "0.0.0.0" \ --metrics-allow-origin "*" \ --metrics-port "5064" \ --use-long-timeouts \ --testnet-dir "/opt/lighthouse/network-configs" \ --builder-proposals --distributed \ --debug-level "trace"

VC Args (Lighthouse)
2024-03-18 20:58:00 15:28:00.673 ERRO vapi Validator api 5xx response: builder-api enabled, but duty proposer requested {"status_code": 500, "message": "Internal server error", "duration": "66.709µs", "vapi_endpoint": "propose_block"} 2024-03-18 20:58:00 core/scheduler/scheduler.go:147 .GetDutyDefinition 2024-03-18 20:58:00 core/validatorapi/validatorapi.go:1067 .getProposerPubkey 2024-03-18 20:58:00 core/validatorapi/validatorapi.go:304 .Proposal 2024-03-18 20:58:00 core/validatorapi/router.go:657 .func8 2024-03-18 20:58:00 core/validatorapi/router.go:319 .func1 2024-03-18 20:58:00 app/app.go:906 .func1

Charon Log Error -

2024-03-18 20:58:00 15:28:00.673 ERRO vapi       Validator api 5xx response: builder-api enabled, but duty proposer requested {"status_code": 500, "message": "Internal server error", "duration": "66.709µs", "vapi_endpoint": "propose_block"}
2024-03-18 20:58:00     core/scheduler/scheduler.go:147 .GetDutyDefinition
2024-03-18 20:58:00     core/validatorapi/validatorapi.go:1067 .getProposerPubkey
2024-03-18 20:58:00     core/validatorapi/validatorapi.go:304 .Proposal
2024-03-18 20:58:00     core/validatorapi/router.go:657 .func8
2024-03-18 20:58:00     core/validatorapi/router.go:319 .func1
2024-03-18 20:58:00     app/app.go:906 .func1

2024-03-18 20:58:01 15:28:01.139 ERRO fetcher Permanent failure calling fetcher/fetch: fetch builder proposer data: beacon api blinded_proposal: nok http response {"label": "blinded_proposal", "status_code": 500, "endpoint": "/eth/v1/validator/blinded_blocks/95?randao_reveal=0xb01f2243ae6f0552c94d8480435a88e4a338918cb3ef5811f9af14f130d921b33b0b0be5b015b6e77497b0030b7089de05ffefe182d3cc19ee2fdbe6d5ffd84b2b4239eb8caaceb0a099b2478f47d61bfb87ec5d0612f34beab25c95b646b156&graffiti=0x636861726f6e2f76302e31392e322d3533653034346300000000000000000000", "method": "GET", "data": "{\"message\":\"Prepared block is not blinded\",\"code\":500}", "duty": "95/builder_proposer"} 2024-03-18 20:58:01 app/eth2wrap/eth2wrap.go:334 .wrapError 2024-03-18 20:58:01 app/eth2wrap/eth2wrap_gen.go:446 .BlindedProposal 2024-03-18 20:58:01 core/fetcher/fetcher.go:304 .fetchBuilderProposerData 2024-03-18 20:58:01 core/fetcher/fetcher.go:65 .Fetch 2024-03-18 20:58:01 core/tracing.go:62 .1 2024-03-18 20:58:01 core/tracking.go:17 .1 2024-03-18 20:58:01 core/retry.go:17 .1 2024-03-18 20:58:01 app/retry/retry.go:118 .DoAsync

VC Error -
2024-03-18 21:04:15 Mar 18 15:34:15.004 INFO Connected to beacon node endpoint: http://node0:3600/, version: obolnetwork/charon/v0.19.2-53e044c/arm64-linux 2024-03-18 21:04:17 Mar 18 15:34:17.005 ERRO Error whilst producing block info: blinded proposal failed, attempting full block, block_slot: Slot(126), error: "Some endpoints failed, num_failed: 1 http://node0:3600/ => RequestFailed(Recoverable(\"Error from beacon node when producing block: HttpClient(url: http://node0:3600/, kind: timeout, detail: operation timed out)\"))", service: block 2024-03-18 21:04:17 Mar 18 15:34:17.007 INFO Requesting unsigned block slot: 126, service: block 2024-03-18 21:04:17 Mar 18 15:34:17.009 INFO Requesting unsigned block slot: 127, service: block 2024-03-18 21:04:17 Mar 18 15:34:17.012 INFO Connected to beacon node endpoint: http://node0:3600/, version: obolnetwork/charon/v0.19.2-53e044c/arm64-linux 2024-03-18 21:04:17 Mar 18 15:34:17.013 INFO Connected to beacon node endpoint: http://node0:3600/, version: obolnetwork/charon/v0.19.2-53e044c/arm64-linux 2024-03-18 21:04:17 Mar 18 15:34:17.013 INFO Connected to beacon node endpoint: http://node0:3600/, version: obolnetwork/charon/v0.19.2-53e044c/arm64-linux 2024-03-18 21:04:17 Mar 18 15:34:17.014 INFO Connected to beacon node endpoint: http://node0:3600/, version: obolnetwork/charon/v0.19.2-53e044c/arm64-linux 2024-03-18 21:04:17 Mar 18 15:34:17.038 CRIT Error whilst producing block info: full block attempted after a blinded failure, block_slot: Slot(126), error: Recoverable("Some endpoints failed, num_failed: 2 http://node0:3600/ => Unavailable(Offline), http://node0:3600/ => RequestFailed(Recoverable(\"Error from beacon node when producing block: ServerMessage(ErrorMessage { code: 500, message: \\\"Internal server error\\\", stacktraces: [] })\"))"), service: block

[gsora]

Do we need a specific flag for Prysm BN to use builder API?

[gsora]

Do we need a specific flag for Prysm BN to use builder API?