chore(meta): follow-ups for the embedded meta in-process gRPC channel

[drmingdrmer]

Follow-ups from reviewing #20391, which replaced the embedded meta service's loopback TCP gRPC listener with an in-process duplex channel. None of these block that PR.

1. The in-process server drops the HTTP/2 pending-reset-stream limit

databend-meta's GrpcServer::do_start() builds the server like this:

// Configure HTTP/2 settings to handle stream reset accumulation.
// The default limit (20) can be too low under high concurrency.
// Setting to None disables the limit entirely.
let builder = Server::builder().http2_max_pending_accept_reset_streams(Some(4096));

The embedded meta service used to start through that method. After #20391 it builds its own server in LocalMetaService::start_in_process_grpc_server (src/meta/store/src/local.rs) and does not carry the setting over:

let result = Server::builder()
    .add_service(grpc_service)
    .serve_with_incoming_shutdown(incoming, async move {
        let _ = shutdown_rx.await;
    })
    .await;

So the embedded server falls back to hyper's default of 20. Once a client resets more than 20 streams before the server accepts them, hyper tears the connection down. The meta client does reconnect — every tonic connection gets a fresh duplex stream — but the RPCs in flight on that connection fail first. Fix:

let result = Server::builder()
    .http2_max_pending_accept_reset_streams(Some(4096))
    .add_service(grpc_service)
    ...

2. test_fixed_dir_survives_restart waits with a fixed sleep

src/meta/store/src/local.rs:

drop(first);

// Dropping the service signals its background server and Meta worker.
tokio::time::sleep(Duration::from_secs(1)).await;

let second =
    LocalMetaService::new_with_fixed_dir::<TokioRuntime>(dir, "persistent-test").await?;

LocalMetaService::drop only signals the gRPC task and the meta worker; it does not wait for either to release the raft dir. One second is plenty on an idle machine — the whole test takes 1.11 s locally — but it is a guess on a loaded CI runner. Retry new_with_fixed_dir in a loop with a deadline instead of sleeping once.

3. next_port() survives only to produce config_id

Both LocalMetaService::new_testing and LocalMetaService::new_with_fixed_dir still call databend_base::testutil::next_port(), which binds 127.0.0.1:0, reads the assigned port and drops the listener.

After #20391 neither listener uses that port: raft_api_port is hard-coded to 0, and gRPC no longer listens at all. The value survives only as config.raft_config.config_id (a string), plus a directory-name suffix in new_testing that already sits inside a unique tempfile::tempdir().

databend-meta's own harness uses GlobalUniq::unique() for config_id (crates/common/test-harness/src/service.rs:139). Doing the same removes the last bind-and-release from embedded startup, which is what #20391 set out to eliminate.

4. The persistent embedded dir layout changed silently

Before #20391: {embedded_dir}/{name}-{next_port()}/raft_dir. After: {embedded_dir}/raft_dir.

next_port() returns a different ephemeral port on every process start, so the old path meant a restarted databend-local or bendpy never reopened its previous store. #20391 is what makes embedded_dir actually persist, and test_fixed_dir_survives_restart is what pins that behaviour — but the PR description does not mention it, and nothing cleans up the {name}-{port} directories earlier runs left behind. Worth recording, and worth a decision on whether the leftovers need removing.

5. Suite-wide assertions live inside ApiBuilder::build()

src/meta/store/tests/it/kv_api.rs:

async fn build(&self) -> MetaStore {
    let store = MetaStore::new_local_testing::<TokioRuntime>().await;
    ...
    assert_eq!(local.config.raft_config.raft_api_port, 0);
    assert!(local.config.grpc.api_address().is_none());
    assert_eq!(store.get_local_addr().await.unwrap(), "127.0.0.1:0");

    store
}

kvapi_test_suite::TestSuite::test_single_node calls build() roughly thirty times, once per sub-test. These three checks — and the extra get_client_info RPC that get_local_addr issues — therefore run on every sub-test, and a failure is reported against whichever sub-test happened to be running. Move them into one dedicated #[test].

Non-goals

Redesigning the in-process channel, adding reflection or TLS support to it, and changing the meta client's endpoint auto-sync behaviour.