Support to retry the host allocation for hybrid converters

[firestarman]

Contributes to #8874

This PR adds the retry protection to host memory allocations used by the C2C converters in Hybrid scans. This is done by introducing a new class named HybridHostRetryAllocator who implements the Hybrid host allocator interface with retry support. It will close the allocated buffers just before each retry starts, to leave more memory for higher priority tasks.

This change also introduces another new trait named HostRetryAllocator to extract the retry code from the hybrid things to avoid loading hybrid jar when running the new unit tests.

This is to try to resolve the following kind of OOMs:

com.nvidia.spark.rapids.jni.CpuRetryOOM: Could not complete allocation after 1000 retries
        at com.nvidia.spark.rapids.HostAlloc.alloc(HostAlloc.scala:272)
        at com.nvidia.spark.rapids.HostAlloc.allocate(HostAlloc.scala:278)
        at ai.rapids.cudf.HostMemoryBuffer.allocate(HostMemoryBuffer.java:138)
        at com.nvidia.spark.rapids.velox.VeloxBatchConverter.createVectorBuilder(VeloxBatchConverter.scala:337)
        at com.nvidia.spark.rapids.velox.VeloxBatchConverter.$anonfun$resetTargetBuffers$3(VeloxBatchConverter.scala:236)
        at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
        at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
        at com.nvidia.spark.rapids.velox.VeloxBatchConverter.resetTargetBuffers(VeloxBatchConverter.scala:232)
        at com.nvidia.spark.rapids.velox.VeloxBatchConverter.<init>(VeloxBatchConverter.scala:183)
        at com.nvidia.spark.rapids.velox.VeloxBatchConverter$.apply(VeloxBatchConverter.scala:384)

[firestarman]

build

[firestarman]

build

[res-life]

Please use the version of hybrid itself like other places:

        <dependency>
            <groupId>com.nvidia</groupId>
            <artifactId>rapids-4-spark-hybrid_${scala.binary.version}</artifactId>
            <version>${spark-rapids-hybrid.version}</version>
            <scope>provided</scope>
        </dependency>

Like private jar, it has its own version.
During the release process, the project.version is not always equal to spark-rapids-hybrid.version.

And set scope as <scope>test</scope>?

[firestarman]

nice catch

[firestarman]

updated. I created an item for hyrid jar in the dependencyManagement section in the root pom to unify its version for sub modules.

[firestarman]

build

[res-life]

LGTM

[firestarman]

build

[sperlingxx]

LGTM