This project uses the ibverbs library to create a core InfiniBand subsystem in C++ as well as the JNI library Ibdxnet which is used as a transport by DXNet an event-driven messaging network subsystem.
You need hardware that supports the ibverbs library. How to setup your hardware is out of the scope of this document. However, I took a few notes for myself that might help you as well here.
This project supports Linux, only, and uses the ibverbs library to access InfiniBand hardware. CMake scripts are used to generate build scripts (e.g. makefile). Just run the build.sh script which will execute the steps required to compile everything.
The output files are located in build/bin and build/lib, e.g. build/lib/libMsgrcJNIBinding.so.
To use Ibdxnet with DXNet (or DXRAM), follow the instructions in the respective repositories.
This is a standalone application to test and benchmark the messaging engine which uses reliable verbs. Execute it with the parameter --help to get a list of parameters with descriptions.
A simple example to run a full bi-directional test with two nodes (node65 and node66): On node65:
./MsgrcLoopback -n 0 -c node65,node66 -d 1 -u 1000
On node66:
./MsgrcLoopback -n 1 -c node65,node66 -d 0 -u 1000
When running benchmarks with Ibdxnet, ensure you compile with statistics removed (IBNET_DISABLE_STATISTICS) to get optimal performance.
