|
| 1 | +# Intel Edge Controls Zedada Enablement Benchmarking Suite |
| 2 | + |
| 3 | +A comprehensive benchmarking framework for real-time performance testing on Intel Edge Computing Infrastructure (ECI). This suite provides containerized benchmarks for evaluating system performance under various workloads with Intel Resource Director Technology (RDT) cache allocation and CPU affinity controls. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +This benchmarking suite enables performance testing of real-time applications and systems using Intel's Edge Computing Infrastructure. It includes multiple specialized benchmarks designed to test different aspects of real-time performance: |
| 8 | + |
| 9 | +- **Real-time latency testing** with cyclictest |
| 10 | +- **Computational workload analysis** with Caterpillar |
| 11 | +- **Industrial automation benchmarks** with CODESYS Control runtime |
| 12 | +- **OPC UA communication performance** testing |
| 13 | +- **System stress testing** capabilities |
| 14 | +- **Long-duration stability testing** with mega-benchmark |
| 15 | +- **Network performance and bandwidth testing** with iperf3 |
| 16 | + |
| 17 | +## Quick Start |
| 18 | + |
| 19 | +### Prerequisites |
| 20 | + |
| 21 | +- Debian 12 (Bookwork) |
| 22 | +- Docker installed and configured |
| 23 | +- Privileged container execution capabilities |
| 24 | +- Intel RDT-enabled system (for cache allocation features) |
| 25 | + |
| 26 | +### Setup ECI Repository |
| 27 | + |
| 28 | +1. Open a terminal |
| 29 | +2. Download the ECI APT key to the system keyring: |
| 30 | + |
| 31 | +```bash |
| 32 | +sudo -E wget -O- https://eci.intel.com/repos/gpg-keys/GPG-PUB-KEY-INTEL-ECI.gpg | sudo tee /usr/share/keyrings/eci-archive-keyring.gpg > /dev/null |
| 33 | +``` |
| 34 | +3. Add the signed APT sources and configure the APT client to use the ECI APT repository: |
| 35 | + |
| 36 | +```bash |
| 37 | +echo "deb [signed-by=/usr/share/keyrings/eci-archive-keyring.gpg] https://eci.intel.com/repos/$(source /etc/os-release && echo $VERSION_CODENAME) isar main" | sudo tee /etc/apt/sources.list.d/eci.list |
| 38 | +echo "deb-src [signed-by=/usr/share/keyrings/eci-archive-keyring.gpg] https://eci.intel.com/repos/$(source /etc/os-release && echo $VERSION_CODENAME) isar main" | sudo tee -a /etc/apt/sources.list.d/eci.list |
| 39 | +``` |
| 40 | +4. Configure the ECI APT reposity to have higher priority over other repositories and ping the version of the libflann packages: |
| 41 | + |
| 42 | +```bash |
| 43 | +sudo bash -c 'echo -e "Package: *\nPin: origin eci.intel.com\nPin-Priority: 1000" > /etc/apt/preferences.d/isar' |
| 44 | +sudo bash -c 'echo -e "\nPackage: libflann*\nPin: version 1.19.*\nPin-Priority: -1\n\nPackage: flann*\nPin: version 1.19.*\nPin-Priority: -1" >> /etc/apt/preferences.d/isar' |
| 45 | +``` |
| 46 | +5. Update your apt sources lists |
| 47 | + |
| 48 | +```bash |
| 49 | +sudo apt update |
| 50 | +``` |
| 51 | +6. Install ECI Grub menu. |
| 52 | + |
| 53 | + |
| 54 | +### Setup Real Time Kernel |
| 55 | + |
| 56 | +```bash |
| 57 | +sudo apt install eci-experimental |
| 58 | +``` |
| 59 | +1. Update the firmware packages |
| 60 | +```bash |
| 61 | +sudo apt-get reinstall '(firmware-linux-nonfree|linux-firmware$)' |
| 62 | +``` |
| 63 | +2. Install 6.12.8 Kernel |
| 64 | +```bash |
| 65 | +sudo apt install linux-intel-rt-experimental |
| 66 | +``` |
| 67 | +3. Reboot the target system |
| 68 | +```bash |
| 69 | +sudo reboot |
| 70 | +``` |
| 71 | + |
| 72 | +4. Update Kernel (Change Isolated cpus) |
| 73 | +```bash |
| 74 | +sudo vim /etc/grub.d/09_eci |
| 75 | +``` |
| 76 | +update `isolcpus` to the core you want isolated |
| 77 | + |
| 78 | +5. Update grub and reboot |
| 79 | +```bash |
| 80 | +sudo update-grub |
| 81 | +sudo reboot |
| 82 | +``` |
| 83 | + |
| 84 | + |
| 85 | +### Build All Benchmarks |
| 86 | + |
| 87 | +```bash |
| 88 | +./benchmarking.sh build |
| 89 | +``` |
| 90 | + |
| 91 | +### Mount Resource Control |
| 92 | +1. Mount Resource Control |
| 93 | + |
| 94 | +```bash |
| 95 | +sudo mount -t resctrl resctrl /sys/fs/resctrl |
| 96 | +``` |
| 97 | +### Run a Benchmark |
| 98 | + |
| 99 | +```bash |
| 100 | +# Example: Run caterpillar benchmark with specific cache and core settings |
| 101 | +./benchmarking.sh caterpillar -l 0xffe -t 15 |
| 102 | + |
| 103 | +# Example: Run with background stressor |
| 104 | +./benchmarking.sh caterpillar --l3-cache-mask 0xffe --t-core 15,16,17 --stressor |
| 105 | +``` |
| 106 | + |
| 107 | +## Available Benchmarks |
| 108 | + |
| 109 | +### 1. Caterpillar |
| 110 | +Real-time computational benchmark for measuring system responsiveness under load. |
| 111 | +- **Purpose**: Deterministic computational workload testing |
| 112 | +- **Source**: Intel ECI repository |
| 113 | +- **Documentation**: [https://eci.intel.com/docs/3.3/development/performance/benchmarks.html?highlight=caterpillar#caterpillar](Caterpillar) |
| 114 | + |
| 115 | +### 2. Cyclictest |
| 116 | +Standard Linux real-time testing tool for measuring timer latency. |
| 117 | +- **Purpose**: High-resolution timer latency measurement |
| 118 | +- **Source**: rt-tests package |
| 119 | +- **Documentation**: [https://eci.intel.com/docs/3.3/development/performance/benchmarks.html?highlight=caterpillar#caterpillar](cyclictest) |
| 120 | + |
| 121 | +### 3. CODESYS Jitter Benchmark |
| 122 | +Industrial automation runtime benchmark testing PLC performance and jitter. |
| 123 | +- **Purpose**: Industrial automation runtime validation |
| 124 | +- **Source**: CODESYS Control for Linux SL |
| 125 | +- **Documentation**: [https://eci.intel.com/docs/3.3/components/codesys.html#codesys-ui-benchmark](Codesys-Jitter-Benchmark) |
| 126 | + |
| 127 | +### 4. CODESYS OPC UA Pub/Sub |
| 128 | +Communication benchmark testing OPC UA publisher/subscriber performance. |
| 129 | +- **Purpose**: Industrial communication protocol testing |
| 130 | +- **Source**: CODESYS OPC UA components |
| 131 | +- **Documentation**: [https://eci.intel.com/docs/3.3/components/codesys.html#codesys-opcua-client-benchmark](Codesys-opcua-Benchmark) |
| 132 | + |
| 133 | +### 5. Mega Benchmark |
| 134 | +Long-duration (48-hour) stability and performance testing. |
| 135 | +- **Purpose**: Extended system stability validation |
| 136 | +- **Source**: Intel ECI realtime benchmarking package |
| 137 | +- **Documentation**: [https://eci.intel.com/docs/3.3/development/performance/benchmarks.html#mega-benchmark](mega-benchmark) |
| 138 | + |
| 139 | +### 6. Iperf3 |
| 140 | +Network performance and bandwidth testing. |
| 141 | +- **Purpose**: Networking measurements |
| 142 | +- **Source**: Open source iperf3 |
| 143 | + |
| 144 | +### 7. Stressor |
| 145 | +Background stress generation for testing system behavior under load. |
| 146 | +- **Purpose**: Realistic background load simulation |
| 147 | +- **Source**: Linux kernel compilation stress |
| 148 | + |
| 149 | +## Documentation |
| 150 | + |
| 151 | +### Complete Documentation |
| 152 | +- [**Comprehensive Guide**](docs/README.rst) - Complete technical documentation |
| 153 | +- [**Usage Guide**](docs/usage-guide.rst) - Quick start and common usage patterns |
| 154 | + |
| 155 | +### Individual Benchmark Documentation |
| 156 | +- [Caterpillar](docs/caterpillar.rst) - Real-time computational benchmark |
| 157 | +- [Cyclictest](docs/cyclictest.rst) - Timer latency testing |
| 158 | +- [CODESYS Jitter](docs/codesys-jitter-benchmark.rst) - Industrial automation testing |
| 159 | +- [CODESYS OPC UA](docs/codesys-opcua-pubsub.rst) - Industrial communication testing |
| 160 | +- [Mega Benchmark](docs/mega-benchmark.rst) - Long-duration stability testing |
| 161 | +- [Iperf3](docs/iperf3.rst) - Network and bandwidth testing |
| 162 | + |
| 163 | +## Architecture |
| 164 | + |
| 165 | +The framework is built on a layered architecture: |
| 166 | + |
| 167 | +### Base Infrastructure |
| 168 | +- **ECI Base Image**: Common foundation with Intel tools and repositories |
| 169 | +- **Intel RDT Integration**: Cache allocation and CPU affinity controls |
| 170 | +- **Container Orchestration**: Unified execution through benchmarking.sh script |
| 171 | + |
| 172 | +### Intel RDT Integration |
| 173 | +All benchmarks leverage Intel Resource Director Technology: |
| 174 | +- **Cache Allocation Technology (CAT)**: L3 cache partitioning |
| 175 | +- **CPU Affinity Control**: Dedicated core assignment |
| 176 | +- **Memory Bandwidth Allocation**: QoS enforcement |
| 177 | + |
| 178 | +### Execution Framework |
| 179 | +```bash |
| 180 | +# Unified command structure |
| 181 | +./benchmarking.sh <benchmark> -l <cache_mask> -t <cores> [--stressor] |
| 182 | + |
| 183 | +# Example with RDT controls |
| 184 | +rdtset -t "l3=${L3_CACHE_MASK};cpu=${T_CORE}" -c ${T_CORE} -k <benchmark_command> |
| 185 | +``` |
| 186 | + |
| 187 | +## Key Features |
| 188 | + |
| 189 | +### Resource Control |
| 190 | +- **Intel RDT Integration**: Precise cache and CPU resource allocation |
| 191 | +- **Container Isolation**: Privileged containers with necessary capabilities |
| 192 | +- **NUMA Awareness**: Topology-aware resource assignment |
| 193 | + |
| 194 | +### Performance Monitoring |
| 195 | +- **Real-time Metrics**: Latency, throughput, and jitter measurement |
| 196 | +- **System Monitoring**: CPU, memory, and cache performance tracking |
| 197 | +- **Comprehensive Logging**: Detailed performance data collection |
| 198 | + |
| 199 | +### Industrial Focus |
| 200 | +- **CODESYS Integration**: Complete industrial automation runtime testing |
| 201 | +- **OPC UA Testing**: Industrial communication protocol validation |
| 202 | +- **Real-time Validation**: Deterministic behavior verification |
| 203 | + |
| 204 | +### Stress Testing |
| 205 | +- **Background Load**: Realistic system stress simulation |
| 206 | +- **Resource Contention**: Cache and CPU competition testing |
| 207 | +- **Long-duration Testing**: 48-hour stability validation |
| 208 | + |
| 209 | +## System Requirements |
| 210 | + |
| 211 | +### Hardware |
| 212 | +- Intel processor with RDT support |
| 213 | +- Multi-core CPU architecture |
| 214 | +- Sufficient memory (8GB minimum, 16GB recommended) |
| 215 | +- Network connectivity for web interfaces |
| 216 | + |
| 217 | +### Software |
| 218 | +- Linux kernel with RDT support enabled |
| 219 | +- Docker with privileged container support |
| 220 | +- Intel CMT-CAT tools |
| 221 | +- Real-time kernel configuration (recommended) |
| 222 | + |
| 223 | +### Configuration |
| 224 | +```bash |
| 225 | +# Kernel boot parameters for optimal performance |
| 226 | +isolcpus=<core_list> # Isolate CPU cores |
| 227 | +nohz_full=<core_list> # Disable timer ticks |
| 228 | +rcu_nocbs=<core_list> # Disable RCU callbacks |
| 229 | +``` |
| 230 | + |
| 231 | +## Usage Examples |
| 232 | + |
| 233 | +### Basic Performance Testing |
| 234 | +```bash |
| 235 | +# Build all containers |
| 236 | +./benchmarking.sh build |
| 237 | + |
| 238 | +# Test timer latency |
| 239 | +./benchmarking.sh cyclictest -l 0xffe -t 15 |
| 240 | + |
| 241 | +# Test computational performance |
| 242 | +./benchmarking.sh caterpillar -l 0xffe -t 16 |
| 243 | + |
| 244 | +# Test Network performance |
| 245 | +./benchmarking.sh iperf3 -l 0xffe -t 15 |
| 246 | +``` |
| 247 | + |
| 248 | +### Industrial Automation Testing |
| 249 | +```bash |
| 250 | +# Test PLC runtime performance |
| 251 | +./benchmarking.sh codesys-jitter-benchmark -l 0xffe -t 15 |
| 252 | + |
| 253 | +# Test industrial communication |
| 254 | +./benchmarking.sh codesys-opcua-pubsub -l 0xffe -t 16 |
| 255 | +``` |
| 256 | + |
| 257 | +### Stress Testing |
| 258 | +```bash |
| 259 | +# Test under system load |
| 260 | +./benchmarking.sh cyclictest -l 0xffe -t 15 --stressor |
| 261 | +./benchmarking.sh caterpillar -l 0xffe -t 16 --stressor |
| 262 | +``` |
| 263 | + |
| 264 | +### Multi-Core Testing |
| 265 | +```bash |
| 266 | +# Allocate multiple cores |
| 267 | +./benchmarking.sh cyclictest -l 0xffe -t 15,16,17,18 |
| 268 | +``` |
| 269 | + |
| 270 | +## Development and Extension |
| 271 | + |
| 272 | +### Adding New Benchmarks |
| 273 | +1. Create benchmark directory with Dockerfile |
| 274 | +2. Update auto-discovery in benchmarking.sh |
| 275 | +3. Add execution logic to main script |
| 276 | +4. Create documentation following existing patterns |
| 277 | + |
| 278 | +### Integration Testing |
| 279 | +- RDT resource allocation validation |
| 280 | +- Container privilege verification |
| 281 | +- Performance metric collection testing |
| 282 | + |
| 283 | +## References |
| 284 | + |
| 285 | +- **Intel ECI Documentation**: https://eci.intel.com |
| 286 | +- **Intel RDT Technology Guide**: Resource Director Technology overview |
| 287 | +- **CODESYS Documentation**: Industrial automation platform |
| 288 | +- **Linux Real-time Documentation**: PREEMPT_RT and real-time best practices |
| 289 | + |
| 290 | +## License |
| 291 | + |
| 292 | +Intel Confidential - See individual source files for specific license terms. |
| 293 | + |
| 294 | +## Support |
| 295 | + |
| 296 | +For support and questions: |
| 297 | +- Review documentation in `docs/` directory |
| 298 | +- Check troubleshooting sections in individual benchmark docs |
| 299 | +- Consult Intel ECI resources at eci.intel.com |
0 commit comments