@@ -131,10 +131,9 @@ on LUMI:
131
131
```
132
132
. /projappl/project_462000752/intel/oneapi/setvars.sh --include-intel-llvm
133
133
134
- module load LUMI/22.08
134
+ module load LUMI
135
135
module load partition/G
136
- module load rocm/5.3.3
137
- module load cce/16.0.1
136
+ module load rocm/6.0.3
138
137
export MPICH_GPU_SUPPORT_ENABLED=1 # Needed for GPU aware MPI
139
138
```
140
139
After this one can load other modules that might be needed for compiling the codes. With the environment set-up we can compile and run the SYCL codes.
@@ -162,13 +161,12 @@ module load openmpi/4.1.2-cuda # This is neeeded for using CUDA aware MPI
162
161
```
163
162
on LUMI:
164
163
```
165
- module load LUMI/22.08
164
+ module load LUMI
166
165
module load partition/G
167
- module load rocm/5.3.3
168
- module load cce/16.0.1
166
+ module load rocm/6.0.3
169
167
export MPICH_GPU_SUPPORT_ENABLED=1
170
- export LD_LIBRARY_PATH=/appl/lumi/SW/LUMI-22.08/G/EB/Boost/1.79.0-cpeCray-22.08/lib:$LD_LIBRARY_PATH
171
- export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so
168
+ # export LD_LIBRARY_PATH=/appl/lumi/SW/LUMI-22.08/G/EB/Boost/1.79.0-cpeCray-22.08/lib:$LD_LIBRARY_PATH ???
169
+ # export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so ??????
172
170
```
173
171
174
172
```
@@ -205,10 +203,9 @@ Similarly on LUMI. First we set up the envinronment and load the modules as indi
205
203
```
206
204
. /projappl/project_462000752/intel/oneapi/setvars.sh --include-intel-llvm
207
205
208
- module load LUMI/22.08
206
+ module load LUMI
209
207
module load partition/G
210
- module load rocm/5.3.3
211
- module load cce/16.0.1
208
+ module load rocm/6.0.3
212
209
export MPICH_GPU_SUPPORT_ENABLED=1
213
210
```
214
211
Now compile with intel compilers:
@@ -218,7 +215,7 @@ icpx -fsycl -fsycl-targets=amdgcn-amd-amdhsa,spir64_x86_64 -Xsycl-target-backend
218
215
```
219
216
Or with AdaptiveCpp:
220
217
```
221
- export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so
218
+ # export LD_PRELOAD=/pfs/lustrep4/appl/lumi/SW/LUMI-22.08/G/EB/rocm/5.3.3/llvm/lib/libomp.so
222
219
/projappl/project_462000752/AdaptiveCpp/bin/acpp -O3 `CC --cray-print-opts=cflags` <sycl_mpi_code>.cpp `CC --cray-print-opts=libs`
223
220
```
224
221
@@ -233,7 +230,7 @@ The `job.sh` file contains all the necessary information (number of nodes, tasks
233
230
234
231
Use [ ` SYCL_PI_TRACE ` ] ( https://intel.github.io/llvm-docs/EnvironmentVariables.html#sycl-pi-trace-options ) to enable runtime tracing (e.g. device discovery):
235
232
236
- export SYCL_PI_TRACE =1
233
+ export SYCL_UR_TRACE =1
237
234
238
235
239
236
### Running on Mahti
@@ -272,7 +269,7 @@ single GPU with single MPI task and a single thread use:
272
269
#SBATCH --job-name=example
273
270
#SBATCH --account=project_2012125
274
271
#SBATCH --partition=gpusmall
275
- #SBATCH --reservation=hlgp-gpu-f2024-thu
272
+ #SBATCH --reservation=hlgp-gpu-f2024-thu ?????????
276
273
#SBATCH --nodes=1
277
274
#SBATCH --ntasks=1
278
275
#SBATCH --time=00:05:00
@@ -297,7 +294,7 @@ LUMI is similar to Mahti.
297
294
#SBATCH --job-name=example
298
295
#SBATCH --account=project_462000752
299
296
#SBATCH --partition=standard
300
- ##SBATCH --reservation=hlgp-cpu-f2024 # The reservation does not work
297
+ ##SBATCH --reservation=hlgp-cpu-f2024 ?????? # The reservation does not work
301
298
#SBATCH --time=00:05:00
302
299
#SBATCH --nodes=1
303
300
#SBATCH --ntasks-per-node=1
@@ -314,7 +311,7 @@ srun my_cpu_exe
314
311
#SBATCH --job-name=example
315
312
#SBATCH --account=project_462000752
316
313
#SBATCH --partition=standard-g
317
- #SBATCH --reservation=hlgp-gpu-f2024
314
+ #SBATCH --reservation=hlgp-gpu-f2024 ??????
318
315
#SBATCH --time=00:05:00
319
316
#SBATCH --nodes=1
320
317
#SBATCH --ntasks-per-node=1
0 commit comments