14_Tutorial 4 ‐ Kafka Import and Preprocessing

This tutorial serves to test the KafkaImportApplication and the KafkaProcessingApplication. The former imports process data from Kafka to Spark while the later preprocesses these data.

1. Requirements

If you have not done this so far, clone the bpmnai-core repository and import the project into your preferred IDE.

2. Import Kafka data

In the next step we can user the KafkaImporterApplication to retrieve data out of Kafka and store them locally. Therefore we use the following run configuration.

`Program arguments: -kb "127.0.0.1:9092" -fd "<path_to_target_folder_for_import_data>" -bm "true"

VM arguments: -Dspark.master="local[*]"

3. Process Kafka data

As a last step the data can be prepocessed using the KafkaProcessingApplication. Therefore we use the following run configuration:

Program arguments: -fs "<path_to_target_folder_for_import_data>" -fd "<path_to_target_folder_for_result_data>" -d ";"

VM arguments: -Dspark.master="local[*]"