Configuration

Euphoria Flink Executor

• euphoria.flink.streaming.windowing.only.after.shuffle: Normally RBK and RSBK operator's input elements are windowed and transformed before shuffle to reduce the amount of data being transferred. However, under certain circumstances, it can be beneficial to do this only after shuffling the original input elements. See #47 for more information. Note that the time-sliding windowing mentioned in that ticket received a very efficient serialization implementation.
• euphoria.flink.streaming.descriptors.cache.max.size: The euphoria flink streaming executor uses the underlying storage provider through Flink itself and therefore must translate the euphoria storage descriptors into Flink native descriptors. The result of this translation is cached since creating Flink native storage descriptors is actually expensive. The number of storage descriptors in a program is typically very small since these are meant to be statically defined. This configuration option defines a threshold to make euphoria flows fail if the mentioned cache grows beyond the defined value preventing accidental out-of-memory upon dynamic allocation of descriptors.
• euphoria.flink.streaming.allow.early.emitting: States are typically accumulating values until flushed, at which point they are allowed to produce output. This configuration option enables experimental support for receiving the output "channel" already at the creation of a state and thus allowing it to produce output early - before the flush event. This is known to cause troubles with 1) merging windowing, e.g. session windowing, and 2) with the time-sliding windowing strategy. Currently, only the Join operator supports this experimental feature on the flink streaming executor.
• euphoria.flink.batch.list-storage.max-memory-elements and euphoria.spark.batch.list-storage.max-memory-elements: Defines the number of items kept in memory by the list storage before starting to spill to disk. Supported only for the flink batch and spark (batch) executors respectively.