Features

• Source API
  • Euphoria allows users to define their own input sources and output sinks.
• Stateless elementwise processing
  • MapElements, FlatMap, Filter
  • Operators allowing user code to process one element at a time to produce new, transformed datasets. There operators are stateless and typically executed distributed and in parallel over the process input dataset.
• Stateful processing
  • ReduceStateByKey, ReduceByKey, ...PerKey, Sort
  • Operators allowing aggregation of values (into a state) per key. These are normally used in conjuction with a windowing strategy (see below.)
  • Windowing
    • Types
      • Global windows
      • Fixed time windows
      • Sliding time windows
      • Session windows
      • Count windows
      • Custom windows
      • Custom merging windows
      • Timestamp control
    • Triggers
      • Event-time triggers
      • Custom triggers
      • Allowed lateness
      • Timers
• Metrics/Accumulators
  • Euphoria supports executor-independent access to pre-defined metrics, e.g. counters, histograms, timers
  • Euphoria does not support user-defined, custom metrics

Executors

The above mentioned features are supported in the following executor modes:

• Spark batch only
• Flink batch and streaming