Our Levyx-Spark Connector un-intrusively makes Apache Spark implementations more efficient and enterprise-grade:

Connects Xenon to Spark

  • Spark RDD/DataFrame maps to Xenon dataset
  • Pushdown of SQL queries to Xenon layer
    • Spark context extended with 3 simple API (i.e., fromXe, toXe, runXe)
  • JIT “C” level compilation/execution

Combined solution provides superior performance vs Native Apache Spark especially in situations involving:

  • Large datasets dealing with sorting, joins, group-by (heavy shuffling)
  • Ideal for workloads involving small Random inserts, point queries
  • Leveraging Index lookups vs  filtering/full-table scan

Apache Spark-Levyx Integration

  • Faster:
    • Higher Performance on IO intensive and/or larger data sets and workloads
    • Persistent RDD/DataFrame option (reduce disk operations-spillovers, ser-des)
  • Cheaper:
    • Fewer nodes (Freed up CPU Utilization, 10:1 cost advantage of Flash vs RAM)
    • Storage-Class Memory architecture
  • Simpler:
    • Persistence Option for RDD/DataFrames – simplify and enable concurrent multi-user/multi-job operations
    • Simpler networks/less shuffling, reduce Spark context switching/scaling/GC/crash issues

String Sort

Iterative Graph