Analytics platform crashes under a heavy load and millions of items from incoming stream are lost while it is restarting

Representative Use Case: Social Event Media Targeting

Phase 1: Architecture Highlights

  • Data (potentially millions of data points per second) is collected in Helium (Persisted and Highly Available)
  • Data is transferred to the Brand-Name Analytics Platform for real time analytics via call back functions or triggers
  • The Brand-Name Analytics Platform makes analyzed data available to business managers who need to visualize the data or create reports

Phase 1: Levyx Provides a Persistent Cache

Phase 2: More optimization with Xenon/Helium

  • Data (potentially millions of data points per second) is collected in Helium
  • Data is transferred to the Brand-Name Analytics Platform for real time analytics via call back functions or triggers
  • HANA makes analyzed data available to business mangers who need to visualize the data or create reports
  • Xenon cluster makes it possible to do online analytics for millions of users

Phase 2: Using Levyx to Provide Persistent Cache and Customer Query Offload

  • Even when the Brand-Name Analytics Platform crashes, the ingest is done in highly available Helium/Xenon Layer
  • Even though Helium/Xenon layer keeps the data in SSD (adding density and persistence), it writes the data to the Brand-Name Analytics Platform at the rate that it is received. It is as fast as if written directly to Brand-Name Analytics Platform
  • Xenon, with the easy to use Apache Spark front-end can offload most of the queries from the Brand-Name Analytics Platform; it can handle millions of small queries from the field (mobile/web users, for example) and free up the Brand-Name Analytics Platform to address more complex, ad-hoc queries from the business side