F1 Lightning HTAP as a Service
1 minute read ∼ Filed in : A paper noteIntroduction
Background & Motivation
Most existing HTAP systems assume a ‘greenfield’ (Single System for OLTP and OLAP).
For the situation where data is stored in a separate system and cannot easily do a wholesale migration, HTAP needs to perform over many storage engines.
Gap
Greenfield systems cannot handle the situation where the data is stored on multiple data stores. In this situation, a federated query engine is needed.
Challenge
It’s challenging to design a federated HTAP engine to support both new and legacy applications that demand transparent, fast queries and transactions.
Goal
The paper tries to propose an HTAP solution that can be enabled across the different options for transactional storage to avoid costly migrations and to permit flexibility in the design of transactional storage systems.
And we want to benefit from the separation of concerns by allowing,
- TP systems to focus on transaction processing.
- AP engines focus on query processing, with an emphasis on analytical queries.
Technique details