Architecture¶
Epsio seamlessly integrates into your existing database, ensuring a short and smooth integration process. It requires zero database migrations, keeps your database secure, and allows you to query your data as if it were stored and processed in a single database.
Today, Epsio offers two deployment architectures:
Deployment Architectures¶
Write-back (recommended)¶
In the writeback deployment method, an Epsio instance is deployed within your environment on a separate compute instance from the database. For each query you define, Epsio fetches a snapshot of the initial data from the relevant underlying tables in your database and writes back the calculated query result to your original database.
Then, whenever the underlying data changes, Epsio uses logical replication/CDC (change data capture) to receive the new changes and incrementally update the previously written query result.
This acrchitecture has the following benefits:
- Zero database migrations - When deploying Epsio, you get the upside of incremental materialized views without the need to migrate your data to a new database.
- Low Latency - Epsio writes the query result directly to a table in your database. This means you can query the result table from your original database with minimal latency, just like any other table. It's even possible to JOIN it with other data in your database.
- High Control of Result Data - With Epsio writing results to standard SQL tables in your database, you have the flexibility to add indexes, partitions, and more to the results table as you would with any other table in your database.
- High Data Availability - Since Epsio writes the query result back to your database and is only responsible for updating previous results, you can still query the data even if Epsio is down (although the results might not be up to date).
Foreign Data Wrappers (FDW) / Federated Tables¶
In the FDW deployment method, an Epsio instance is deployed within your environment on a separate compute instance from the database. For each query you define, Epsio retrieves a snapshot of the initial data from the relevant underlying tables in your database. It then creates a foreign table (FDW in PostgreSQL, or another type in different databases) in your database. This foreign table refers to a data store running on the Epsio instance, which contains the result of the query.
Then, whenever the underlying data changes, Epsio utilizes logical replication/CDC (change data capture) to receive the new changes and incrementally update the previously generated query result in its data store.
This acrchitecture has the following benefits:
- Zero database migrations - When deploying Epsio, you get the benefit of incremental materialized views without needing to migrate your data to a new database.
- Zero Effect on your database - As Epsio writes/updates the results in its own data store, it does not write/update anything in your database. Therefore, it does not add any backpressure to your database.
- Not a Proxy - Since Epsio operates "behind" your database, it does not interrupt or affect any queries that are not configured to be forwarded to Epsio.
Data Privacy and Security¶
At Epsio, we take data privacy very seriously. We understand that keeping sensitive data secure is crucial to building trust with our users. That's why we developed Epsio with an architecture that ensures Epsio Cloud does not have access to your sensitive data.
Control Plan and Data Plane¶
The architecture of an Epsio deployment is split into two separate planes. The control plane is the plane where Epsio manages the size of the deployment, gets usage statistics, and runs software updates. The data plane handles the actual data processing of database data.
How is the control plane separated from the data plane?¶
Epsio Cloud communicates with Epsio Deployments to manipulate resources, obtain usage statistics, and manage software upgrades. However, all data processing is done entirely on the compute instances deployed in the customer's environment, and Epsio Cloud cannot access the processed data.
The separation of management and data processing ensures that Epsio provides the benefits of a managed service while maintaining the privacy and security of customer data.