Kubernetes for PostgreSQL

EPSIO PRIVATE PREVIEW

At this time, Epsio running in "kubernetes mode" is only available in private preview.
To get early access, contact us at support@epsio.io

This guide will walk you through the process of deploying Epsio in an Kubernetes environment.

Requirements:

• Epsio Operator currently only supports Epsio Deployments of version 17.1.3 and above.

1. Prepare your database for Epsio

Open a connection to your database and follow the steps below.

Create a database user for Epsio's exclusive use:

CREATE USER epsio_user WITH PASSWORD '<password>';

Create a schema for Epsio

CREATE SCHEMA IF NOT EXISTS epsio;
GRANT CREATE, USAGE ON SCHEMA epsio TO epsio_user;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA epsio TO epsio_user;

Grant user permissions
Grant the epsio_user read-only access to all tables in your schema by running the following commands:

GRANT SELECT ON ALL TABLES IN SCHEMA public TO epsio_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO epsio_user;
GRANT CREATE, USAGE ON SCHEMA public TO epsio_user;

If you plan to access schemas other than the public schema, you'll need to run these commands for each schema.
Replace public with the name of your schema.

---

2. Configure logical replication

2.1 Check if logical replication is enabled

Run the following command to check if your instance is already configured with logical replication:

AWS RDSAWS AuroraSelf Hosted

postgres> SHOW rds.logical_replication;
 rds.logical_replication 
-------------------------
 off
(1 row)

If the result is on (or 1), it means that logical replication is enabled, skip to set up replication.
If not, follow the steps below to enable logical replication.

postgres> SHOW rds.logical_replication;
 rds.logical_replication 
-------------------------
 off
(1 row)

If the result is on (or 1), it means that logical replication is enabled, skip to set up replication.
If not, follow the steps below to enable logical replication.

postgres=> SHOW wal_level;
 wal_level 
-----------
 replica
(1 row)

If the result is logical, it means that logical replication is enabled, skip to set up replication.
If not, follow the steps below to enable logical replication.

2.2 Enable logical replication

AWS RDSAWS AuroraSelf Hosted

Create a custom RDS parameter group. If your instance already uses a custom parameter group, skip to the next stage.

Edit the custom parameter group. set the rds.logical_replication parameter to 1.

Optional: Set the max_slot_wal_keep_size parameter to 4096 to limit the amount of WAL data that is retained for logical replication slots. (Postgres 13+)

Associate the custom parameter group with your RDS instance. Go to the RDS management console, select your instance and click on the "Modify" button.

In the "Modify DB Instance" page, select the custom parameter group you created in the previous step.

Make sure you choose "Apply Immediately" to apply the changes immediately.

Wait for the parameter group configuration to change to "Pending reboot" status.

The parameter group status can be found in the "Configuration" tab of your RDS instance.

Then, reboot the database for the changes to take effect.

You'll know that the changes have taken affect when the status of your DB instance Parameter Group changes to "In Sync".

Verify that the rds.logical_replication parameter is set to "on" (or 1).

SHOW rds.logical_replication;
 rds.logical_replication
-------------------------
    on
    (1 row)

Create a custom Aurora parameter group. If your instance already uses a custom parameter group, skip to the next stage.

Edit the custom parameter group, set the rds.logical_replication parameter to 1.

Optional: Set the max_slot_wal_keep_size parameter to 4096 to limit the amount of WAL data that is retained for logical replication slots. (Postgres 13+)

Associate the custom parameter group with your Aurora cluster. Go to the RDS management console, select your instance and click on the "Modify" button.

In the "Modify DB Instance" page, select the custom parameter group you created in the previous step.

Make sure you choose "Apply Immediately" to apply the changes immediately.

Wait for the parameter group configuration to change to "Pending reboot" status.

The parameter group status can be found in the "Configuration" tab of your RDS instance.

Then, reboot the database for the changes to take effect.

You'll know that the changes have taken affect when the status of your DB instance Parameter Group changes to "In Sync".

Verify that the rds.logical_replication parameter is set to "on" (or 1).

SHOW rds.logical_replication;
 rds.logical_replication
-------------------------
    on
    (1 row)

To enable logical replication in a PostgreSQL database, you need to set the wal_level parameter in your database configuration to logical. For standard PostgreSQL installations, you can do this by either:

• Method 1: Adding a wal_level = logical line to the postgresql.conf file.
• Method 2: Running ALTER SYSTEM SET wal_level = logical;;

Optional: Set the max_slot_wal_keep_size parameter to 4096 to limit the amount of WAL data that is retained for logical replication slots. (Postgres 13+)

Restart your database for the changes to take effect.

Verify that the wal_level parameter is set to "logical".

SHOW wal_level;
 wal_level
-------------------------
    logical
    (1 row)

2.3 Set up replication

Next, you'll need to set up replication by running the following commands in your database:

AWS RDSAWS AuroraSelf Hosted

CREATE PUBLICATION epsio_publication;
CREATE PUBLICATION epsio_command_publication;

GRANT rds_replication TO epsio_user;

CREATE PUBLICATION epsio_publication;
CREATE PUBLICATION epsio_command_publication;

GRANT rds_replication TO epsio_user;

CREATE PUBLICATION epsio_publication;
CREATE PUBLICATION epsio_command_publication;

ALTER USER epsio_user WITH REPLICATION;

4. Install the Epsio Operator

4.1 Create a Kubernetes namespace for Epsio

To ensure the Epsio operator is separated from other components, we recommend creating a dedicated namespace for Epsio. You can use an existing namespace as long as it does not contain an existing Epsio operator.

kubectl create namespace <epsio-namespace>

4.2 Install the Epsio Operator

YAML bundleHelm Chart

Epsio for Kubernetes bundle is published as a single YAML file. To install the Epsio operator, run the following command:

# https://docs.epsio.io/about/operator_release_notes/
EPSIO_OPERATOR_VERSION=v1.0.2

kubectl apply -n <epsio-namespace> -f https://storage.googleapis.com/epsio/k8s/$EPSIO_OPERATOR_VERSION/epsio-operator-definition.yaml

Epsio for Kubernetes is also available as a Helm chart. To install the Epsio operator using Helm, run the following command:

helm install epsio-operator oci://us-docker.pkg.dev/epsio-io/helm/epsio-operator --namespace <epsio-namespace>

4.3 Verify the Epsio Operator is running

Check the operator deployment to verify it’s running in your namespace:

kubectl get deployment -n <epsio-namespace> epsio-operator-deployment

You should see a result similar to this:

NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
epsio-operator-deployment   1/1     1            1           5m12s

5. Deploy Epsio

An Epsio deployment is created from an Epsio custom resource that contains deployment specifications.

5.1 Create necessary secrets

Epsio requires two secrets to be created in order to connect to your database:

• epsio-account-token - Your account's access token which can be retrieved from the cloud console settings page.
• epsio-user-secret - The password of the epsio_user you created in step 1.

To create the secrets, run the following commands:

kubectl create -n <epsio_namespace> secret generic epsio-account-token --from-literal=token=<your_account_token>
kubectl create -n <epsio_namespace> secret generic epsio-user-secret --from-literal=password=<your-password>

5.2 Deploy Epsio instance

Epsio Custom ResourceHelm Chart

Create a file (my-epsio.yaml) that defines an Epsio:

⁠apiVersion: epsio.io/v1
kind: Epsio
metadata:
name: example-epsio
spec:
version: "<your-desired-version>" # Minimum 20

accountTokenSecret: epsio-account-token

sourceDatabase:
    host: <your-database-host>
    port: 5432
    dbName: <your-database-name>
    passwordSecretName: epsio-user-secret

storage:
    size: 100Gi # Change to your desired storage size
    storageClass: gp2 # Change to your desired storage class

resources:
    memory: "8Gi" # Change to your desired memory size
    cpu: "4" # Change to your desired CPU size

And run the following command to apply the Epsio CRD:

kubectl apply -n <epsio-namespace> -f my-epsio.yaml

To install the Epsio instance via Helm, run the following command:

helm install epsio-instance oci://us-docker.pkg.dev/epsio-io/helm/epsio-instance --values <values.yaml> --namespace <namespace>

Replace <values.yaml> with the path to your values file. To retrieve the default values of an Epsio instance, you can run the following command:

helm inspect values oci://us-docker.pkg.dev/epsio-io/helm/epsio-instance

5.4 Verify the Epsio is running

Check the deployment to verify it’s running in your namespace:

kubectl get statefulset -n <epsio-namespace> example-epsio

You should see a result similar to this:

NAME            READY   AGE
example-epsio   1/1     13m

Congratulations! You've successfully deployed Epsio.

---

You are set to go and can create your first view. Visit the create_view for further details.

Troubleshooting

To troubleshoot, check the Epsio deployment logs:

kubectl -n <epsio-namespace> logs example-epsio-0