Stream changes from your Neon database to anywhere

Neon's Logical Replication features makes it possible to detect every change in your database. It can be used to power read-replicas and backups, but can also be used to add streaming characteristics to Neon.

Sequin uses logical replication to send records and changes in your database to your applications and services in real-time. It's designed to never miss an insert, update, or delete and provide exactly-once processing of all changes.

Changes are sent as messages via HTTP push (webhooks) or pull (SQS-like, with Sequin SDKs). Out of the box, you can start triggering side-effects when a new record is created, fan out work to cloud functions, or activate workflows in services like trigger.dev.

In this guide, we'll show you how to connect your database to Sequin to start sending changes anywhere you need.

• A Sequin account
• A Neon account
• Read the important notices about logical replication in Neon before you begin

Sequin uses the Write Ahead Log (WAL) to capture changes from your Postgres database. In this step, we'll enable logical replication for your Lakebase Postgres project.

To enable logical replication:

1. Select your project in the Neon Console.
2. On the Dashboard, select Settings.
3. Select Postgres, then Logical replication.
4. Click Enable to enable logical replication.

You can verify that logical replication is enabled by running the following query from the Neon SQL Editor:

SHOW wal_level;
wal_level
-----------
logical

After enabling logical replication on Neon, you'll now connect your Neon database to Sequin. Follow these steps:

1. In Neon, copy your database connection string. You can find the it by clicking the Connect button in the Console nav. It will look similar to this:

   postgresql://neondb_owner:AbC123dEf@ep-cool-darkness-123456.us-east-2.aws.neon.tech/neondb?sslmode=require&channel_binding=require

2. In the Sequin Console, click on the Connect Database button, and then auto-complete your database credentials by clicking the Autofill with URL button and pasting in your database connection string.

3. Use the SQL Editor in your Neon project to create a replication slot by executing the following SQL query:

   SELECT pg_create_logical_replication_slot('sequin_slot', 'pgoutput');

   This creates a replication slot named sequin_slot.

4. Create a publication to indicate which tables will publish changes to the replication slot. Run the following SQL command:

   CREATE PUBLICATION sequin_pub FOR TABLE table1, table2, table3;

5. Back in the Sequin Console, enter the name of the replication slot (sequin_slot) and publication (sequin_pub) you just created. Then, name your database (for example neondb) and click Create Database.

With these steps completed, your Neon database is now connected to Sequin via a replication slot and publication. Sequin is now detecting changes to your tables.

Set up a consumer in Sequin to stream changes from your database.

1. In the Sequin Console, navigate to the Consumers page and click Create Consumer.

2. Select the Neon database you just created and then select the specific table you want to process changes for.

3. Define any filters for the changes you want to capture. For example, you might want to only process orders with a value greater than a certain amount, or accounts with a certain status.

4. Choose whether you want your consumer to process rows or changes:

   • Rows: Captures the latest state of records when a row is inserted or updated.
   • Changes: Captures every insert, update, and delete, including OLD values for updates and deletes.

5. Select your preferred method for receiving changes:

   • HTTP Push (Webhooks): Sequin sends changes to your specified endpoint.
   • HTTP Pull (similar to SQS): Your application pulls changes from Sequin.

6. Enter the final details for your consumer:

   • Give your consumer a name (for example, neon-changes-consumer).
   • If using HTTP Push, provide the endpoint URL where Sequin should send the changes. You can also provide encrypted headers.
   • Optionally, set a timeout and add an endpoint path.

7. Click Create Consumer to finalize the setup.

Your consumer is now created and will start processing changes from your Neon database according to your specified configuration.

You're now using Sequin with Neon to capture and stream changes from your database. From here, you can tailor your implementation for your use case:

• Use Sequin to trigger workflows in tools like Inngest or trigger.dev, activate side-effects in your app, setup audit logs, or generate denormalized views.
• Tailor your consumer's filtering and settings to meet your requirements.
• Try a pull consumer with our SDKs to completely manage how you retrieve changes at scale.