Remove events from JDBC Kafka Connect Source
I'm playing around with the Kafka Connect JDBC connector and looking specifically at what actual data format fits into the theme.
I was able to see new inserts and updates in the database, but I was unable to locate the deleted data from the database.
First: Does the JDBC source support these changes? I can't find any documentation anyway.
If so, what format does the actual theme take?
source to share
The conflicting source JDBC connector is capable of capturing "soft deletes" where "deleted" rows are simply marked as such by your application, but are not actually removed from the table. Since the lines still exist, the connector can see their changes. However, the connector is unable to grab rows deleted from a table because the connector queries the original tables via JDBC and therefore cannot see rows deleted from the tables.
Other connectors can record all changes by inserting the database transaction logs or writing the logs forward using techniques known as change data capture, or CDC. Each DBMS is different and therefore requires a connector written specifically for that DBMS. For example, the Debezium project has Kafka Connect connectors for MySQL, PostgreSQL, and MongoDB, and is working on connectors for Oracle and SQL Server.
source to share