Polling interval for Kafka Connect SourceTask
I am using a custom source connector using the Kafka-Connect API which can be used to poll the REST API and stream JSON response to the Kafka theme. Now I'm wondering how to implement the polling interval for the SourceTask as the JDBC Connector suggests it. Somewhere I have to set a thread to sleep, but where should I do this?
source to share
I decided to use this case in my implementation SourceTask
by adding a private type field long
to preserve the timestamp. On the first call, the poll()
field has not yet been initialized, so the configured REST APIs get polled. In this first call, the specified field is long
initialized with the current timestamp. In all subsequent poll()
invocations, this timestamp of the previous call will be checked. If the number of milliseconds elapsed since the previous one is poll()
less than the specified interval between two polls, then I send the thread to sleep since the configured milliseconds have elapsed.
source to share