Kafka Consumer stop conuming with Kerberos authentication
I found a problem that kafkaconsumer lost connection to kafka server after running well for some time. There are no errors in streams log and in kafka log. after restart the job, the connection will be built again. is there any mechanism to know when the connection is lost and to reconnect to kafa? environment: kafka consumer modified with kerberos authentication enabled, original toolkit is 3.0.0. streams version 4.0.1 os powerlinux 6.6. kafka version 0.8.2 with streamtool setproperty -i instanceTrace.defaultlevel=info please see the attached log when the consumer stop consume. link textthere is no errors in the kafka logs.
The PE log has INFO level messages but only covers a three minute period and do not show any activity other than the TCP Operator heartbeat thread. I see no connect or disconnect events. Please continue to run with INFO set and after the next disconnect, upload the logs along with the date and time the disconnect occurred so we can focus on this timeframe for information.
There is no auto reconnect in the toolkit operator but in our tests we crashed a server and when it comes back online we automatically reconnect (the Kafka API handles this automatically). So while we don't have a Streams reconnection policy, Kafka's is used. Could there be an authentication issue preventing Kafka from reconnecting? Are there any security, network or system events shown in syslogs at the time of the disconnects?