How Can I Connect to Kafka Bootstrap Server Over SASL_SSL With rdkafka?

I am trying to connect to a message broker using the rdkafka Github, attempting to connect to a Confluent Boostrap Server. However my attempt to get messages fails with authentication failure.

I do not have access to the remote broker server, messages are provided as an "open data" service to the general public.

Kafka bootstrap server: ###.europe-west2.gcp.confluent.cloud:9092
SASL mechanism: PLAIN
SASL config: org.apache.kafka.common.security.plain.PlainLoginModule
Security protocol: SASL_SSL

My PHP consumer file uses the below configuration using a username and password given to me by the data provider

$conf = new RdKafka\Conf();
$conf->set('group.id', 'myGroup');
$conf->set('bootstrap.servers', '##.europe-west2.gcp.confluent.cloud:9092');
$conf->set('sasl.username', '####');
$conf->set('sasl.mechanisms', 'PLAIN');
$conf->set('sasl.password', '####');
$conf->set('security.protocol', 'SASL_SSL');

The error response given by the rdkafka program:

%3|1729596043.057|FAIL|rdkafka#consumer-1| [thrd:sasl_ssl://###.europe-west2.gcp.confluent.cloud:9092/boot]: sasl_ssl://pkc-l6wr6.europe-west2.gcp.confluent.cloud:9092/bootstrap: SASL authentication error: Authentication failed (after 5002ms in state AUTH_REQ)

%3|1729596043.058|ERROR|rdkafka#consumer-1| [thrd:sasl_ssl://###.europe-west2.gcp.confluent.cloud:9092/boot]: 1/1 brokers are down

%3|1729596043.058|ERROR|rdkafka#consumer-1| [thrd:app]: rdkafka#consumer-1: sasl_ssl://###.europe-west2.gcp.confluent.cloud:9092/bootstrap: SASL authentication error: Authentication failed (after 5002ms in state AUTH_REQ)

The issue appears similar to this Confluent Forum question, I have run the telnet command on the port and it successfully connects.

Does anyone know what I could be missing from my configuration? I have considered adding sasl_jaas_config property but JAAS isn’t supported by rdkafka. Needless to say, this Github issue seems to suggest I don’t need it, just specifying a sasl_username and password should suffice…

Thanks. New to Kafka.

https://github.com/confluentinc/librdkafka/issues/2510

You need to sign in to view this answers