Topic configuration

Nearly each Kafka topic can be configured in its behavior with some limitations.

Configuration possibilities

The behavior of the Kafka topic can be configured with the following options:

Number of partitions

The number of partitions to use. The more partitions are available, the more clients can consume the messages in parallel because a partition always can have only 1 client from these parallel running clients.
The maximum number of partitions in the NEXT environment is set to 10.

Retention time

Message storage can be limited by time until they are deleted. If this information is set, the messages are deleted after the amount in milliseconds.
The maximum time duration is 2678000000 ms which corresponds to 7 days.
This deletion by time can also be disabled.

Retention size

Message storage can be limited by the size of the complete partition. If this information is set, the oldest messages are deleted when the size limit is reached Please be aware, that this information is per partition and not per topic. This deletion by size can be disabled.

Cleanup policy

Policy for what should happen with entries that fall out of retention. The standard behavior is to delete the entry. Alternatively the last entry per key can be retained with the compact policy.

Topic namings and accessibility

The internal topics are all created with the following naming concept:

<company>.<project>.int.<topic_name>

where
company and project are defined with your project settings and topic_name can be configured. The topic name can contain [A-Za-z0-9.-_].

The topic is accessible by the technical user created for the project. More information can be found in connect to Kafka.