kafka_neu-1778570057504.zip-extract/kafka-4.2.0-src/docs/streams/developer-guide/manage-topics.md

Path
kafka_neu-1778570057504.zip-extract/kafka-4.2.0-src/docs/streams/developer-guide/manage-topics.md
Status
scanned
Type
file
Name
manage-topics.md
Extension
.md
Programming language

      
    
Mime type
text/plain
File type
exported SGML document, ASCII text, with very long lines
Tag

      
    
Rootfs path

      
    
Size
4288 (4.2 KB)
MD5
1980749997bf60e38622174d2c59c082
SHA1
eceae9adf12ca4d4551370ddfecf6eaa1af397d7
SHA256
729959f3218258ccfe3f9a14d175fe20bce27f8547dfd3cb1e2826cee498f121
SHA512

      
    
SHA1_git
aff7d8151cdb003afac23e7d5c070610fb308a3c
Is binary

      
    
Is text
True
Is archive

      
    
Is media

      
    
Is legal

      
    
Is manifest

      
    
Is readme

      
    
Is top level

      
    
Is key file

      
    
manage-topics.md | 4.2 KB |

--- title: Managing Streams Application Topics description: weight: 11 tags: ['kafka', 'docs'] aliases: keywords: type: docs --- <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> A Kafka Streams application continuously reads from Kafka topics, processes the read data, and then writes the processing results back into Kafka topics. The application may also auto-create other Kafka topics in the Kafka brokers, for example state store changelogs topics. This section describes the differences these topic types and how to manage the topics and your applications. Kafka Streams distinguishes between user topics and internal topics. # User topics User topics exist externally to an application and are read from or written to by the application, including: Input topics Topics that are specified via source processors in the application's topology; e.g. via `StreamsBuilder#stream()`, `StreamsBuilder#table()` and `Topology#addSource()`. Output topics Topics that are specified via sink processors in the application's topology; e.g. via `KStream#to()`, `KTable.to()` and `Topology#addSink()`. User topics must be created and manually managed ahead of time (e.g., via the [topic tools](../../kafka/post-deployment.html#kafka-operations-admin)). If user topics are shared among multiple applications for reading and writing, the application users must coordinate topic management. If user topics are centrally managed, then application users then would not need to manage topics themselves but simply obtain access to them. Note You should not use the auto-create topic feature on the brokers to create user topics, because: * Auto-creation of topics may be disabled in your Kafka cluster. * Auto-creation automatically applies the default topic settings such as the replicaton factor. These default settings might not be what you want for certain output topics (e.g., `auto.create.topics.enable=true` in the [Kafka broker configuration](http://kafka.apache.org/0100/documentation.html#brokerconfigs)). # Internal topics Internal topics are used internally by the Kafka Streams application while executing, for example the changelog topics for state stores. These topics are created by the application and are only used by that stream application. If security is enabled on the Kafka brokers, you must grant the underlying clients admin permissions so that they can create internal topics set. For more information, see [Streams Security](security.html#streams-developer-guide-security). Note The internal topics follow the naming convention `<application.id>-<operatorName>-<suffix>`, but this convention is not guaranteed for future releases. The following settings apply to the default configuration for internal topics: * For all internal topics, `message.timestamp.type` is set to `CreateTime`. * For internal repartition topics, the compaction policy is `delete` and the retention time is `-1` (infinite). * For internal changelog topics for key-value stores, the compaction policy is `compact`. * For internal changelog topics for windowed key-value stores, the compaction policy is `delete,compact`. The retention time is set to 24 hours plus your setting for the windowed store. * For internal changelog topics for versioned state stores, the cleanup policy is `compact`, and `min.compaction.lag.ms` is set to 24 hours plus the store's historyRetentionMs` value. * [Documentation](/documentation) * [Kafka Streams](/documentation/streams) * [Developer Guide](/documentation/streams/developer-guide/)
Detected license expression
apache-2.0
Detected license expression (SPDX)
Apache-2.0
Percentage of license text
19.16
Copyrights

      
    
Holders

      
    
Authors

      
    
License detections License expression License expression SPDX
apache_2_0-4bde3f57-78aa-4201-96bf-531cba09e7de apache-2.0 Apache-2.0
URL Start line End line
http://www.apache.org/licenses/LICENSE-2.0 19 19
http://kafka.apache.org/0100/documentation.html#brokerconfigs 49 49