kafka delete topic

Don‘t think it is supported yet. Take a look at this JIRA tracking Delete Topic. For delete manually

  1. Shutdown the cluster
  2. Clean kafka log dir (specified by the log.dir attribute in kafka config file ) as well the zookeeper data
  3. Restart the cluster

For any given topic what you can do is

  1. Stop kafka
  2. Clean kafka log specific to partition, kafka stores its log file in a format of "logDir/topic-partition" so for a topic named "MyTopic" the log for partition id 0 will be stored in /tmp/kafka-logs/MyTopic-0 where /tmp/kafka-logs is specified by the log.dir attribute
  3. Restart kafka

This is NOT a good and recommended approach but it should work. In the Kafka broker config file the log.retention.hours.per.topic attribute is used to define The number of hours to keep a log file before deleting it for some specific topic

bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --zookeeper localhost:2181 --topic test

命令删除后数据文件依然存在,上边的方法是根本删除

create kafka-delete-topic.sh file

#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
base_dir=$(dirname $0)
export KAFKA_OPTS="-Xmx512M -server -Dcom.sun.management.jmxremote -Dlog4j.configuration=file:$base_dir/kafka-console-consumer-log4j.properties"
$base_dir/kafka-run-class.sh kafka.admin.DeleteTopicCommand [email protected]

https://issues.apache.org/jira/browse/KAFKA-330

http://stackoverflow.com/questions/17730905/is-there-a-way-to-delete-all-the-data-from-a-topic-or-delete-the-topic-before-ev

kafka delete topic 问题跟进

https://issues.apache.org/jira/browse/KAFKA-330

kafka delete topic,布布扣,bubuko.com

时间: 2024-10-14 18:53:46

kafka delete topic的相关文章

Kafka中Topic级别配置

一.Kafka中topic级别配置 1.Topic级别配置 配置topic级别参数时,相同(参数)属性topic级别会覆盖全局的,否则默认为全局配置属性值. 创建topic参数可以设置一个或多个--config "Property(属性)",下面是创建一个topic名称为"my-topic"例子,它设置了2个参数max message size 和 flush rate. (A)创建topic时配置参数 bin/kafka-topics.sh --zookeeper

kafka删除topic的方法

0.8的官方文档提供了一个删除topic的命令: kafka-topics.sh --delete 但是在运行时会报错找不到这个方法. kafka-topics.sh最终是运行了kafka.admin.TopicCommand这个类,在0.8的源码中这个类中没有找到有delete topic相关的代码. 在kafka的admin包下,提供了一个DeleteTopicCommand的类,可以实现删除topic的功能. kafka.admin.DeleteTopicCommand 其中删除topic

【帖子】怎么彻底删除kafka的topic,然后重建?

怎么彻底删除kafka的topic,然后重建? 网上都说用kafka-run-class.shkafka.admin.DeleteTopicCommand 命令删除topic,但是并没有成功,用kafka-topics.sh命令查看依然可以查看到topic,应该怎样才能彻底删除topic? 2016-01-01 添加评论 分享 2 个回复 OpenSkill - OpenSkill官方账号 赞同来自: Ansible .being **kafka 0.8.1.1以及之前版本**都无法使用类似一条

【转】Kafka某topic无法消费解决方案&Kafka某Topic数据清理

由于项目原因,最近经常碰到Kafka消息队列某topic在集群宕机重启后无法消费的情况.碰到这种情况,有三步去判断原因所在: step A:如果用kafka串口(即console-consumer)是可以正常消费该topic,则排除kafka集群出现故障 step B:若平台业务能正常消费其他topic的消息,则排除平台业务代码逻辑问题 step C:不到万不得已,则只能手动删除kafka的对应topic的Log,但是清理Kafka Log又不能单纯的去删除中间环节产生的日志,中间关联的很多东西

0022-如何永久删除Kafka的Topic

温馨提示:要看高清无码套图,请使用手机打开并单击图片放大查看. 1.问题描述 使用kafka-topics --delete命令删除topic时并没有真正的删除,而是把topic标记为:"marked for deletion",导致重新创建相同名称的Topic时报错"already exists". 2.问题复现 1.登录Kafka集群所在的服务器,创建一个test的topic [[email protected] ~]# kafka-topics --creat

Kafka删除topic操作步骤

1.如果配置了delete.topic.enable=true 适合以下操作 通过命令删除:./kafka-topics.sh --zookeeper 127.0.0.1:2181 --delete --topic apply_test 通过zookeeper-client 删除掉broker下的topic, rmr /brokers/topics/apply_test 删除kafka节点的 kafka/logs/apply_test-0, 然后重启kafka集群 重新创建即可: /data1/

kafka删除topic后再创建同名的topic报错(ERROR org.apache.kafka.common.errors.TopicExistsException)

[[email protected] logs]$ kafka-topics.sh --delete --zookeeper datanode1:2181 --topic firstTopic first is marked for deletion.Note: This will have no impact if delete.topic.enable is not set to true. [[email protected] logs]$ kafka-topics.sh  --creat

kafka删除topic

手动: 删除kafka存储目录(server.properties文件log.dirs配置,默认为"/tmp/kafka-logs")相关topic目录 删除zookeeper "/brokers/topics/"目录下相关topic节点 命令+手动: bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --topic mrtopic --zookeeper cmcc1:2181,cmcc2:2181,cmcc

Kafka消息topic分区

kafka是为分布式环境设计的,因此如果日志文件,其实也可以理解成消息数据库,放在同一个地方,那么必然会带来可用性的下降,一挂全挂,如果全量拷贝到所有的机器上,那么数据又存在过多的冗余,而且由于每台机器的磁盘大小是有限的,所以即使有再多的机器,可处理的消息还是被磁盘所限制,无法超越当前磁盘大小.因此有了partition的概念. kafka对消息进行一定的计算,通过hash来进行分区.这样,就把一份log文件分成了多份.如上面的分区读写日志图,分成多份以后,在单台broker上,比如快速上手中,