IllegalArgumentException: requirement failed: Corrupt index found

今天突然接到客户反映线上服务器发送消息异常,登录服务器查看是kafka服务出现了问题,想重启一下服务,结果重启出现一下报错

[2017-06-30 19:29:13,708] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)

java.lang.IllegalArgumentException: requirement failed: Corrupt index found, index file (/data/apps/log/kafka/metrics-topic-13/00000000000000000000.index) has non-zero size but the last offset is 0 and the base offset is 0

at scala.Predef$.require(Predef.scala:219)

at kafka.log.OffsetIndex.sanityCheck(OffsetIndex.scala:352)

at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:184)

at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:183)

at scala.collection.Iterator$class.foreach(Iterator.scala:750)

at scala.collection.AbstractIterator.foreach(Iterator.scala:1202)

at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)

at scala.collection.AbstractIterable.foreach(Iterable.scala:54)

at kafka.log.Log.loadSegments(Log.scala:183)

at kafka.log.Log.<init>(Log.scala:67)

at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$7$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:142)

at kafka.utils.Utils$$anon$1.run(Utils.scala:54)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

[2017-06-30 19:29:13,711] INFO [Kafka Server 2], shutting down (kafka.server.KafkaServer)

解决方法如下:

find kafka-logs/  -size 10485760c -name *.index

将以上命令中搜索到的所有文件删除,在重启kafka,kafka会重新生成相应的文件,询问研发人员说删除的数据不会对线上服务造成影响,但是如果你也碰到了这种问题最好还是要先问问你们部门的研发人员,在做相关的操作

时间: 2024-08-26 08:19:39

IllegalArgumentException: requirement failed: Corrupt index found的相关文章

Kafka日志报错:[error] k.m.j.KafkaJMX$ - Failed to connect to service:jmx:rmi:///jndi/rmi://10.1.3.116:-1/jmxrmi java.lang.IllegalArgumentException: requirement failed: No jmx port but jmx polling enabled!

一.Kafka日志报错:[error] k.m.j.KafkaJMX$ - Failed to connect to service:jmx:rmi:///jndi/rmi://10.1.3.116:-1/jmxrmi java.lang.IllegalArgumentException: requirement failed: No jmx port but jmx polling enabled! 以上报错,并没有对Kafka对使用造成直接影响: 1>.启动生产者:kafka-console

[java][spark streamming]java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute

No output streams registered, so nothing to execute解决原因:没有触发DStream需要的aciton解决方法:使用以下方法之一触发: print() foreachRDD() saveAsObjectFiles() saveAsTextFiles() saveAsHadoopFiles() 原文地址:https://www.cnblogs.com/fadedlemon/p/11738440.html

Oozie Spark on YARN requirement failed

软件环境: CDH:5.7.3:Oozie:4.1.0-CDH5.7.3 : Spark:1.6.0-cdh5.7.3-hadoop2.6.0-cdh5.7.3 : Hadoop:hadoop2.6.0-cdh5.7.3(HDFS 采用HA方式): 问题描述: 在使用CDH5.7.3版本的时候,发起一个Oozie工作流,该工作流使用Spark On YARN的方式提交一个Spark程序,但是在Oozie中该程序运行失败,同时找到YARN监控中对应的任务,发现出现下面的错误(该Spark任务如果使

大数据常见错误

1.用./bin/spark-shell启动spark时遇到异常:java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries! 解决方法:add export SPARK_LOCAL_IP="127.0.0.1" to spark-env.sh 2.java Kafka producer error:ERROR kafka.utils.Util

Programming in Scala (Second Edition) 读书笔记6 函数式对象

1.什么是函数式对象(functional object) ? The emphasis in this chapter is on classes that define functional objects, that is, objects that do not have any mutable state. 状态不可变的对象 2. 对函数式对象的取舍(immutable object trade-offs) Immutable objects offer several advanta

Programming In Scala Reading Note 5

Functional Object Advantages and Disadvantages of Immutable Object Adv 1. 不可变对象更具容易被推断出来,因为它不会随着时间的变化而造成值的变化. Adv 2. 不可变对象可以自动地传递不会发生变化,而传递可变对象的话,需要首先对可变对象进行备份,以防止在其它过程中对对象的状态造成了修改. Adv 3. 线程不能够更改(corrupt:腐化,堕落)不可变对象的状态,注定他是线程安全的. Adv 4. 不可变对象拥有安全的Ha

大数据常见错误解决方案(转载)

1.用./bin/spark-shell启动spark时遇到异常:java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries! 解决方法:add export SPARK_LOCAL_IP="127.0.0.1" to spark-env.sh 2.java Kafka producer error:ERROR kafka.utils.Util

Scala断言

断言:提供了一组断言函数以用作在代码中记录和动态检查不变量的方式. import scala.Predef._ def addNaturals(nats: List[Int]): Int = { // assert(nats forall (_ >= 0), "List contains negative numbers") require(nats forall (_ >= 0), "List contains negative numbers") n

scala 基本语法_2

[[email protected] test]# scala Welcome to Scala version 2.9.3 (Java HotSpot(TM) Client VM, Java 1.7.0_67). Type in expressions to have them evaluated. Type :help for more information. scala> val oneHalf = new Rational(1, 2); <console>:7: error: