向Kafka中输入数据，抛异常org.apache.kafka.common.errors.RecordTooLargeException
官网两个参数描述如下：
| message.max.bytes | The maximum size of message that the server can receive | int | 1000012 | [0,...] | high |
| fetch.message.max.bytes | 1024 * 1024 | The number of byes of messages to attempt to fetch for each topic-partition in each fetch request. These bytes will be read into memory for each partition, so this helps control the memory used by the consumer. The fetch request size must be at least as large as the maximum message size the server allows or else it is possible for the producer to send messages larger than the consumer can fetch. |
message.max.bytes：server能接受消息体的最大值。
fetch.message.max.bytes：consumer从partition中获取消息体放入内存中，这个参数控制conusmer所用的内存大小。如果message.max.bytes大于fetch.message.max.bytes，就会导致consumer分配的内存放不下一个message。
因此，在server.properties中添加两个配置项
#broker能接收消息的最大字节数 message.max.bytes=20000000 #broker可复制的消息的最大字节数 replica.fetch.max.bytes=20485760
如果不想修改配置文件，可以采用修改topic配置的方法，与server配置项message.max.bytes对应的topic配置项是max.message.bytes
bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic my-topic --config max.message.bytes=128000
时间： 2024-08-02 02:51:07