Kafka spring 集成

下载配置kafka参考该链接:http://www.cnblogs.com/super-d2/p/4534323.html

pom.xml:

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.10</artifactId>
            <version>0.8.2.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.integration</groupId>
            <artifactId>spring-integration-kafka</artifactId>
            <version>1.2.0.RELEASE</version>
        </dependency>

producer配置文件:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:int-kafka="http://www.springframework.org/schema/integration/kafka"
       xmlns:int="http://www.springframework.org/schema/integration"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
       http://www.springframework.org/schema/context
       http://www.springframework.org/schema/context/spring-context-4.0.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-4.0.xsd
       http://www.springframework.org/schema/integration
       http://www.springframework.org/schema/integration/spring-integration-4.1.xsd
       http://www.springframework.org/schema/integration/kafka
       http://www.springframework.org/schema/integration/kafka/spring-integration-kafka.xsd
       ">

    <!--kafka config-->
    <int:channel id="inputToKafka"/>

    <int-kafka:outbound-channel-adapter kafka-producer-context-ref="kafkaProducerContext"
                                        auto-startup="true"
                                        channel="inputToKafka"
                                        order="1">
    </int-kafka:outbound-channel-adapter>

    <bean id="producerProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
        <property name="properties">
            <props>
                <prop key="topic.metadata.refresh.interval.ms">3600000</prop>
                <prop key="message.send.max.retries">5</prop>
                <prop key="send.buffer.bytes">5242880</prop>
            </props>
        </property>
    </bean>

    <bean id="stringSerializer" class="org.apache.kafka.common.serialization.StringSerializer"/>

    <int-kafka:producer-context id="kafkaProducerContext" producer-properties="producerProperties">
        <int-kafka:producer-configurations>

            <int-kafka:producer-configuration broker-list="localhost:9092"
                                              key-serializer="stringSerializer"
                                              value-class-type="java.lang.String"
                                              value-serializer="stringSerializer"
                                              topic="helloworldTopic"/>

        </int-kafka:producer-configurations>
    </int-kafka:producer-context>
</beans>

consumer配置:

<?xml version="1.0" encoding="UTF-8"?><beans xmlns="http://www.springframework.org/schema/beans"       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"       xmlns:int="http://www.springframework.org/schema/integration"       xmlns:int-kafka="http://www.springframework.org/schema/integration/kafka"       xsi:schemaLocation="      http://www.springframework.org/schema/integration/kafka http://www.springframework.org/schema/integration/kafka/spring-integration-kafka.xsd      http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd      http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <int:channel id="inputFromKafka"/>

    <int-kafka:zookeeper-connect id="zookeeperConnect" zk-connect="localhost:2181"                                 zk-connection-timeout="6000"                                 zk-session-timeout="6000"                                 zk-sync-time="2000"/>

    <int-kafka:inbound-channel-adapter            id="kafkaInboundChannelAdapter"            kafka-consumer-context-ref="consumerContext"            auto-startup="false"            channel="inputFromKafka">        <int:poller fixed-delay="1" time-unit="MILLISECONDS"/>    </int-kafka:inbound-channel-adapter>

    <bean id="kafkaDecoder" class="org.springframework.integration.kafka.serializer.common.StringDecoder">    </bean>

    <bean id="consumerProperties" class="org.springframework.beans.factory.config.PropertiesFactoryBean">        <property name="properties">            <props>                <prop key="auto.offset.reset">smallest</prop>                <prop key="socket.receive.buffer.bytes">10485760</prop> <!-- 10M -->                <prop key="fetch.message.max.bytes">5242880</prop>                <prop key="auto.commit.interval.ms">1000</prop>            </props>        </property>    </bean>

    <bean id="kafkaService" class="cn.innmall.union.function.service.work.KafkaService">    </bean>

    <int:outbound-channel-adapter channel="inputFromKafka"                                  ref="kafkaService" method="processMessage" />

    <int-kafka:consumer-context id="consumerContext"                                consumer-timeout="1000"                                zookeeper-connect="zookeeperConnect" consumer-properties="consumerProperties">        <int-kafka:consumer-configurations>            <int-kafka:consumer-configuration group-id="default1"                                              value-decoder="kafkaDecoder"                                              key-decoder="kafkaDecoder"                                              max-messages="5000">                <int-kafka:topic id="helloworldTopic" streams="4"/>            </int-kafka:consumer-configuration>        </int-kafka:consumer-configurations>    </int-kafka:consumer-context></beans>

producer 测试代码:

public class KafkaServiceImpl implements KafkaService {
    @Autowired
    @Qualifier("inputToKafka")
    MessageChannel channel;

    public void sendUserInfo(String key, Object obj) {
        Message msg = MessageBuilder.withPayload(obj)
                .setHeader("kafkaUser", key)
                .setHeader(KafkaHeaders.TOPIC, "helloworldTopic").build();
        channel.send(msg);
    }
}

consumer测试代码:

package cn.innmall.union.function.service.work;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Map;

/**
 * Created by yujinghui on 4/23/16.
 *
 */
public class KafkaService {
    static final Logger logger = LoggerFactory.getLogger(KafkaService.class);

    public void processMessage(Map<String, Map<Integer, String>> msgs) {
        for (Map.Entry < String,Map<Integer, String>>entry:
        msgs.entrySet()){
            System.out.println("Consumer Message received: ");
            logger.debug("Suchit Topic:" + entry.getKey());
            for (String msg : entry.getValue().values()) {
                logger.info("Suchit Consumed Message: " + msg);
            }
        }
    }

}
时间: 2024-11-08 19:38:40

Kafka spring 集成的相关文章

RabbitMQ安装和使用(和Spring集成)

一.安装Rabbit MQ Rabbit MQ 是建立在强大的Erlang OTP平台上,因此安装Rabbit MQ的前提是安装Erlang.通过下面两个连接下载安装3.2.3 版本: 下载并安装 Eralng OTP For Windows (vR16B03) 运行安装 Rabbit MQ Server Windows Installer (v3.2.3) 具体操作步骤参考:在 Windows 上安装Rabbit MQ 指南 本人遇到的问题 当安装RabbitMQ后,使用rabbitmqctl

spring集成quartz

spring集成quartz 注意:出现异常"Caused by: java.lang.IncompatibleClassChangeError: class org.springframework.scheduling.quartz.CronTriggerBean has interface org.quartz.CronTrigger as super class" Spring3.0不支持Quartz2.0,因为org.quartz.CronTrigger在2.0从class变成

Hessian入门(包括与Spring集成)

A.纯Hessian接口开发步骤 Hessian-JAVA服务器端必须具备以下几点: * 1.包含Hessian的jar包(hessian-4.0.37.jar) * 2.设计一个接口,用来给客户端调用(IHessian.java) * 3.实现该接口的功能(IHessianImpl.java) * 4.配置web.xml,配好相应的Servlet(web.xml) * 5.对象必须实现Serializable接口(Foo.java) Hessian-JAVA服务端代码预览图: 1.包含Hess

Struts2+Spring集成合并

前边单独总结了Struts2,Spring和Ibaits框架了,那么怎么结合使用呢?这次先来看一下Sturts2和Spring的集成合并.其实挺简单的,就是导入各自的jar包以及连接彼此的jar包,分好彼此的工作就可以了. 好看一下Struts2+Spring的集成方案!  Struts2和Spring集成有两种方案,是根据action的创建来划分的!  方案一,Struts2负责流程,Spring负责对象的创建:Action由Struts2框架负责创建:Service由Spring框架负责创建

Java Persistence with MyBatis 3(中文版) 第五章 与Spring集成

MyBatis-Spring是MyBatis框架的子模块,用来提供与当前流行的依赖注入框架Spring的无缝集成. Spring框架是一个基于依赖注入(Dependency Injection)和面向切面编程(Aspect Oriented Programming,AOP)的Java框架,鼓励使用基于POJO的编程模型.另外,Spring提供了声明式和编程式的事务管理能力,可以很大程度上简化应用程序的数据访问层(data access layer)的实现.在本章中,我们将看到在基于Spring的

spring集成Log4j以及log4j配置简要说明

Spring集成: web.xml中配置log4j <context-param> <param-name>log4jConfigLocation</param-name> <param-value>WEB-INF/log4j.xml</param-value></context-param> <!-- 加载Spring框架中的log4j监听器Log4jConfigListener --><listener>

rabbitMQ第五篇:Spring集成RabbitMQ

前面几篇讲解了如何使用rabbitMq,这一篇主要讲解spring集成rabbitmq. 首先引入配置文件org.springframework.amqp,如下 <dependency> <groupId>org.springframework.amqp</groupId> <artifactId>spring-rabbit</artifactId> <version>1.6.0.RELEASE</version> <

Spring 集成 RMI

Maven <dependency> <groupId>org.springframework</groupId> <artifactId>spring-remoting</artifactId> </dependency> 服务端 <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.spri

activiti搭建(二)与Spring集成

转载请注明源地址:http://www.cnblogs.com/lighten/p/5876773.html 本文主要讲解如何将Activiti和Spring框架集成,再过一段时间将会将一个基础的demo放在github上,地址将会在activiti系列导读文章中给出,如果未给出,则是因为没有构建完成(学习中,更新较慢).按照搭建篇章的顺序,也能搭建起来. 上一章将了数据库的初始化,继续那章建立的maven项目.Activiti与Spring集成需要相应的jar包,下面将补充其他maven依赖: