window10单机安装storm集群

适合范围:storm自由开源的分布式实时计算系统,擅长处理海量数据。适合处理实时数据而不是批处理。

安装前的准备

1.安装zookeeper

①下载zookeeperhttps://zookeeper.apache.org/,点击download进入新页面之后,--->download,选择一个镜像文件下载到本地;

②下面是我选择的文件http://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/ ,下载的版本是zookeeper-3.4.14;

③解压后复制D:\******\zookeeper-3.4.14\conf\zoo_sample.cfg,并将其命名为zoo.cfg,打开zoo.cfg,将dataDir=/tmp/zookeeper修改为

dataDir=D:\\***\\zookeeper-3.4.14\\data
dataLogDir=D:\\***\\zookeeper-3.4.14\\log 

其中,***为省略的路径。

④配置环境变量(可省略,在指定目录下⑤,启动即可)

⑤进入D:\******\zookeeper-3.4.14\bin\,启动zookeeper。

zkServer.cmd 

2.安装python

①下载https://www.python.org/,选择download,进入新页面--->Files--->Windows x86-64 web-based installer,下载到本地;

②点击安装包,安装到指定目录D:\***\python-3.7.3;

3.安装storm

①点击http://storm.apache.org/downloads.html,点击右上角download,选择apache-storm-1.2.2.tar.gz下载到本地;

②解压

③配置环境变量(可省略,在指定文件夹下启动即可。)

④D:\***\apache-storm-1.2.2\bin\storm-config.cmd

⑤修改配置文件D:\***\apache-storm-1.2.2\conf\storm.yaml

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

########### These MUST be filled in for a storm configuration
 storm.zookeeper.servers:
     - "127.0.0.1"
#     - "server2"
#
# nimbus.seeds: ["host1", "host2", "host3"]
 nimbus.seeds: ["127.0.0.1"]
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"

## Metrics Consumers
## max.retain.metric.tuples
## - task queue will be unbounded when max.retain.metric.tuples is equal or less than 0.
## whitelist / blacklist
## - when none of configuration for metric filter are specified, it‘ll be treated as ‘pass all‘.
## - you need to specify either whitelist or blacklist, or none of them. You can‘t specify both of them.
## - you can specify multiple whitelist / blacklist with regular expression
## expandMapType: expand metric with map type as value to multiple metrics
## - set to true when you would like to apply filter to expanded metrics
## - default value is false which is backward compatible value
## metricNameSeparator: separator between origin metric name and key of entry from map
## - only effective when expandMapType is set to true
# topology.metrics.consumer.register:
#   - class: "org.apache.storm.metric.LoggingMetricsConsumer"
#     max.retain.metric.tuples: 100
#     parallelism.hint: 1
#   - class: "org.mycompany.MyMetricsConsumer"
#     max.retain.metric.tuples: 100
#     whitelist:
#       - "execute.*"
#       - "^__complete-latency$"
#     parallelism.hint: 1
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"
#     expandMapType: true
#     metricNameSeparator: "."

## Cluster Metrics Consumers
# storm.cluster.metrics.consumer.register:
#   - class: "org.apache.storm.metric.LoggingClusterMetricsConsumer"
#   - class: "org.mycompany.MyMetricsConsumer"
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"
#
# storm.cluster.metrics.consumer.publish.interval.secs: 60

# Event Logger
# topology.event.logger.register:
#   - class: "org.apache.storm.metric.FileBasedEventLogger"
#   - class: "org.mycompany.MyEventLogger"
#     arguments:
#       endpoint: "event-logger.mycompany.org"

# Metrics v2 configuration (optional)
#storm.metrics.reporters:
#  # Graphite Reporter
#  - class: "org.apache.storm.metrics2.reporters.GraphiteStormReporter"
#    daemons:
#        - "supervisor"
#        - "nimbus"
#        - "worker"
#    report.period: 60
#    report.period.units: "SECONDS"
#    graphite.host: "localhost"
#    graphite.port: 2003
#
 supervisor.slots.ports:
    - 9700
    - 9701
    - 9702
    - 9703
 storm.local.dir: "D:\\****\\storm"
 ui.port: 6066
#  # Console Reporter
#  - class: "org.apache.storm.metrics2.reporters.ConsoleStormReporter"
#    daemons:
#        - "worker"
#    report.period: 10
#    report.period.units: "SECONDS"
#    filter:
#        class: "org.apache.storm.metrics2.filters.RegexFilter"
#        expression: ".*my_component.*emitted.*"

⑥启动storm,分别启动nimbus,supervisor,ui【需要打开三个窗口输入命令】

进入D:\***\apache-storm-1.2.2\bin

storm.py nimbus
storm.py supervisor
storm.py ui

⑦依次启动完毕之后,在浏览器输入http://127.0.0.1:6066/访问【访问UI的端口号可自定义,在上述文件中修改即可】

ui.port: 6066

⑧启动页面

原文地址:https://www.cnblogs.com/sugar-6847/p/10774782.html

时间: 2024-10-11 20:41:27

window10单机安装storm集群的相关文章

安装storm集群

手工安装Storm集群 注:最新的Storm已不再必须依赖ZeroMQ,各种依赖的库和软件也已经有更新的版本. 要手工安装Storm,需要先安装以下软件 Zookeeper集群(安装方法详见管理向导) Java6.0 Python2.6.6 Unzip命令 NOTE: Nimbus和管理进程将要依赖Java.Python和unzip命令 安装本地库: 安装ZeroMQ: 1 wget http://download.zeromq.org/historic/zeromq-2.1.7.tar.gz

CentOS6.5 安装Storm集群

1.首先安装依赖包 [[email protected] ~]# yum install uuid* [[email protected] ~]# yum install libuuid [[email protected] ~]# yum install libuuid-devel [[email protected] ~]# yum install libtool [[email protected] ~]# yum install e2fsprogs [[email protected]

在Ubuntu15.10下安装Storm集群(一个机器上运行Nimbus和Supervisor 适合初学者)

1. 下载所需的资源 jdk-8u65-linux-x64.tar.gz zeromq-4.1.4.tar.gz jzmq-master.zip zookeeper-3.5.1-alpha.tar.gz apache-storm-0.10.0.tar.gz 2. 安装jdk 我们打算把jdk安装在/usr/lib/jvm中:首先把Windows上下载好的jdk包复制到Ubuntu系统的用户文件夹(/home/<你的用户名>)中(如果无法复制可能是你的VMTools没有正确安装,需要重新安装VM

Elasticsearch单机安装_集群搭建_索引基本操作_Head插件安装与基本操作_ik分词器配置_logstash实现数据同步_教程

一.Elasticsearch单机安装 1.将es安装包传到服务器上 这是安装包 这里我是用的是WinSCP上传工具,上传到/home/plugins文件下. 进入Xshell,验证一下是否上传成功. 2.创建es目录 mkdir -p /usr/local/es(es1,es2,es3暂时不要安装,确保第一步es下面目录是空的,由于已经安装过,这三个文件留着了) 2.解压文件至 es 目录 tar -zxvf /home/plugins/elasticsearch-2.4.5.tar.gz -

Storm集群安装部署步骤【详细版】

作者: 大圆那些事 | 文章可以转载,请以超链接形式标明文章原始出处和作者信息 网址: http://www.cnblogs.com/panfeng412/archive/2012/11/30/how-to-install-and-deploy-storm-cluster.html 本文以Twitter Storm官方Wiki为基础,详细描述如何快速搭建一个Storm集群,其中,项目实践中遇到的问题及经验总结,在相应章节以“注意事项”的形式给出. 1. Storm集群组件 Storm集群中包含两

1.1 Storm集群安装部署步骤

安装storm集群,需要依赖以下组件: Zookeeper Python Zeromq Storm JDK JZMQ 故安装过程根据上面的组件分为以下几步: 安装JDK 安装Zookeeper集群 安装Python及依赖 安装Storm 另外,操作系统环境为:Centos6.4,安装用户为:root. 1. 安装JDK 安装jdk有很多方法,可以参考文博客使用yum安装CDH Hadoop集群中的jdk安装步骤,需要说明的是下面的zookeeper集群安装方法也可以参考此文. 不管你用什么方法,

Storm集群的安装与测试

首先安装zookeeper集群,然后安装storm集群. 我使用的是centos 32bit的三台虚拟机. MachineName ip namenode 192.168.99.110 datanode01 192.168.99.111 datanode02 192.168.99.112 首先配置好ssh,可以无密码互访. 安装好jdk,我本机安装的是1.7. 安装好python.机器自带2.6.6. 安装zookeeper的时候,下载文件,解压,在 /etc/profile 中设置环境变量.

Storm集群安装详解

Storm集群安装详解 storm有两种操作模式: 本地模式和远程模式. 本地模式:你可以在你的本地机器上开发测试你的topology, 一切都在你的本地机器上模拟出来; 远端模式:你提交的topology会在一个集群的机器上执行. 本文以Twitter Storm官方Wiki为基础,详细描述如何快速搭建一个Storm集群,其中,项目实践中遇到的问题及经验总结,在相应章节以“注意事项”的形式给出. 1.   Strom集群组件 Storm集群中包含两类节点:主控节点(Master Node)和工

Storm集群的安装配置

Storm集群的安装分为以下几步: 1.首先保证Zookeeper集群服务的正常运行以及必要组件的正确安装 2.释放压缩包 3.修改storm.yaml添加集群配置信息 4.使用storm脚本启动相应服务并查看服务状态 5.通过web查看storm集群的状态 安装Storm之前首先保证之前安装的Zookeeper服务正常运行,包括配置hosts映射,主机名修改,防火墙都已经设置完好 Storm是由java编写,因此必须依赖JDK运行,系统首先应正确安装JDK 部分需要依赖Python,红帽系列L