Yarn 集群资源管理系统
Yarn 角色及概念
?Yarn 是 Hadoop 的一个通用的资源管理系统
? Yarn 角色
– Resourcemanager
– Nodemanager
– ApplicationMaster
– Container
– Client
? ResourceManager
– 处理客户端请求
– 启动 / 监控 ApplicationMaster
– 监控 NodeManager
– 资源分配与调度
? NodeManager
– 单个节点上的资源管理
– 处理来自 ResourceManager 的命令
– 处理来自 ApplicationMaster 的命令
? Container
– 对任务运行行环境的抽象,封装了 CPU 、内存等
– 多维资源以及环境变量、启动命令等任务运行相关的信息资源分配与调度
? ApplicationMaster
– 数据切分
– 为应用程序申请资源,并分配给内部任务
– 任务监控与容错
? Client
– 用户与 YARN 交互的客户端程序
– 提交应用程序、监控应用程序状态,杀死应用程序等
Yarn 结构
? YARN 的核心思想
? 将 JobTracker 和 TaskTacker 进行分离,它由下面几大构成组件:
– ResourceManager 一个全局的资源管理器
– NodeManager 每个节点(RM)代理
– ApplicationMaster 表示每个应用
– 每一个 ApplicationMaster 有多个 Container 在NodeManager 上运行
系统规划
主机 角色 软件
192.168.4.1 master Resource Manager YARN
192.168.4.2 node1 Node Manager YARN
192.168.4.3 node2 Node Manager YARN
192.168.4.4 node3 Node Manager YARN
Yarn 安装与配置
具体实验准备 可以参考 http://blog.51cto.com/13558754/2066708
# ssh 192.168.4.1
# cd /usr/local/hadoop/
# cd etc/hadoop/
# cp mapred-site.xml.template mapred-site.xml
# vim mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value> // 配置使用yarn 资源管理系统
</property>
</configuration>
# vim yarn-site.xml
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value> //配置 Resource Manager 角色
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value> // 一个 java 的类 真实环境与开发人员沟通
</property>
</configuration>
配置完成以后
# for i in node{1..3} //将配置文件同步到所有主机
> do
> rsync -azSH --delete /usr/local/hadoop/etc/hadoop/ ${i}:/usr/local/hadoop/etc/hadoop -e 'ssh'
> done
# cd /usr/local/hadoop/
启动 yarn 服务
# ./sbin/start-yarn.sh
在所有主机上执行 jps, 查看是否启动成功
# for i in master node{1..3}
> do
> echo ${i}
> ssh ${i} "jps"
> done
master
3312 Jps
3005 ResourceManager
node1
3284 Jps
3162 NodeManager
node2
2882 NodeManager
3004 Jps
node3
2961 Jps
2831 NodeManager
显示所有可用的计算节点
# ./bin/yarn node -list
18/01/31 06:41:56 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.4.1:8032
Total Nodes:3
Node-Id Node-StateNode-Http-AddressNumber-of-Running-Containers
node3:46007 RUNNING node3:8042 0
node2:54895 RUNNING node2:8042 0
node1:51087 RUNNING node1:8042
resourcemanager
nodemangager
验证 Yarn
# bin/hadoop fs -ls /input
Found 3 items
-rw-r--r-- 2 root supergroup 84854 2018-01-29 21:37 /input/LICENSE.txt
-rw-r--r-- 2 root supergroup 14978 2018-01-29 21:37 /input/NOTICE.txt
-rw-r--r-- 2 root supergroup 1366 2018-01-29 21:37 /input/README.txt
使用yarn 统计 样本文件中单词出现频率
# ./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount hdfs://master:9000/input hdfs://master:9000/output
查看结果
# ./bin/hadoop fs -cat hdfs://master:9000/output/*
""AS 2
"AS 17
"COPYRIGHTS 1
"Contribution" 2
"Contributor" 2
"Derivative 1
"GCC 1
"Legal 1
"License" 1
"License"); 2
"Licensed 1
"Licensor" 1
"Losses") 1
"NOTICE" 1
"Not 1
"Object" 1
"Program" 1
"Recipient" 1
"Software"), 5
"Source" 1
"Work" 1
"You" 1
"Your") 1
"[]" 1
"control" 1
"printed 1
"submitted" 1
& 1
'Aalto 1
'Apache 4
'ArrayDeque', 1
'Bouncy 1
'Caliper', 1
'Compress-LZF', 1
'HPACK', 1
'JBoss 1
'JCTools', 1
'JZlib', 1
'Protocol 1
'SLF4J', 1
'Snappy', 1
'Webbit', 1
'jbzip2', 1
'jfastlz', 1
'libdivsufsort', 1
'license' 1
'lz4', 1
'lzma-java', 1
'originates' 1
("AGREEMENT"). 1
("Commercial 1
("Indemnified 1
(1) 3
(1) Modifications 1
(1) for 2
(2) 3
(2) for 2
(2) the 1
(3) 1
(3) under 1
(50%) 3
(ASL2 1
(Apache 11
(BIS), 1
(BSD 2
(C) 3
(CDDL) 1
(CDDL)Version 1
(Don't 1
(ECCN) 1
(GNU 1
(INCLUDING 15
(INCLUDING, 11
(June 2
(MIT 5
(MIT) 1
(NOT 2
(New 3
(Oct. 2
(Public 4
(Sept. 2
(TSU) 1
(We 1
(a) 11
(a) rename 1
(a) the 1
(an 1
(and 3
(as 2
(b) 13
(b) otherwise 1
(b) ownership 1
(based 1
(c) 18
(collectively 1
(d) 5
(except 7
(excluding 5
(http://code.google.com/p/leveldb/), 1
(http://code.google.com/p/snappy/), 1
(http://www.apache.org/). 2
(http://www.one-lab.org) 1
(http://www.opensource.org/licenses/bsd-license.php) 1
(https://github.com/dain/leveldb), 1
(https://github.com/fusesource/hawtjni), 1
(https://github.com/fusesource/leveldbjni), 1
(hypertext 1
(i) 4
(i) the 1
(i) third 1
(if 4
(ii) 4
(ii) the 2
(iii) 1
(including 7
([email protected]) 1
(meaning 2
(or 17
(other 4
(see 1
(see: 1
(such 2
(the 10
* 174
*/ 3
+ 2
, 1
- 16
------ 8
--------------------------------------------------------------------- 1
------------------------------------------------------------------------------- 1
-------------------------------------------------------------------------------- 14
/* 1
/** 2
0.1.42 1
0.52 1
0.9.94 1
034819 1
1 1
1. 5
1.0 4
1.0. 1
1.0.4.1, 1
1.1 1
1.1. 1
1.1. Contributor means 1
1.10. 1
1.10. Original 1
1.11. 1
1.11. Patent 1
1.12. 1
1.12. Source 1
1.13. 1
1.13. You 1
1.2. 1
1.2.17, 1
1.2. Contributor 1
1.3 1
1.3. 1
1.3.04 1
1.3. Covered 1
1.4, 1
1.4. 1
1.4. Executable means 1
1.5. 1
1.5. Initial 1
1.6. 1
1.6. Larger 1
1.7. 1
1.7.10 1
1.7. License means 1
1.8 1
1.8. 1
1.8.5 1
1.8. Licensable means 1
1.9 1
1.9. 1
1.9. Modifications means 1
1. Definitions. 1
10. 2
12.212 3
1995), 4
1995). 2
1995-2000 1
1995-2016 1
1996 1
1999, 2
1999-2007 1
1999. 1
2(a) 1
2(b) 2
2-Clause 1
2. 6
2.0 5
2.0) 10
2.0, 2
2.0.0 1
2.1 3
2.1(a) 1
2.1(b) 1
2.1) 1
2.1. 2
2.101 1
2.2 3
2.2(a) 1
2.2(b) 3
2.2. 2
2.2.2 1
2.2.3 1
2.2.4, 1
2.3 1
2.5 1
2.5.0 1
2.6, 1
2.9.1, 1
2001 1
2001-2016, 1
2002 2
2002-2012 1
2004 2
2005, 1
2005-2006 1
2008 1
2008,2009,2010 1
2008-2016, 1
2010 1
2011 4
2011, 1
2011-2014, 1
2014 2
2014, 1
227.7202-1 2
227.7202-4 2
252.227-7014(a)(1)) 1
3-clause 1
3. 4
3.0.0 1
3.1 4
3.1. 2
3.2 1
3.2. 2
3.3. 2
3.4. 2
3.5. 2
3.6. 2
30 2
4(d) 1
4. 4
4.1. 2
4.11 1
4.2. 2
4.3, 2
4.3. 2
48 8
48 C.F.R. 12.212 1
48 C.F.R. 2.101 1
5. 4
5D002.C.1, 1
6. 4
6.1 1
6.1. 2
6.1.26, 1
6.2 2
6.2. 2
6.3. 2
6.4. 1
60 6
7. 4
740.13) 1
8. 3
9 1
9. 3
: 3
<COPYRIGHT 1
<[email protected]> 1
<http://www.wassenaar.org/> 1
<organization> 1
<[email protected]> 1
== 6
==== 2
================= 1
============================================================== 2
========================================================================= 2
=============================================================================== 1
A 22
A. 2
ACCEPTANCE 1
ACCOMPANYING 1
ACTION 5
ADVISED 12
AGREEMENT, 2
AGREEMENT. 1
AIX/POWER, 1
ALL 3
ALLOW 2
ALPN 2
AN 10
AND 63
ANY 94
APACHE 1
API 4
API, 2
APIs 1
APIs' 1
APPENDIX: 1
APPLICABLE 2
APPLY 4
ARE 11
ARISING 17
AS 5
ASL2 4
ASL2. 1
ASM 1
ASSUME 2
AUTHOR 2
AUTHORIZED 2
AUTHORS 5
AbstractNodeQueue, 1
Accepting 1
Additional 3
Administration 1
Adrien 1
Agreement 21
Agreement, 7
Agreement. 3
Agreement; 2
Aki 1
Alec 1
Aleksander 1
Alexander 1
All 12
Also, 1
America. 1
And 1
Any 13
Apache 20
Appendix 1
Application 2
April 1
As 3
Assorted 1
Atkinson 1
Atomicops 2
Authors. 1
Availability 2
B. 2
BASIS, 6
BE 18
BEEN 2
BEFORE 1
BIS 1
BSD 6
BSD-style 1
BUSINESS 11
BUT 27
BY 11
Base64 1
Based 1
Bay 2
Beams 1
Beider 1
Bloch 1
Bloomberg 1
Bozhanov 1
Broeke, 1
Buffer 2
Buffers 1
Buffers', 1
Bureau 1
Burrows-Wheeler 1
By 1
C 1
C. 2
C.F.R. 8
CAUSED 12
CDDL 3
CHARACTER 2
CIRCUMSTANCES 2
CLAIM, 5
CLAIMS. 2
COMMERCIAL 3
COMMON 2
COMPUTER 2
CONDITIONS 7
CONNECTION 5
CONSEQUENTIAL 16
CONSTITUTES 3
CONTRACT, 19
CONTRIBUTOR) 2
CONTRIBUTOR, 2
CONTRIBUTORS 20
COPYRIGHT 22
CORRECTION. 2
COST 2
COVERED 12
CRC 1
Castle 1
Catholique 1
Chris 1
Claims 6
Claims means 1
Claims” 1
Code 17
Code. 3
Code. 1
Codec 1
Code means 1
Code” 1
Collections 1
Collet. 1
Commerce, 1
Commercial 10
Commission 1
Commodity 1
Commons 5
Components 1
Concurrency 1
Conditioned 4
Consistent 2
Consulting 2
Container 2
Contracts 2
Contribution 10
Contribution(s) 3
Contribution, 2
Contribution. 2
Contribution." 1
Contributions 3
Contributions) 3
Contributions. 2
Contributor 91
Contributor") 2
Contributor's 2
Contributor, 6
Contributor. 8
Contributor. 2
Contributor: 1
Contributors 9
Contributors, 2
Contributors. 2
Control 1
Convention 2
Copyright 26
Core 4
Corp. 4
Corporation 2
Corporation., 1
Covered 46
Crypto 1
D3 1
DAMAGE. 11
DAMAGES 23
DAMAGES, 2
DAMAGES. 3
DATA, 11
DEALINGS 5
DEATH 2
DEFECTIVE 2
DEFECTS, 2
DEFINITIONS 1
DEVELOPER 2
DEVELOPER, 2
DEVELOPMENT 3
DFAR, 2
DIRECT, 12
DISCLAIMED. 11
DISCLAIMER 5
DISCLAIMER. 1
DISCLAIMER. 1
DISTRIBUTION 6
DISTRIBUTOR 2
DO 2
Dain 1
Daniel 1
Database 1
Decoder, 1
Definitions. 2
Department 1
Derivative 17
Developer 34
Developer) 2
Developer, 2
Developer. 1
Developer. 1
Developer means 1
Developer” 1
Development 3
Deville. 1
Disclaimer 1
Distribution 4
Dmitriy 1
Domain 1
Domain) 4
Doug 1
Dumbster 1
ECLIPSE 1
EG, 1
EITHER 3
ENC 1
END 3
ENTIRE 2
EPL 2
EPL. 1
ESSENTIAL 2
EVEN 14
EVENT 16
EXCEPT 4
EXCLUSION 4
EXEMPLARY, 12
EXERCISE 1
EXPRESS 17
EXPRESSED 2
EXPRESSLY 2
EXTENT 2
Each 8
Eclipse 5
Effect 2
Encoder 1
End 2
Eng 1
Entity 3
Entity" 1
European 1
Everyone 1
Except 3
Exception 1
Exception" 1
Executable 16
Export 2
External 1
FAILURE 2
FAR, 2
FIT 2
FITNESS 18
FOR 46
FORTH 2
FREE 2
FROM 2
FROM, 5
Fast 1
FastLZ 1
Fennell 1
File 1
Finance 1
FindBugs-jsr305 1
For 18
For: 7
Foundation 9
Framework 1
Framework,under 1
Francis. 1
FreeBSD 1
FuseSource 4
GENERAL 1
GOODS 11
GOODWILL, 2
GOVERNMENT 2
GPL 1
GPLv2 2
GRANT 1
GRANTED 1
GROUP, 2
GlassFish 1
Goods 2
Google 7
Google's 2
Google, 1
Government 7
Grand. 1
Grant 2
Grant. 4
Grants. 2
Greene: 1
Group 3
Group. 1
Group." 1
Group: 2
Gson 1
HADOOP 1
HAVE 3
HEREUNDER 2
HEREUNDER, 1
HOLDER> 1
HOLDERS 15
HOMEPAGE: 24
HOWEVER 12
HPACK 1
HSQL 4
HSQLDB 1
HSQLDB.ORG, 1
HTTP 1
HTTP/2 2
HYPERSONIC 1
Hadoop 2
Hadoop, 1
Hamcrest 1
Harder's 1
Harmony', 1
Hat 1
Header 1
Hoeller, 1
How 1
However, 4
Hypersonic 4
IBM 2
IF 14
IMPLIED 23
IMPLIED, 7
IN 59
INCIDENTAL 2
INCIDENTAL, 14
INCLUDING 5
INCLUDING, 16
INDIRECT, 14
INFORMED 2
INITIAL 4
INJURY 2
INTERRUPTION) 11
IS 27
IS" 12
IS"" 2
IS", 5
IS'' 1
IS” 1
If 17
In 7
Inc, 1
Inc. 7
Inc., 1
Inc: 1
Indemnified 3
Industry 1
Initial 40
Institute 1
International 2
Iris 1
It 8
Ivan 1
J. 1
JAXB 2
JLine 1
JSR-166 1
JSch 1
JURISDICTIONS 2
JUnit 1
JVM 1
JVM, 1
January 1
Jason 2
Java 16
Java, 2
Jersey 1
Jetty 6
Josh 1
Juergen 1
KIND, 11
Kevin 1
Kinney. 1
Kitchen 1
Kuhn's 1
LAW 2
LEGAL 2
LGPL 1
LIABILITY 6
LIABILITY, 29
LIABILITY. 2
LIABLE 18
LICENSE 6
LICENSE. 2
LICENSE.<component>.txt 1
LICENSE: 25
LICENSES 1
LIMITATION 9
LIMITATION, 5
LIMITATION. 2
LIMITED 27
LOSS 13
LOSSES, 2
LOST 2
LP. 1
LZ 1
LZ4 4
LZF 1
LZMA 1
Laddad, 1
Lang 1
Lang', 1
Larger 10
Lea, 1
Legal 3
LevelDB 1
Liability. 2
Library 2
Licensable 6
License 71
License) 12
License); 2
License, 12
License. 33
License. 9
License; 2
Licensed 3
Licensor 8
Licensor, 1
Limitation 1
Log4J', 1
Log4j 1
Logging', 1
Losses 1
Louvain 1
Ltd 1
Ltd. 2
MALFUNCTION, 2
MAY 2
MERCHANTABILITY 12
MERCHANTABILITY, 6
MERCHANTABLE, 2
MISCELLANEOUS. 2
MIT 1
MPSC 1
Marshalling', 1
Massachusetts 1
Matthew 1
Microsystems, 1
Microsystems., 1
Mockito 1
Modification. 2
Modification; 2
Modifications 23
Modifications, 8
Modifications. 3
Modifications. 1
Modifications; 2
Modified 2
Mori. 1
Morse. 1
Mort 2
MortBay 1
Mortbay 1
NECESSARY 2
NEGLIGENCE 14
NEGLIGENCE), 2
NEITHER 1
NO 23
NON-INFRINGEMENT, 2
NON-INFRINGING. 2
NONINFRINGEMENT. 5
NOR 1
NOT 33
NOTICE 7
Nations 2
Neither 8
Netty 4
New 5
Niels 1
Nitsan 1
No 3
Nothing 2
Notice 2
Notices. 2
Notwithstanding 5
Number 1
OF 161
ON 15
OR 162
OTHER 16
OTHERWISE) 12
OTHERWISE, 7
OUT 17
OW2 2
OWNER 7
Object 4
Obligations. 1
Obligations. 1
OneLab 1
OpenJDK 2
Oracle 3
Original 37
Otherwise, 2
P. 1
PART 2
PARTICULAR 20
PARTIES, 2
PARTY 2
PARTY'S 1
PARTYS 1
PERFORMANCE 2
PERSON 2
PERSONAL 2
POSSIBILITY 14
PROCUREMENT 11
PROFITS), 1
PROFITS, 1
PROFITS; 11
PROGRAM 4
PROHIBITS 2
PROVE 2
PROVIDED 20
PUBLIC 1
PURPOSE 18
PURPOSE. 2
ParaNamer 1
Parser 1
Participant 15
Participant) 3
Participant, 2
Participant. 1
Participant. 1
Patent 7
Patents 1
Patents" 1
Patents. 1
Paul 2
Performance 1
Permission 7
Please 1
Portions 1
Product 2
Program 22
Program, 3
Program. 6
Program; 1
Progress 1
Project 3
Project. 1
Protocol 3
Providing 1
Provisions 2
Provos 1
Pty 2
Public 3
PureJavaCrc32C 1
QUALITY 2
RECIPIENT 1
RECIPIENT'S 1
REPAIR 2
REPRODUCTION 1
REPRODUCTION, 1
REQUIREMENTS 1
RESPECT, 2
RESPONSIBILITY 2
RESULTING 2
RI 1
RIGHTS 2
RISK 2
RUntime 1
Ramnivas 1
Recipient 11
Recipient's 6
Recipients 1
Red 1
Redistribution 11
Redistribution. 1
Redistributions 22
Regulations, 1
Required 2
ResolverUtil.java 1
Rights 2
Robert 1
Roland 1
SE, 1
SERVICES; 11
SERVICING, 2
SET 2
SHALL 23
SHOULD 2
SLF4J 1
SMTP 1
SO 2
SOFTWARE 31
SOFTWARE, 13
SOFTWARE. 5
SOME 2
SPECIAL, 14
SQL 5
SSL 1
STOPPAGE, 2
STRICT 12
SUBCOMPONENTS: 1
SUBSTITUTE 11
SUCH 22
SUPPLIER 2
Sale 2
Saloranta 1
Saloranta. 1
Section 10
Sections 7
Sections 2.1 1
Sections 2.1(a) 1
Sections 2.2(a) 1
Sections 6.1 1
Section 2.1(b) 1
Section 2.2(b) 1
Security 1
See 3
Short 1
Snappy 1
Snappy: 1
Software 79
Software, 21
Software. 13
Software. 2
Software means 2
Software” 2
Source 28
Spring 1
SpryMedia 1
State 1
States 3
Stephen 1
Steward 3
Steward. 1
Streaming 1
StringUtils.containsWhitespace()) 1
Subject 4
Submission 1
Sun 2
Sundstrom 1
T. 1
TERMINATION. 1
TERMINATION. 1
TERMS 3
THAT 2
THE 111
THEORY 12
THEORY, 2
THIS 38
TITLE, 2
TO 15
TO, 22
TORT 19
Tatu 2
Technology 1
Technology. 1
Terms. 2
That 1
The 76
Therefore, 1
These 1
This 54
Tim 1
To 1
Tomcat 1
Tools 1
Trademarks. 1
Twitter. 1
TypeUtil.java 1
U.S. 7
UCL 1
UNDER 9
URIs 1
USE 19
USE, 13
USERS. 2
Under 1
United 5
University 1
Unix 1
UnixCrypt 1
UnixCrypt.java 1
Unless 4
Unrestricted 1
Use 1
Users 2
V3 1
Van 1
Version 10
Version) 2
Version, 2
Version; 2
Versions 2
Versions. 8
Version means 1
Version” 1
Vyukov's 1
W3C 1
WARRANTIES 23
WARRANTIES, 11
WARRANTY 10
WARRANTY. 2
WAY 12
WHETHER 19
WITH 7
WITHOUT 17
WORK 2
Wakart's 1
Warranty 1
Warranty. 1
We 1
Web 2
WebSocket 1
When 3
While 2
William 1
Williams 1
Work 24
Work, 4
Work. 1
Work; 4
Works 12
Works" 1
Works, 2
Works. 2
Works; 3
Work means 1
Work” 1
Wysoker 1
X, 1
X. 1
X.509 1
XML 4
XML', 1
Xerces 2
Xerces2 1
YOU 2
YOU, 2
YOU. 3
YOU. 1
Yann 1
York 1
Yoshida, 1
You 125
You. 2
You: 2
Your 27
Your) means 1
Yuta 1
[name 1
[yyyy] 1
``AS 1
a 157
a) 5
about 1
above 28
above, 8
above. 1
absence 2
absolutely 4
accept 1
acceptance 1
accepting 2
accordance 1
according 1
account 1
acquire 3
acquired, 4
act 1
acting 2
action 3
actions 1
actions) 2
acts 1
acts) 1
actual 1
add 2
added 2
addendum 1
addition 3
addition, 1
additional 4
additions 4
addresses 2
administration 2
admission 2
advised 1
affect 1
affiliates. 1
after 3
against 14
against, 1
agree 10
agreed 4
agreement 5
agreement, 2
agreement: 1
agrees 2
algorithm 2
algorithms. 1
all 31
alleged 1
alleging 5
allow 2
allowed 1
allows 1
alone 7
alone, 4
alone. 1
along 1
alongside 1
alphabet 1
also 5
alter 5
alternative 1
alters 2
although 1
always 3
amount 1
an 22
and 281
and/or 37
and 2
annotations, 1
another 1
any 129
any), 2
any, 6
any. 1
anyone 2
apache-hadoop-common 1
apparatus 2
appear. 1
appears 1
applicable 10
application 2
applies 3
apply 7
appropriate 1
appropriateness 2
archives. 1
are 70
arising 2
arising, 2
arose. 1
array 1
artifacts 10
as 54
assert 5
asserted 1
assign 1
associated 9
assume 1
assumes 2
assurances 1
asymmetric 1
at 17
at: 26
attach 1
attached 1
attempt 2
attorneys 1
attorneys' 1
attribution 6
author 1
authorized 2
authors 1
authors. 1
authorship, 2
authorship. 1
automatically 4
available 32
available. 2
avoid 1
aware 3
b) 5
based 5
basis, 2
basis. 2
be 73
becoming 3
been 6
before 1
behalf 11
behalf, 2
behalf. 1
being 2
believe 2
below 4
below). 1
below: 1
benchmark 1
benchmarking 1
beneficial 3
between 2
beyond 2
binaries 12
binary 42
bind 1
body 1
boilerplate 1
both 3
brackets 1
brackets!) 1
breach 2
breach. 2
bring 1
brought 2
bundle 1
bundles 20
business 1
but 7
by 133
by, 5
by: 4
bzip2 1
c) 1
calculation 1
can 27
cannot 1
carry 1
case 5
case, 2
cause 5
caused 5
causes 1
cease 1
certain 1
certificate 1
changed 1
changes 4
character 1
charge 3
charge, 5
check 1
choice, 2
choose 6
claim 9
claim(s), 2
claim, 1
claims 10
claims, 8
class 1
classes 1
classes. 1
classes: 1
classified 1
classpath 3
clause 4
clear 6
code 39
code, 2
code. 3
code. 1
combination 15
combination), 2
combination). 1
combination). 1
combinations 2
combined 1
combines 2
combining 2
comment 1
commercial 10
common 5
communication 3
compiled 1
compiler 1
complete 2
compliance 11
complies 2
comply 4
complying 2
components 1
compression 6
computer 11
concerning 3
conclusions 1
condition 1
conditions 45
conditions, 1
conditions. 3
conditions: 6
configuration 1
conflict-of-law 2
conjunction 1
connection 1
connections, 1
consequential 2
consistent 1
consisting 2
consists 2
conspicuously 1
constant-size 1
constitute 2
constitutes 1
construct 1
construed 3
contact 1
contain 2
containd 1
contained 9
containing 6
contains 19
content 2
contents 3
continue 3
contract 6
contract, 1
contribute 2
contributed 2
contributes 2
contribution 1
contributions 3
contributors 8
contributors, 1
contributory 1
control 7
control, 2
controlled 3
controls, 2
conversions 1
conveyed 4
cooperate 1
copies 13
copies./ 1
copy 16
copy, 7
copying 1
copyright 46
copyright, 3
copyright:Copyright 1
copyrighted 3
corresponding 1
costs 4
costs, 2
counterclaim 2
countries) 2
country 1
country's 1
country, 1
court 3
courts 2
covered 2
create 9
created 2
creates 2
creation 2
creation(s) 2
cross-claim 2
cryptographic 3
cryptography 1
cure 3
currently 1
customarily 3
customary 1
d) 1
damage 1
damages 7
damages, 3
damages. 2
data 3
data, 1
date 7
day 4
days 4
de 1
deal 5
declaratory 2
decoding 1
decompression 4
deemed 2
defend 2
defend, 1
defense 1
defined 5
definition, 4
delete 2
deleted 2
deletion 2
deliberate 1
den 1
dependencies 8
depends 13
derivative 5
derived 10
derived.) 1
describing 1
description 1
descriptive 2
designated 1
details 1
details, 1
determining 3
developed 10
devices. 1
devices. 1
devices; 2
differ 6
different 5
differs 2
direct 4
direct, 2
direction 3
directly 7
directories 1
directory 1
disclaimer 11
disclaimer. 11
disclaimers, 1
disclaims 2
discussing 1
display 1
display, 6
dispose 4
distinguishing 3
distribute 34
distribute, 7
distributed 15
distributes 5
distributing 2
distribution 29
distribution, 1
distribution. 12
distributor 2
distributor) 2
distributors 1
do 13
document. 2
document. 1
documentation 23
documentation, 3
documentation” 1
documented 1
does 7
downloaded 1
drafter 2
dual 1
each 21
easier 1
ecj-4.3.1.jar 1
editorial 1
effect 2
effective 4
effectively 2
either 11
elaborations, 1
elect 1
electronic 1
electronic, 1
eligible 1
enclosed 2
encoding 1
encryption 3
end 3
endorse 8
enforceability 1
enforceable. 3
entities 1
entities, 2
entity 14
entity, 3
entity. 5
entity. 1
equipment, 1
equitable 2
equivalent 1
errors, 1
estoppel 1
even 1
event 4
every 7
exact 1
example 1
example, 2
except 5
exception 1
exception. 2
exceptions. 1
exchange. 2
exchange. 1
excluded. 2
excludes 1
excluding 5
executed 1
exercise 2
exercising 4
exist, 1
expense. 1
expenses. 2
expiration 2
explicitly 1
export 5
express 4
expressed 1
expressly 4
expressly, 1
extensions 1
extent 8
external 1
externally 1
facade 1
facilitate 1
fail 4
fails 1
failure 2
fee 5
fees 2
fields 1
fifty 3
file 20
file, 3
file. 2
filed. 2
files 11
files, 1
files. 1
files; 1
first 6
fitness 1
following 77
following: 4
for 74
for, 3
form 51
form, 4
form. 3
form: 1
format, 2
format. 1
forms, 11
forth 6
forum 1
found 1
framework, 3
free 5
from 51
from) 1
from, 1
fulfilled 2
full 1
functionality. 1
functions 1
furnished 5
further 1
gcc, 1
generate 1
generated 3
generating 1
generic 1
github 1
give 1
given 3
giving 2
goodwill, 1
governed 10
governing 2
grant 6
grant, 2
granted 20
granted, 6
granted: 4
granting 1
grantor. 1
grantor. 1
grants 9
grossly 1
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/util/tree.h 1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.0.2 1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-full-2.0.0.min.js 1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-helpers-1.1.1.min.js 1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery-1.10.2.min.js 1
hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css 1
hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css 1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js 1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE 1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js 1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js 1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/ 1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery 1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js.gz 1
hardware 1
hardware) 1
harmless 1
has 20
have 14
having 2
held 2
hereafter 2
hereby 20
herein 5
herein, 1
herein. 3
herein. 1
hereof. 2
hereto, 1
hereunder 4
hereunder, 1
hereunder. 3
history 1
history, 1
hold 1
hosted 1
how 3
http://archive.apache.org/dist/harmony/ 1
http://asm.ow2.org/license.html 1
http://aspell.net/test/orig/batch0.tab.Copyright 1
http://code.google.com/p/caliper/ 1
http://code.google.com/p/lz4/ 1
http://code.google.com/p/protobuf/ 1
http://code.google.com/p/snappy/ 2
http://commons.apache.org/logging/ 1
http://fusesource.com 2
http://gcc.gnu.org/onlinedocs/libstdc++/manual/license.html 1
http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/ 1
http://hadoop.apache.org/ 1
http://hadoop.apache.org/core/ 1
http://iharder.sourceforge.net/current/java/base64/ 1
http://logging.apache.org/log4j/ 1
http://netty.io/ 1
http://openjdk.java.net/legal/gplv2+ce.html 1
http://stevemorse.org/phoneticinfo.htm 1
http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/ 1
http://wiki.apache.org/hadoop/ 1
http://wiki.fasterxml.com/AaltoHome 1
http://www.apache.org/licenses/ 1
http://www.apache.org/licenses/LICENSE-2.0 2
http://www.apache.org/licenses/LICENSE-2.0.html 1
http://www.bouncycastle.org/ 1
http://www.eclipse.org/legal/epl-v10.html 1
http://www.ibm.com. 1
http://www.jboss.org/jbossmarshalling 1
http://www.jcraft.com/jzlib/ 1
http://www.slf4j.org/ 1
http://www.sun.com. 1
https://code.google.com/p/jbzip2/ 1
https://code.google.com/p/jfastlz/ 1
https://code.google.com/p/libdivsufsort/ 1
https://commons.apache.org/proper/commons-lang/ 1
https://github.com/JCTools/JCTools 1
https://github.com/akka/akka/blob/wip-2.2.3-for-scala-2.11/akka-actor/src/main/java/akka/dispatch/AbstractNodeQueue.java 1
https://github.com/joewalnes/webbit 1
https://github.com/jponge/lzma-java 1
https://github.com/jpountz/lz4-java 1
https://github.com/jquery/jquery 1
https://github.com/ning/compress 1
https://github.com/twitter/hpack 1
https://glassfish.dev.java.net/nonav/public/CDDL+GPL.html 2
https://groups.google.com/forum/#!forum/lz4c 1
https://jquery.org/ 1
i) 2
iClick, 1
identification 1
identifies 2
identify 2
identifying 1
if 23
if, 1
ii) 2
iii) 1
implementation 2
implements 1
implication, 1
implied 1
implied, 3
implied. 2
import 1
import, 3
impose 2
improvement 1
improving 1
in 171
in, 1
inability 1
incidental 1
incidental, 1
include 11
included 13
includes 17
including 20
including, 3
inclusion 3
inconsistency 1
incorporated 2
incurred 5
indemnify 5
indemnify, 1
indemnity 6
indemnity, 1
indicated 1
indirect, 5
indirectly 5
indirectly, 2
individual 9
individuals 1
individuals. 1
inform 2
information 2
information. 2
information: 1
informational 1
informs 1
infringe 1
infringed 8
infringement 5
infringement, 1
infringement. 1
infringements 4
infringes 4
initial 7
initiation 1
input 2
institute 1
institutes 1
intellectual 15
intended 3
intentionally 2
interchange 1
interfaces 1
interpreted 1
interruption 1
into 1
invalid 1
irrevocable 2
is 100
issue 1
it 20
it. 2
item, 1
item,” 1
its 31
itself 4
iv) 1
jQuery 1
jars, 1
java.lang 1
java.sun.security.ssl 1
javax.annotation:javax.annotation-api 1
javax.servlet:javax.servlet-api 1
javax.transaction:javax.transaction-api 1
javax.websocket:javax.websocket-api 1
jsp-api 1
judgment 2
jurisdiction 6
jurisdiction's 1
jurisdictions 1
jury 1
knowledge 1
language 4
latest 1
law 8
law, 3
laws 4
laws, 2
lawsuit) 2
lawsuits 1
least 1
legal 7
leveldb 1
leveldbjni 1
leveldbjni-all 1
liability 15
liability. 2
liability. 1
liable 1
libraries 2
library 10
library, 1
libstdc++. 1
licensable 2
license 46
license) 1
license, 2
license. 4
license/LICENSE.aalto-xml.txt 1
license/LICENSE.abstractnodequeue.txt 1
license/LICENSE.base64.txt 1
license/LICENSE.bouncycastle.txt 1
license/LICENSE.caliper.txt 1
license/LICENSE.commons-lang.txt 1
license/LICENSE.commons-logging.txt 1
license/LICENSE.compress-lzf.txt 1
license/LICENSE.deque.txt 1
license/LICENSE.harmony.txt 1
license/LICENSE.hpack.txt 1
license/LICENSE.jboss-marshalling.txt 1
license/LICENSE.jbzip2.txt 1
license/LICENSE.jctools.txt 1
license/LICENSE.jfastlz.txt 1
license/LICENSE.jsr166y.txt 1
license/LICENSE.jzlib.txt 1
license/LICENSE.libdivsufsort.txt 1
license/LICENSE.log4j.txt 1
license/LICENSE.lz4.txt 1
license/LICENSE.lzma-java.txt 1
license/LICENSE.protobuf.txt 1
license/LICENSE.slf4j.txt 1
license/LICENSE.snappy.txt 1
license/LICENSE.webbit.txt 1
license: 13
license: 2
licensed 3
licensed. 3
licensee) 1
licensees 1
licenses 15
licenses. 1
licenses; 1
licensing 2
lieu 2
like. 1
limit 2
limitation 5
limitation, 5
limitations 2
limited 5
link 1
linked 2
list 22
lists, 1
litigation 6
litigation, 1
litigation. 1
located 6
location 2
logging 3
losing 2
loss 2
losses), 1
losses, 1
lost 1
made 26
made, 5
made. 1
mailing 1
maintained 1
make 23
make, 6
makes 8
making 1
making, 4
malfunction, 1
managed 1
management 3
manner 6
manner. 1
many 2
marked 1
marks, 1
material 1
materials 11
matter 2
maximum 2
may 53
mean 11
means 20
means: 1
mechanical 1
media 1
medium 3
medium, 1
meet 1
memory 1
merchantability 1
merely 1
merge, 5
met: 11
method, 2
micro- 1
might 1
minimum 1
modification 4
modification, 12
modifications 8
modifications, 3
modifications: 1
modified 19
modififications 1
modify 6
modify, 10
modifying 1
module 1
modules 1
more 9
mortbay.org. 1
must 50
must: 1
name 11
name) 1
names 9
names, 1
native 1
nature, 2
necessarily 2
necessary 3
needed, 1
negligence), 1
negligent 1
negotiate 1
negotiations. 1
new 12
no 10
no-charge, 2
node_modules 1
non-blocking 1
non-commercial 1
non-exclusive 4
non-exclusive, 4
non-infringement, 1
non-intrusive 1
noncompliance. 1
none 1
nor 8
normally 1
not 45
note 2
noted. 1
nothing 1
notice 25
notice, 23
notices 14
notices, 1
notices: 9
notify 1
now 2
number. 3
object 5
obligation 2
obligations 5
obligations, 1
obtain 5
obtained 23
obtaining 5
obtaining, 1
of 445
of, 8
of: 2
offer 8
offer, 3
offer. 3
offer. 1
offered 5
offering 1
offering, 2
offering. 1
offers 1
official 1
omissions 1
on 56
on. 1
one 8
only 13
open 1
operations. 1
optionally 12
or 276
or, 1
order 2
org.apache.commons.codec.language.bm 1
org.apache.hadoop.util.bloom.* 1
org.apache.taglibs:taglibs-standard-impl 1
org.apache.taglibs:taglibs-standard-spec 1
org.apache.tomcat:tomcat-api 1
org.apache.tomcat:tomcat-el-api 2
org.apache.tomcat:tomcat-jasper 1
org.apache.tomcat:tomcat-jasper-el 2
org.apache.tomcat:tomcat-jsp-api 1
org.apache.tomcat:tomcat-juli 1
org.apache.tomcat:tomcat-util 1
org.apache.tomcat:tomcat-util-scan 1
org.eclipse.jetty.orbit:javax.mail.glassfish 1
org.eclipse.jetty.orbit:javax.security.auth.message 1
org.eclipse.jetty.orbit:org.eclipse.jdt.core 1
org.eclipse.jetty.toolchain:jetty-schemas 1
org.fusesource.hawtjni:hawtjni-runtime 1
org.iq80.leveldb:leveldb-api 1
org.mortbay.jasper:apache-el 1
org.mortbay.jasper:apache-jsp 1
org.ow2.asm:asm 1
org.ow2.asm:asm-commons 1
origin 1
original 7
originally 9
originate 1
originator 2
other 52
others. 1
otherwise 28
otherwise), 2
otherwise, 5
otherwise. 2
our 2
out 3
outstanding 3
own 10
owned 3
owner 5
owner. 1
owner] 1
ownership 5
package 1
page" 1
part 12
participate 1
particular 4
parties 1
partners 1
parts 2
party 16
party. 2
party; 1
password 1
patent 31
patent(s), 1
patent, 3
pay 2
payment 1
per 1
percent 3
perform, 6
performance 4
performing 1
period 3
period, 2
permission 7
permission. 8
permissions 4
permit 5
permitted 12
permitted) 1
permitted. 1
perpetual, 2
person 6
persons 5
pertain 2
php 1
places: 1
please 3
policies 1
policies, 1
port 1
portion 10
portions 23
possession, 2
possibility 1
possible, 2
potential 1
power, 3
practicable. 1
practice, 2
pre-existing 1
preferred 1
preferred, 1
prepare 2
previous 4
prior 13
process, 2
processor, 1
produced 1
product 59
product. 2
products 8
profits; 1
program 1
programs 1
prohibiting 2
project 3
prominent 1
promote 8
promptly 1
property 15
prospectively 2
protection. 1
provide 3
provided 35
provides 8
provision 9
provisions 1
provisions. 2
public 1
publicly 4
publish 3
publish, 5
published 2
published, 1
pure 1
purpose 3
purpose; 1
purposes 7
pursuant 2
qualify, 1
queue. 1
re-export 2
re-implementation 1
read 1
read, 1
readable 1
reason 1
reasonable 8
reasonably 2
received 5
received. 1
receives 2
recipient's 1
recipients 7
recipients' 1
recommend 3
redistributed 2
redistributing 2
refer 1
references 2
referred 2
reformed 3
regarding 1
regulation 4
regulations 3
related 3
relating 4
released 2
remain 3
remainder 1
remove 5
rename 1
replace/modify 1
replaced 1
repository 1
represent 2
represent, 1
representatives, 1
representing 1
represents 3
reproduce 12
reproduce, 6
reproducing 1
reproduction, 3
required 6
requirements 2
requires 2
reserved. 10
reserves 1
reside 1
resolved 1
respect 3
responsibilities 1
responsibility 6
responsibility, 1
responsible 8
restriction, 5
restrictions 1
restricts 2
result 5
result, 1
resulting 2
results 2
retain 11
retain, 1
revised 2
revision 1
revisions) 1
revisions, 1
right 6
rights 55
risks 3
royalty 1
royalty-free 2
royalty-free, 6
sale 1
sale, 4
same 1
se 1
section 2
section) 1
section, 1
secure 1
security 1
see 2
see: 1
selected 1
selected. 1
self-signed 1
sell 5
sell, 8
selling 4
sent 1
separable 1
separate 4
serialization 1
serve 1
server 1
server, 1
service 1
servlet-api 1
set 6
settlement 1
settlement) 1
shall 44
shall, 2
shares 2
shares, 1
should 4
simple 2
single 2
site 1
slicing-by-8 1
snappy 1
so 3
so, 5
software 56
software, 2
software. 2
software: 1
software” 1
sole 2
solely 2
soon 1
source 35
source, 1
special, 2
specific 10
specified 4
src/google/protobuf/stubs/atomicops_internals_generic_gcc.h. 1
src/google/protobuf/stubs/atomicops_internals_power.h. 1
src/main/native/src/org/apache/hadoop/io/compress/lz4/{lz4.h,lz4.c,lz4hc.h,lz4hc.c}, 1
src/main/native/src/org/apache/hadoop/util: 1
src/test/org/apache/commons/codec/language/DoubleMetaphoneTest.javacontains 1
standalone 1
state 1
stated 3
statement 1
states 2
statically 1
stating 1
steward 6
steward. 1
steward. 1
stoppage, 1
string 2
style 1
subcomponents 2
subject 15
sublicense 5
sublicense, 6
submit 1
submitted 2
submitted. 1
subsequent 6
subsequently 3
subsidiaries 1
substantial 5
such 87
sufficient 3
suffix 1
suitable 1
suite 1
supersede 1
supersedes, 2
support 4
support, 7
sure 2
survive 2
survive. 2
survive. 1
syntax 1
systems 2
systems, 1
taken 1
temporary 1
term 4
terminate 7
terminate, 1
termination 6
termination. 1
termination. 1
terms 61
terms. 1
test 2
text 8
than 12
that 118
that: 1
the 693
their 7
them, 1
then 9
theory, 1
thereof 3
thereof), 4
thereof). 1
thereof); 3
thereof, 3
thereof. 2
thereof. 1
these 9
they 2
third 11
third-party 2
this 181
those 9
through 6
time 7
time. 3
title 1
to 253
to, 2
tort 1
tracking 1
trade 1
trademark 2
trademark) 4
trademark, 1
trademarks, 1
transfer 2
transformation 1
transformed 1
translated 1
translation 1
trial 1
types. 1
ultra-high 1
unavailability 1
under 94
under, 2
understands 1
understood, 1
unenforceable 1
unenforceable, 2
unilaterally 2
union 1
unless 4
unmodified 2
upon 6
usage 1
use 23
use, 27
used 19
user 2
users 1
users, 1
uses 1
using 7
using, 2
utilities 1
utilization 2
v 1
v2 1
valid 1
validity 1
validly 2
value 2
venue 2
verbal, 1
version 25
version. 1
versions 3
viewable 1
views 1
visit 2
voluntary 3
waives 1
want 2
warranties 6
warranties, 1
warranty 1
warranty, 7
was 3
way 1
we 1
web 1
website 1
were 3
when 6
where 7
wherever 1
whether 9
which 61
which, 2
which: 1
who 2
whole, 2
whom 8
wiki, 1
will 7
with 103
withdraw 2
within 20
without 42
work 11
work, 3
work. 2
works 4
works, 1
world-wide, 4
worldwide, 4
would 1
writing 2
writing, 4
written 19
xmlenc 1
year 1
you 12
your 5
zlib 1
252.227-7014(a)(1)) 1
§ 1
“AS 1
“Contributor 1
“Contributor” 1
“Covered 1
“Executable” 1
“Initial 1
“Larger 1
“Licensable” 1
“License” 1
“Modifications” 1
“Original 1
“Participant”) 1
“Patent 1
“Source 1
“Your”) 1
“You” 2
“commercial 3
“control” 1
原文地址：http://blog.51cto.com/13558754/2067497
时间： 2024-11-10 01:13:09