PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)

目录：

　　一、Thrift1和Thrift2的简要介绍

　　　　1) 写在前面

　　　　2) Thrift1和Thrift2的区别 

　　二、Thrift0.9.2的安装

　　　　1) 安装依赖插件

　　　　2) Thrift0.9.2的编译 

　　　　3) Thrift0.9.2编译后，配置成可执行文件

　　三、Hbase0.96.2的安装

　　　　1) Hbase的介绍

　　　　2) Hbase的安装

　　　　3) Hbase基于源码的Thrift生成接口文件

　　四、PHP和Golang使用Thrift1和Thrift2访问Hbase

　　　　1) PHP基于Thrift1访问Hbase

　　　　2) Golang基于Thrift2访问Hbase

　　五、常见问题

　　六、扩展阅读

 

目录：

　　一、Thrift1和Thrift2的简要介绍

　　　　1) 写在前面

　　从Hbase0.94.11开始有两套thrift接口（可以叫Thrift1和Thrift2），根据官方文档，Thrift1很可能被抛弃，但网上的文章基本是介绍Thrift1的，本文则兼容介绍Thrift1和Thrift2，目前在网上是比较详细的介绍Thrift2文章了。

　　Thrift2相比较Thrift1做了简化和合成，但不提供查询所有Table、创建Table、删除Table的功能了。日常在工作中应该也很少会在Hbase中使用Thrift来创建Table。

　　从Thrift0.9.1开始支持golang语言，相关使用和demo可以参考这里《Golang、Php、Python、Java基于Thrift0.9.1实现跨语言调用》，不过由于写本文时Thrift0.9.1中对Golang访问Hbase支持的并不友好，在0.9.2中的Thrift2已经进行了改进可以很好的支持Golang访问Hbase。所以后面在使用Golang访问Hbase时，使用的是Thrift0.9.2。

　　网上关于Thrift1的文章一搜索有很多，也就不在这里多描述，在后面的演示中Thrift2中90%的方法都会有，不过在实际使用中可能还需要根据你的系统要求进行二次封装 。

　　　　2) Thrift1和Thrift2的区别

　　现在官网也没有太详细的介绍，下面的对比也是从IDL描述中找到的。这是一位网友整理的，后面的扩展阅读中链接了他的文章，更多的介绍可以去那里看。


Thrift
Thrift2

结构
struct TCell

struct ColumnDescriptor

struct TRegionInfo

struct Mutation

struct BatchMutation

struct TIncrement

struct TColumn

struct TRowResult

struct TScan
struct TTimeRange

struct TColumn

struct TColumnValue

struct TColumnIncrement

struct TResult

struct TGet

struct TPut

struct TDelete

struct TIncrement

struct TScan

struct TRowMutations

异常
exception IOError

exception IllegalArgument

exception AlreadyExists
exception TIOError

exception TIllegalArgument

其他
union TMutation

enum TDeleteType

enum TDurability

服务
名称为：Hbase

void enableTable()

void disableTable()

bool isTableEnabled()

void compact()

void majorCompact()

list<Text> getTableNames()

map<Text,ColumnDescriptor> getColumnDescriptors()

list<TRegionInfo> getTableRegions()

void createTable()

void deleteTable()

list<TCell> get()

list<TCell> getVer()

list<TCell> getVerTs()

list<TRowResult> getRow()

list<TRowResult> getRowWithColumns()

list<TRowResult> getRowTs()

list<TRowResult> getRowWithColumnsTs()

list<TRowResult> getRows()

list<TRowResult> getRowsWithColumns()

list<TRowResult> getRowsTs()

list<TRowResult> getRowsWithColumnsTs()

void mutateRow()

void mutateRowTs()

void mutateRows()

void mutateRowsTs()

i64 atomicIncrement()

void deleteAll()

void deleteAllTs()

void deleteAllRow()

void increment()

void incrementRows()

void deleteAllRowTs()

ScannerID scannerOpenWithScan()

ScannerID scannerOpen()

ScannerID scannerOpenWithStop()

ScannerID scannerOpenWithPrefix()

ScannerID scannerOpenTs()

ScannerID scannerOpenWithStopTs()

list<TRowResult> scannerGet()

list<TRowResult> scannerGetList()

void scannerClose()

list<TCell> getRowOrBefore()

TRegionInfo getRegionInfo()
名称为：THBaseService

bool exists(...)

TResult get(...)

list<TResult> getMultiple(...)

void put(...)

bool checkAndPut(...)

void putMultiple(...)

void deleteSingle(...)

list<TDelete> deleteMultiple(...)

bool checkAndDelete(...)

TResult increment(...)

i32 openScanner(...)

list<TResult> getScannerRows(...)

void closeScanner(...)

void mutateRow(...)

list<TResult> getScannerResults(...)

　　二、Thrift0.9.2的安装

　　　　1) 安装依赖插件

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

[email protected]:/home/hadoop#sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev

#下载thirft0.9.2版本

[email protected]:/home/hadoop# git clone https://github.com/apache/thrift.git thrift-git0.9.2

Cloning into ‘thrift-git0.9.2‘...

remote: Counting objects: 37221, done.

remote: Compressing objects: 100% (195/195), done.

remote: Total 37221 (delta 312), reused 407 (delta 272)

Receiving objects: 100% (37221/37221), 9.61 MiB | 74 KiB/s, done.

Resolving deltas: 100% (25830/25830), done.

[email protected]:/home/hadoop#cd thrift-git0.9.2

# 切换成相应所要用的分支，如果你编译通过，就不用像我现在这样切换了

[email protected]:/home/hadoop/thrift-git0.9.2# git checkout --track origin/0.9.2

Branch 0.9.2 set up to track remote branch 0.9.2 from origin.

Switched to a new branch ‘0.9.2’

　　　　2) Thrift0.9.2的编译

　　#编译Thrift0.9.2版本，注意这个版本和0.9.1编译方法不一样，详情可以参考/home/hadoop/thrift-git0.9.2/compiler/cpp/README.md中的说明

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

[email protected]:/home/hadoop/thrift-git0.9.2# mkdir -p compiler/cpp/build

[email protected]:/home/hadoop/thrift-git0.9.2# cd compiler/cpp/build

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# cmake ..

-- The C compiler identification is GNU

-- The CXX compiler identification is GNU

-- Check for working C compiler: /usr/bin/gcc

-- Check for working C compiler: /usr/bin/gcc -- works

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Check for working CXX compiler: /usr/bin/c++

-- Check for working CXX compiler: /usr/bin/c++ -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Thrift package version: 0.9.2

-- Thrift version: 0.9.2 (0.9.2)

-- Found FLEX: /usr/bin/flex (found version "2.5.35")

-- Found BISON: /usr/bin/bison (found version "2.5")

-- Configuring done

-- Generating done

-- Build files have been written to: /home/hadoop/thrift-git0.9.2/compiler/cpp/build

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# make

[  3%] [BISON][thrifty] Building parser with bison 2.5

[  6%] Generating thrifty.h

[  9%] [FLEX][thriftl] Building scanner with flex 2.5.35

Scanning dependencies of target libparse

[ 12%] Building CXX object CMakeFiles/libparse.dir/thrifty.cc.o

[ 15%] Building CXX object CMakeFiles/libparse.dir/thriftl.cc.o

Linking CXX static library liblibparse.a

[ 15%] Built target libparse

Scanning dependencies of target thrift

[ 18%] Building CXX object CMakeFiles/thrift.dir/src/main.cc.o

[ 21%] Building C object CMakeFiles/thrift.dir/src/md5.c.o

[ 25%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_generator.cc.o

[ 28%] Building CXX object CMakeFiles/thrift.dir/src/parse/t_typedef.cc.o

[ 31%] Building CXX object CMakeFiles/thrift.dir/src/parse/parse.cc.o

[ 34%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_c_glib_generator.cc.o

[ 37%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_cpp_generator.cc.o

[ 40%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_java_generator.cc.o

[ 43%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_as3_generator.cc.o

[ 46%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_csharp_generator.cc.o

[ 50%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_py_generator.cc.o

[ 53%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_rb_generator.cc.o

[ 56%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_perl_generator.cc.o

[ 59%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_php_generator.cc.o

[ 62%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_erl_generator.cc.o

[ 65%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_cocoa_generator.cc.o

[ 68%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_st_generator.cc.o

[ 71%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_ocaml_generator.cc.o

[ 75%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_hs_generator.cc.o

[ 78%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_xsd_generator.cc.o

[ 81%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_html_generator.cc.o

[ 84%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_js_generator.cc.o

[ 87%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_javame_generator.cc.o

[ 90%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_delphi_generator.cc.o

[ 93%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_go_generator.cc.o

[ 96%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_d_generator.cc.o

[100%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_lua_generator.cc.o

Linking CXX executable thrift

[100%] Built target thrift

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build#

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# ./thrift -version

Thrift version 0.9.2

　　　　3) Thrift0.9.2编译后，配置成可执行文件

　　#将编译后的Thrift0.9.2放到可执行程序目录，并命名为thrift092

?


1

2

3

4

5

6

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# whereis thrift

thrift: /usr/local/bin/thrift

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# cp thrift /usr/local/bin/thrift092

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# thrift092 -version

Thrift version 0.9.2

[email protected]:/home/hadoop/thrift-git0.9.2/compiler/cpp/build#

　　三、Hbase0.96.2的安装

　　　　1) Hbase的介绍

　　hbase的介绍，参考这里：http://baike.baidu.com/view/1993870.htm

　　　　2) Hbase的安装

　　请参考这篇文章：《ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式环境部署》

　　后面的测试，也是基于上面这篇文章的环境测试的。

　　　　3) Hbase基于源码的Thrift生成接口文件

　　#下载Hbase的源码包（后面会使用里面的Thrift1和Thrift2）

?


1

2

[email protected]:/home/hadoop# wget http://mirrors.hust.edu.cn/apache/hbase/hbase-0.96.2/hbase-0.96.2-src.tar.gz

[email protected]:/home/hadoop# tar xzvf hbase-0.96.2-src.tar.gz

　　#创建测试目录

?


1

2

[email protected]:/home/hadoop# mkdir thrift_hbase

[email protected]:/home/hadoop# cd thrift_hbase/

　　#生成Php基于Hbase的Thrift1的接口文件，由于Thrift0.9.1版本对Golang访问Hbase支持不够后，后面Golang访问Hbase用Thrift0.9.2来测试

?


1

2

3

4

5

6

7

[email protected]:/home/hadoop/thrift_hbase# thrift --gen php /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift

[email protected]:/home/hadoop/thrift_hbase# ll

总用量 16

drwxr-xr-x  4 root   root   4096 Aug 19 21:19 ./

drwxr-xr-x 47 hadoop hadoop 4096 Aug 19 21:15 ../

drwxr-xr-x  3 root   root   4096 Aug 19 21:19 gen-php/

[email protected]:/home/hadoop/thrift_hbase#

　　四、PHP和Golang使用Thrift1和Thrift2访问Hbase

　　　　1) PHP基于Thrift1访问Hbase

　　#将Php用到的Thrift1包复制到thrift_hbase里面

?


1

2

3

4

5

[email protected]:/home/hadoop/thrift_hbase# mkdir libphp                            

[email protected]:/home/hadoop/thrift_hbase# cp -r /home/hadoop/thrift-git/lib/php/src/* libphp/

[email protected]:/home/hadoop/thrift_hbase# cp -r /home/hadoop/thrift-git/lib/php/lib/Thrift libphp/

[email protected]:/home/hadoop/thrift_hbase# cp -r gen-php/Hbase libphp/

[email protected]:/home/hadoop/thrift_hbase# rm -rf gen-php/

　　#编写Hbase.php文件

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

<?php

/**

 * PHP基于Hbase0.96.2的Thrift1访问Hbase

 *

 * @author     迦壹

 * @copyright  Copyright (c) 2014 (http://idoall.org)

 * @license    GNU General Public License 2.0

 * @version    1

 */

 $startTime = getMillisecond();

$GLOBALS[‘THRIFT_ROOT‘] = ‘./libphp‘;   # 指定库目录，可以是绝对路径或是相对路径

require_once $GLOBALS[‘THRIFT_ROOT‘].‘/Thrift/ClassLoader/ThriftClassLoader.php‘;

 

use Thrift\ClassLoader\ThriftClassLoader;

use Thrift\Type\TMessageType;

use Thrift\Type\TType;

use Thrift\Exception\TException;

use Thrift\Factory\TStringFuncFactory;

use Thrift\StringFunc\TStringFunc;

use Thrift\StringFunc\Core;

use Thrift\Protocol\TBinaryProtocol;

use Thrift\Transport\TSocket;

use Thrift\Transport\TSocketPool;

use Thrift\Transport\TBufferedTransport;

use Hbase\HbaseClient;

use Hbase\ColumnDescriptor;

use Hbase\Mutation;

$loader = new ThriftClassLoader();

$loader->registerNamespace(‘Thrift‘, $GLOBALS[‘THRIFT_ROOT‘]); # 加载thrift

$loader->register();

require_once( $GLOBALS[‘THRIFT_ROOT‘].‘/Hbase/Hbase.php‘ );

require_once( $GLOBALS[‘THRIFT_ROOT‘].‘/Hbase/Types.php‘ );

 

$socket = new TSocket( ‘m1‘, 9090 );

$socket->setSendTimeout( 10000 ); // Ten seconds (too long for production, but this is just a demo ;)

$socket->setRecvTimeout( 20000 ); // Twenty seconds

$transport = new TBufferedTransport( $socket );

$protocol = new TBinaryProtocol( $transport );

$client = new HbaseClient( $protocol );

$transport->open();

$tmpendTime = getMillisecond();

echo "----建立连接用时:".$tmpendTime."-".$startTime."=".($tmpendTime-$startTime)."毫秒\n";

//测试表的名称

$tempTableName = "idoallorg_student";

$tmpstartTime = getMillisecond();

$tables = $client->getTableNames();

sort($tables);

echo "----列出所有的Table：----\n";

foreach ( $tables as $name ) {

  echo( "\tfound: {$name}\n" );

  if ( $name == $tempTableName ) {

    if ($client->isTableEnabled( $tempTableName )) {

      echo( "\t表已经存在，即将关闭: {$tempTableName}\n");

      $client->disableTable( $tempTableName );

    }

    echo( "\t删除表: {$tempTableName}\n" );

    $client->deleteTable( $tempTableName );

  }

}

$tmpendTime = getMillisecond();

echo "----列出所有Table用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

echo( "----创建表: {$tempTableName}----\n" );

$tmpstartTime = getMillisecond();

$columns = array(

  new ColumnDescriptor( array(

    ‘name‘ => ‘id‘,

    ‘maxVersions‘ => 10

  ) ),

  new ColumnDescriptor( array(

    ‘name‘ => ‘name:‘

  ) ),

  new ColumnDescriptor( array(

    ‘name‘ => ‘score:‘

  ) )

);

try {

  $client->createTable( $tempTableName, $columns );

} catch ( AlreadyExists $ae ) {

  echo( "WARN: {$ae->message}\n" );

}

$tmpendTime = getMillisecond();

echo "----创建表: {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

//读取创建后的表结构

//$tmpstartTime = getMillisecond();

echo( "----查看表结构： {$tempTableName}:----\n" );

$tmpstartTime = getMillisecond();

$descriptors = $client->getColumnDescriptors( $tempTableName );

asort( $descriptors );

foreach ( $descriptors as $col ) {

  echo( "\tcolumn: {$col->name}, maxVer: {$col->maxVersions}\n" );

}

$tmpendTime = getMillisecond();

echo "----查看表结构： {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

//

echo "----向表{$tempTableName} 中写入1条数据----\n";

$tmpstartTime = getMillisecond();

$row = ‘1‘;

$valid = "foobar-000000";

$valid1 = "foobar-11111111";

$mutations = array(

    new Mutation(array(

            ‘column‘ => ‘id:‘,

            ‘value‘ => 1,

        )),

    new Mutation(array(

            ‘column‘ => ‘score:a‘,

            ‘value‘ => $valid,

        )),

    new Mutation(array(

            ‘column‘ => ‘name:a‘,

            ‘value‘ => "idoall.org",

        )),

);

$client->mutateRow($tempTableName, $row, $mutations, null);

$tmpendTime = getMillisecond();

echo "----向表{$tempTableName} 中写入1条数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

//

echo "----获取刚写入的score列的数据->get----\n";

$tmpstartTime = getMillisecond();

$arr = $client->get($tempTableName, $row, "score:a", null);

foreach ($arr as $k => $v) {

    // $k = TCell

    echo "\t ------ get one : value = {$v->value}\n";

    echo "\t ------ get one : timestamp = {$v->timestamp}\n";

}

$tmpendTime = getMillisecond();

echo "----获取刚写入的 score列的数据->get用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

//

echo "----获取刚写入的数据->getRow----\n";

$tmpstartTime = getMillisecond();

$result = $client->getRow($tempTableName, $row, null);

foreach ($result as $key => $TRowResult) {

    echo "\trowkey:$TRowResult->row\tcolumns(array):\r\n";

    foreach ($TRowResult->columns as $key => $value) {

        echo "\t\t\tkey:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";

    }

}

$tmpendTime = getMillisecond();

echo "----获取刚写入的数据->getRow用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

//

echo "----测试更新数据----\n";

$tmpstartTime = getMillisecond();

try {

  $mutations = array(

    new Mutation( array(

      ‘column‘ => ‘score:a‘,

      ‘value‘ => $valid1

    ) ),

  );

  $client->mutateRow( $tempTableName, $row, $mutations, null );

} catch ( IOError $e ) {

  echo( "expected error: {$e->message}\n" );

}

$tmpendTime = getMillisecond();

echo "----测试更新数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

echo "----批量更新10条数据----\n";

$tmpstartTime = getMillisecond();

$mutationsbatch = array();

for($i=0;$i<10;$i++)

{

    $rowkey=$i+10;

    $mutations = array(

        new Mutation(array(

                ‘column‘ => ‘id:int‘,

                ‘value‘ => $i+10,

            )),

        new Mutation(array(

                ‘column‘ => ‘score:txt‘,

                ‘value‘ => "score".$i,

            )),

        new Mutation(array(

                ‘column‘ => ‘name:txt‘,

                ‘value‘ => "idoall.org".$i,

            )),

    );

    array_push($mutationsbatch, new \Hbase\BatchMutation(array(‘row‘=>$rowkey, ‘mutations‘=>$mutations)));

}

$client->mutateRows($tempTableName, $mutationsbatch,null);

$tmpendTime = getMillisecond();

echo "----批量更新10条数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

echo "----scanner get scannerOpen-----\r\n";

$tmpstartTime = getMillisecond();

$startRow = 0;

$scan = $client->scannerOpen($tempTableName, $startRow, array (‘column‘ => ‘score‘, ‘column‘=>‘name‘), null);

$nbRows = 20;

$arr = $client->scannerGetList($scan, $nbRows);

echo ‘count of result :‘.count($arr)."\n";

foreach ($arr as $k => $TRowResult) {

    echo "\trow:$TRowResult->row\tcolumns(array):";

    foreach ($TRowResult->columns as $key => $value) {

        echo "key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";

    }

}

$client->scannerClose($scan);

$tmpendTime = getMillisecond();

echo "----scanner get scannerOpen用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

echo "----通过RowKey的范围获取数据 scannerOpenWithStop-----\r\n";

$tmpstartTime = getMillisecond();

$result = $client->scannerOpenWithStop($tempTableName , $startRow, $nbRows, array (‘column‘ => ‘score‘, ),null);

$recordArray = array();

while (true) {

    $record = $client->scannerGet($result);

    if ($record == NULL) {

        break;

    }

    foreach($record as $TRowResult) {

        echo "\trow:$TRowResult->row\tcolumns(array):";

        foreach ($TRowResult->columns as $key => $value) {

            echo "key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";

        }

    }

}

$tmpendTime = getMillisecond();

echo "----通过RowKey的范围获取数据 scannerOpenWithStop用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

echo "\r\n";

//删除表

echo "----删除表 {$tempTableName}----\n";

$tmpstartTime = getMillisecond();

try {

    $tmpstartTime_disableTable = getMillisecond();

    $client->disableTable( $tempTableName );

    $tmpendTime_disableTable = getMillisecond();

    echo "\t其中关闭表 {$tempTableName}用时:".$tmpendTime_disableTable."-".$tmpstartTime_disableTable."=".($tmpendTime_disableTable-$tmpstartTime_disableTable)."毫秒\n";

    $tmpstartTime_disableTable = getMillisecond();

    $client->deleteTable($tempTableName);

    $tmpendTime_disableTable = getMillisecond();

    echo "\t其中删除表 {$tempTableName}用时:".$tmpendTime_disableTable."-".$tmpstartTime_disableTable."=".($tmpendTime_disableTable-$tmpstartTime_disableTable)."毫秒\n";

} catch ( AlreadyExists $ae ) {

  echo( "WARN: {$ae->message}\n" );

}

$tmpendTime = getMillisecond();

echo "----删除表 {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";

$endTime = getMillisecond();

 

 echo "\r\n";

echo "PHP调用 总计 用时:".$endTime."-".$startTime."=".($endTime-$startTime)."毫秒\n";

 

function getMillisecond() {

list($t1, $t2) = explode(‘ ‘, microtime());    

return (float)sprintf(‘%.0f‘, (floatval($t1) + floatval($t2)) * 1000); 

}

$transport->close();

?>

　　#启动Hbase的Thrift，如果不了解，看下这篇文章吧，本文的环境都是基于这篇文章的《ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式环境部署》

?


1

[email protected]:/home/hadoop/thrift_hbase# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift

　　#运行Hbase.php，如果你也可以看到下面和我一样的提示，那么恭喜你也成功地使用PHP通过Thrift1访问到Hbase

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

[email protected]:/home/hadoop/thrift_hbase# php hbase.php

----建立连接用时:1408545184136-1408545184102=34毫秒

----列出所有的Table：----

        found: demo_table

        found: hbase2hive_idoall

        found: hive2hbase_idoall

        found: test_idoall_org

----列出所有Table用时:1408545184229-1408545184136=93毫秒

----创建表: idoallorg_student----

----创建表: idoallorg_student用时:1408545185026-1408545184229=797毫秒

----查看表结构： idoallorg_student:----

        column: id:, maxVer: 10

        column: name:, maxVer: 3

        column: score:, maxVer: 3

----查看表结构： idoallorg_student用时:1408545185047-1408545185026=21毫秒

----向表idoallorg_student 中写入1条数据----

----向表idoallorg_student 中写入1条数据用时:1408545185288-1408545185047=241毫秒

----获取刚写入的score列的数据->get----

         ------ get one : value = foobar-000000

         ------ get one : timestamp = 1408545184901

----获取刚写入的 score列的数据->get用时:1408545185373-1408545185289=84毫秒

----获取刚写入的数据->getRow----

        rowkey:1        columns(array):

                        key:id: value:1 timestamp:1408545184901

                        key:name:a      value:idoall.org        timestamp:1408545184901

                        key:score:a     value:foobar-000000     timestamp:1408545184901

----获取刚写入的数据->getRow用时:1408545185383-1408545185373=10毫秒

----测试更新数据----

----测试更新数据用时:1408545185389-1408545185384=5毫秒

----批量更新10条数据----

----批量更新10条数据用时:1408545185442-1408545185389=53毫秒

----scanner get scannerOpen-----

count of result :11

        row:1   columns(array):key:name:a       value:idoall.org        timestamp:1408545184901

        row:10  columns(array):key:name:txt     value:idoall.org0       timestamp:1408545185053

        row:11  columns(array):key:name:txt     value:idoall.org1       timestamp:1408545185053

        row:12  columns(array):key:name:txt     value:idoall.org2       timestamp:1408545185053

        row:13  columns(array):key:name:txt     value:idoall.org3       timestamp:1408545185053

        row:14  columns(array):key:name:txt     value:idoall.org4       timestamp:1408545185053

        row:15  columns(array):key:name:txt     value:idoall.org5       timestamp:1408545185053

        row:16  columns(array):key:name:txt     value:idoall.org6       timestamp:1408545185053

        row:17  columns(array):key:name:txt     value:idoall.org7       timestamp:1408545185053

        row:18  columns(array):key:name:txt     value:idoall.org8       timestamp:1408545185053

        row:19  columns(array):key:name:txt     value:idoall.org9       timestamp:1408545185053

----scanner get scannerOpen用时:1408545185484-1408545185443=41毫秒

----通过RowKey的范围获取数据 scannerOpenWithStop-----

        row:1   columns(array):key:score:a      value:foobar-11111111   timestamp:1408545185003

        row:10  columns(array):key:score:txt    value:score0    timestamp:1408545185053

        row:11  columns(array):key:score:txt    value:score1    timestamp:1408545185053

        row:12  columns(array):key:score:txt    value:score2    timestamp:1408545185053

        row:13  columns(array):key:score:txt    value:score3    timestamp:1408545185053

        row:14  columns(array):key:score:txt    value:score4    timestamp:1408545185053

        row:15  columns(array):key:score:txt    value:score5    timestamp:1408545185053

        row:16  columns(array):key:score:txt    value:score6    timestamp:1408545185053

        row:17  columns(array):key:score:txt    value:score7    timestamp:1408545185053

        row:18  columns(array):key:score:txt    value:score8    timestamp:1408545185053

        row:19  columns(array):key:score:txt    value:score9    timestamp:1408545185053

----通过RowKey的范围获取数据 scannerOpenWithStop用时:1408545185536-1408545185484=52毫秒

----删除表 idoallorg_student----

        其中关闭表 idoallorg_student用时:1408545186819-1408545185536=1283毫秒

        其中删除表 idoallorg_student用时:1408545186974-1408545186819=155毫秒

----删除表 idoallorg_student用时:1408545186974-1408545185536=1438毫秒

本次调用 总计 用时:1408545186974-1408545184102=2872毫秒

　　　　2) Golang基于Thrift2访问Hbase

　　#生成Golang基于Hbase的Thrift2的接口文件

?


1

[email protected]:/home/hadoop/thrift_hbase# thrift092 -r --gen go /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift2/hbase.thrift

　　#将Thrift2生成的开发库复制到GOPATH的src中

　　#删除掉0.9.1的文件，可能作者比较匆忙，在我使用0.9.2生成Thrift2的时候，还存在Thrift1的接口，这会对我们的测试带来干扰（后来我下载thrift1.0.0 dev时发现已经删除了这个文件，如果你生成IDL后的接口文件中，不包含这个文件，可以不用删除）

?


1

2

[email protected]:/home/hadoop/thrift_hbase# rm gen-go/hbase/t_h_base_service.go

[email protected]:/home/hadoop/thrift_hbase# cp -r gen-go/hbase $GOPATH/src

　　#编写hbase.go文件

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

//

//Golang基于Hbase0.96.2的Thrift2访问Hbase

//

//@author     迦壹

//@copyright  Copyright (c) 2014 (http://idoall.org)

//@license    GNU General Public License 2.0

//@version    1

package main

import (

    "encoding/binary"

    "fmt"

    "git.apache.org/thrift.git/lib/go/thrift"

    "hbase"

    "net"

    "os"

    "reflect"

    "strconv"

    "time"

)

const (

    HOST       = "m1"

    PORT       = "9090"

    TESTRECORD = 10

)

func main() {

    startTime := currentTimeMillis()

    logformatstr_ := "----%s\n"

    logformatstr := "----%s 用时:%d-%d=%d毫秒\n\n"

    logformattitle := "建立连接"

    rowkey := "1"

    temptable := "test_idoall_org"

    protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()

    transport, err := thrift.NewTSocket(net.JoinHostPort(HOST, PORT))

    if err != nil {

        fmt.Fprintln(os.Stderr, "error resolving address:", err)

        os.Exit(1)

    }

    client := hbase.NewTHBaseServiceClientFactory(transport, protocolFactory)

    if err := transport.Open(); err != nil {

        fmt.Fprintln(os.Stderr, "Error opening socket to "+HOST+":"+PORT, " ", err)

        os.Exit(1)

    }

    tmpendTime := currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, startTime, (tmpendTime - startTime))

    defer transport.Close()

    //--------------Exists

    logformattitle = "调用Exists方法"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime := currentTimeMillis()

    //

    isexists, err := (client.Exists([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))

    fmt.Printf("rowkey{%s} in table{%s} Exists:%t\t", rowkey, temptable, isexists)

    if err != nil {

        fmt.Printf("Exists err:%s\n", err)

    }

    fmt.Println("")

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //--------------put

    logformattitle = "调用Put方法写数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    cvarr := []*hbase.TColumnValue{

        &hbase.TColumnValue{

            Family:    []byte("name"),

            Qualifier: []byte("idoall.org"),

            Value:     []byte("welcome idoall.org")}}

    temptput := hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}

    err = client.Put([]byte(temptable), &temptput)

    if err != nil {

        fmt.Printf("Put err:%s\n", err)

    } else {

        fmt.Println("Put done")

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------Get---------------

    logformattitle = "调用Get方法获取新增加的数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    //

    result, err := (client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))

    if err != nil {

        fmt.Printf("Get err:%s\n", err)

    } else {

        fmt.Println("Rowkey:" + string(result.Row))

        for _, cv := range result.ColumnValues {

            printscruct(cv)

        }

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //--------------put update

    logformattitle = "调用Put update方法‘修改‘数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    cvarr = []*hbase.TColumnValue{

        &hbase.TColumnValue{

            Family:    []byte("name"),

            Qualifier: []byte("idoall.org"),

            Value:     []byte("welcome idoall.org---update")}}

    temptput = hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}

    err = client.Put([]byte(temptable), &temptput)

    if err != nil {

        fmt.Printf("Put update err:%s\n", err)

    } else {

        fmt.Println("Put update done")

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------Get  update---------------

    logformattitle = "调用Get方法获取‘修改‘后的数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    //

    result, err = (client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))

    if err != nil {

        fmt.Printf("Get update err:%s\n", err)

    } else {

        fmt.Println("update Rowkey:" + string(result.Row))

        for _, cv := range result.ColumnValues {

            printscruct(cv)

        }

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------DeleteSingle------------

    logformattitle = "调用DeleteSingle方法删除一条数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    tdelete := hbase.TDelete{Row: []byte(rowkey)}

    err = client.DeleteSingle([]byte(temptable), &tdelete)

    if err != nil {

        fmt.Printf("DeleteSingle err:%s\n", err)

    } else {

        fmt.Printf("DeleteSingel done\n")

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //-------------PutMultiple----------------

    logformattitle = "调用PutMultiple方法添加" + strconv.Itoa(TESTRECORD) + "条数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    var tputArr []*hbase.TPut

    for i := 0; i < TESTRECORD; i++ {

        putrowkey := strconv.Itoa(i)

        tputArr = append(tputArr, &hbase.TPut{

            Row: []byte(putrowkey),

            ColumnValues: []*hbase.TColumnValue{

                &hbase.TColumnValue{

                    Family:    []byte("name"),

                    Qualifier: []byte("idoall.org"),

                    Value:     []byte(time.Now().String())}}})

    }

    err = client.PutMultiple([]byte(temptable), tputArr)

    if err != nil {

        fmt.Printf("PutMultiple err:%s\n", err)

    } else {

        fmt.Printf("PutMultiple done\n")

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //------------------GetMultiple-----------------------------

    logformattitle = "调用GetMultiple方法获取" + strconv.Itoa(TESTRECORD) + "数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    //

    var tgets []*hbase.TGet

    for i := 0; i < TESTRECORD; i++ {

        putrowkey := strconv.Itoa(i)

        tgets = append(tgets, &hbase.TGet{

            Row: []byte(putrowkey)})

    }

    results, err := client.GetMultiple([]byte(temptable), tgets)

    if err != nil {

        fmt.Printf("GetMultiple err:%s", err)

    } else {

        fmt.Printf("GetMultiple Count:%d\n", len(results))

        for _, k := range results {

            fmt.Println("Rowkey:" + string(k.Row))

            for _, cv := range k.ColumnValues {

                printscruct(cv)

            }

        }

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //-------------------TMutation

    //TMutation包含一个TGet一个TPut，就不做测试了

    //可以和MutateRow结合使用

    //

    //-------------------OpenScanner

    logformattitle = "调用OpenScanner方法"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    startrow := make([]byte, 4)

    binary.LittleEndian.PutUint32(startrow, 1)

    stoprow := make([]byte, 4)

    binary.LittleEndian.PutUint32(stoprow, 10)

    scanresultnum, err := client.OpenScanner([]byte(temptable), &hbase.TScan{

        StartRow: startrow,

        StopRow:  stoprow,

        //     FilterString: []byte("RowFilter(=, ‘regexstring:00[1-3]00‘)"),

        //     FilterString: []byte("PrefixFilter(‘1407658495588-‘)"),

        Columns: []*hbase.TColumn{

            &hbase.TColumn{

                Family:    []byte("name"),

                Qualifier: []byte("idoall.org")}}})

    if err != nil {

        fmt.Printf("OpenScanner err:%s\n", err)

    } else {

        fmt.Printf("OpenScanner %d done\n", scanresultnum)

        scanresult, err := client.GetScannerRows(scanresultnum, 100)

        if err != nil {

            fmt.Printf("GetScannerRows err:%s\n", err)

        } else {

            fmt.Printf("GetScannerRows %d done\n", len(scanresult))

            for _, k := range scanresult {

                fmt.Println("scan Rowkey:" + string(k.Row))

                for _, cv := range k.ColumnValues {

                    printscruct(cv)

                }

            }

        }

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //--closescanner

    logformattitle = "调用CloseScanner方法"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    err = client.CloseScanner(scanresultnum)

    if err != nil {

        fmt.Printf("CloseScanner err:%s\n", err)

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //-------------------GetScannerResults

    logformattitle = "调用GetScannerResults方法"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis() //

    gsr, err := client.GetScannerResults([]byte(temptable), &hbase.TScan{

        StartRow: startrow,

        StopRow:  stoprow,

        //     FilterString: []byte("RowFilter(=, ‘regexstring:00[1-3]00‘)"),

        //     FilterString: []byte("PrefixFilter(‘1407658495588-‘)"),

        Columns: []*hbase.TColumn{

            &hbase.TColumn{

                Family:    []byte("name"),

                Qualifier: []byte("idoall.org")}}}, 100)

    if err != nil {

        fmt.Printf("GetScannerResults err:%s\n", err)

    } else {

        fmt.Printf("GetScannerResults %d done\n", len(gsr))

        for _, k := range gsr {

            fmt.Println("scan Rowkey:" + string(k.Row))

            for _, cv := range k.ColumnValues {

                printscruct(cv)

            }

        }

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    //---------------DeleteMultiple--------------

    logformattitle = "调用DeleteMultiple方法删除" + strconv.Itoa(TESTRECORD) + "数据"

    fmt.Printf(logformatstr_, logformattitle)

    tmpstartTime = currentTimeMillis()

    var tdelArr []*hbase.TDelete

    for i := 0; i < TESTRECORD; i++ {

        putrowkey := strconv.Itoa(i)

        tdelArr = append(tdelArr, &hbase.TDelete{

            Row: []byte(putrowkey)})

    }

    r, err := client.DeleteMultiple([]byte(temptable), tdelArr)

    if err != nil {

        fmt.Printf("DeleteMultiple err:%s\n", err)

    } else {

        fmt.Printf("DeleteMultiple %d done\n", TESTRECORD)

        fmt.Println(r)

    }

    tmpendTime = currentTimeMillis()

    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))

    endTime := currentTimeMillis()

    fmt.Printf("\nGolang调用总计用时:%d-%d=%d毫秒\n", endTime, startTime, (endTime - startTime))

}

func printscruct(cv interface{}) {

    switch reflect.ValueOf(cv).Interface().(type) {

    case *hbase.TColumnValue:

        s := reflect.ValueOf(cv).Elem()

        typeOfT := s.Type()

        //获取Thrift2中struct的field

        for i := 0; i < s.NumField(); i++ {

            f := s.Field(i)

            fileldformatstr := "\t%d: %s(%s)= %v\n"

            switch f.Interface().(type) {

            case []uint8:

                fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.Type(), string(f.Interface().([]uint8)))

            case *int64:

                var tempint64 int64

                if f.Interface().(*int64) == nil {

                    tempint64 = 0

                } else {

                    tempint64 = *f.Interface().(*int64)

                }

                fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.Type(), tempint64)

            default:

                fmt.Printf("I don‘t know")

            }

        }

    default:

        fmt.Printf("I don‘t know")

        fmt.Print(reflect.ValueOf(cv))

    }

}

func currentTimeMillis() int64 {

    return time.Now().UnixNano() / 1000000

}

　　#在hbase创建一个测试表“test_idoall_org”,用于使用Golang基于Thrift2的客户端来测试，因为Thrift2并没有提供创建和删除表的方法

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

[email protected]:/home/hadoop# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase shell

2014-07-27 09:31:07,601 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available

HBase Shell; enter ‘help<RETURN>‘ for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014

hbase(main):001:0> list

TABLE                                                                                                                                                                                                                 

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/home/hadoop/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

0 row(s) in 2.8030 seconds

=> []

hbase(main):002:0> version

0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014

hbase(main):003:0> status

2 servers, 0 dead, 1.0000 average load

hbase(main):004:0> create ‘test_idoall_org‘,‘uid‘,‘name‘

0 row(s) in 0.5800 seconds

=> Hbase::Table - test_idoall_org

hbase(main):005:0> list

TABLE                                                                                                                                                                                                                 

test_idoall_org                                                                                                                                                                                                       

1 row(s) in 0.0320 seconds

=> ["test_idoall_org"]

hbase(main):006:0> put ‘test_idoall_org‘,‘10086‘,‘name:idoall‘,‘idoallvalue‘

0 row(s) in 0.1090 seconds                 ^

hbase(main):009:0> get ‘test_idoall_org‘,‘10086‘

COLUMN                                                 CELL                                                                                                                                                           

 name:idoall                                           timestamp=1406424831473, value=idoallvalue                                                                                                                     

1 row(s) in 0.0450 seconds

hbase(main):010:0> scan ‘test_idoall_org‘

ROW                                                    COLUMN+CELL                                                                                                                                                    

 10086                                                 column=name:idoall, timestamp=1406424831473, value=idoallvalue                                                                                                 

1 row(s) in 0.0620 seconds

hbase(main):011:0>

　　#启动Hbase的Thrift2

?


1

2

3

4

5

6

7

8

9

10

11

12

13

[email protected]:/home/hadoop/thrift_hbase# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift2

#在这里可以看到thrift2的日志记录到下面的路径中

starting thrift2, logging to /home/hadoop/hbase-0.96.2-hadoop2/bin/../logs/hbase-root-thrift2-m1.out

[email protected]:/home/hadoop/thrift_hbase# jps

5705 Main

3614 ResourceManager

9426 HMaster

9944 Jps

3844 DFSZKFailoverController

9679 ThriftServer

2773 QuorumPeerMain

2944 JournalNode

[email protected]:/home/hadoop/thrift_hbase#

　　#运行基于Thrift2的Golang客户端，在之前要先确保hbase已经打开，在浏览器输入：http://m1:60010/master-status,可以看到下图效果，如果看不到，去调试下hbase吧，还没有启动~~~：

　　#运行hbase.go，如果你看到和我一样的提示信息，再次恭喜你我们的Golang也可以正常访问到Hbase了。

?


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

[email protected]:/home/hadoop/thrift_hbase# go run hbase.go

----建立连接 用时:1408639700869-1408639700867=2毫秒

----调用Exists方法

rowkey{1} in table{test_idoall_org} Exists:false

----调用Exists方法 用时:1408639700894-1408639700869=25毫秒

----调用Put方法写数据

Put done

----调用Put方法写数据 用时:1408639700905-1408639700894=11毫秒

----调用Get方法获取新增加的数据

Rowkey:1

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= welcome idoall.org

        3: Timestamp(*int64)= 1408639700843

----调用Get方法获取新增加的数据 用时:1408639700909-1408639700905=4毫秒

----调用Put update方法‘修改‘数据

Put update done

----调用Put update方法‘修改‘数据 用时:1408639700916-1408639700909=7毫秒

----调用Get方法获取‘修改‘后的数据

update Rowkey:1

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= welcome idoall.org---update

        3: Timestamp(*int64)= 1408639700858

----调用Get方法获取‘修改‘后的数据 用时:1408639700918-1408639700916=2毫秒

----调用DeleteSingle方法删除一条数据

DeleteSingel done

----调用DeleteSingle方法删除一条数据 用时:1408639700931-1408639700918=13毫秒

----调用PutMultiple方法添加10条数据

PutMultiple done

----调用PutMultiple方法添加10条数据 用时:1408639700941-1408639700931=10毫秒

----调用GetMultiple方法获取10数据

GetMultiple Count:10

Rowkey:0

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931138625 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:1

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931689108 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:2

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931691688 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:3

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931693392 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:4

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931694819 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:5

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931696588 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:6

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931698745 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:7

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931700279 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:8

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931702153 +0800 CST

        3: Timestamp(*int64)= 1408639700881

Rowkey:9

        0: Family([]uint8)= name

        1: Qualifier([]uint8)= idoall.org

        2: Value([]uint8)= 2014-08-22 00:48:20.931704203 +0800 CST

        3: Timestamp(*int64)= 1408639700881

----调用GetMultiple方法获取10数据 用时:1408639700950-1408639700941=9毫秒

----调用OpenScanner方法

OpenScanner 23 done

GetScannerRows 0 done

----调用OpenScanner方法 用时:1408639700956-1408639700950=6毫秒

----调用CloseScanner方法

----调用CloseScanner方法 用时:1408639700957-1408639700956=1毫秒

----调用GetScannerResults方法

GetScannerResults 0 done

----调用GetScannerResults方法 用时:1408639700961-1408639700958=3毫秒

----调用DeleteMultiple方法删除10数据

DeleteMultiple 10 done

[]

----调用DeleteMultiple方法删除10数据 用时:1408639700967-1408639700961=6毫秒

[email protected]:/home/hadoop/thrift_hbase#

五、常见问题

　　　　1) 并发访问线程数

首先，为了尽可能减少由于网络传输带来的时间开销，HBase的Thrift Server最好和应用客户端部署在同一台机器上。Thrift Server启动时可以通过参数配置并发线程数，否则很容易导致Thrift Server线程满了不响应客户端的读写请求，具体命令：/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase- daemon.sh start thrift –threadpool -m 200 -w 500（更多参数参考这里：/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift -h）。

　　　　2) 最大堆内存配置

如果客户端与Thrift Server进行scan操作顺序读取数据，而且设置了一定的cache记录条数（通过TScan的int32_t caching变量设置），那么这些被caching的记录数可能会占用Thrift Server相当部分的堆内存，尤其在多客户端并发访问时更明显。

　　因此，在Thrift Server启动前，可以调大最大堆内存，否则可 能由于java.lang.OutOfMemoryError异常而导致进程被杀掉，尤其是当Scan时设置了较大的caching记录条数的情况（默认为expor t HBASE_HEAPSIZE=1000MB，可以在/home/hadoop/hbase-0.96.2-hadoop2/hbase- env.sh中设置）。

　　六、扩展阅读

　　article/details/12163047">Thrift介绍与应用（三）—hbase的thrift接口

--------
-------------------------------

博文作者：迦壹

博客地址：:/idoall.org/home.php?mod=space&uid=1&do=blog&id=556">PHP和Golang使
用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)

转载声明：可以转载, 但必须以超链接形式标
明文章原始出处和作者信息及版权声明，谢谢合作！

---------------------------------------

Golang调用总计用时:1408639700967-1408639700867=100毫秒

PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04),布布扣,bubuko.com

时间： 2024-10-23 08:24:20

PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)的相关文章

JavaAPI访问Hadoop2.2HA的配置下访问Hbase0.96.2

1.确保Hadoop和Hbase服务已经正常启动了 2.把hbase-site.xml,core-site.xml,hdfs-site.xml配置文件放到Java工程的src目录下 3.引入相关的依赖包 4.Java Client测试访问Hbase集群 package com.hbase.test; import java.util.ArrayList; import java.util.List; import org.apache.hadoop.conf.Configuration; imp

PHP通过thrift2访问HBASE

前一段时间需要在网页上显示HBASE查询的结果,考虑用PHP来实现,在网上搜了一下,普遍都是用thrift作为接口来实现的.? 参考博文:? http://www.cnblogs.com/scotoma/archive/2013/05/16/3081236.html 用上述网址里提供的PHP代码,可以访问公司里的一个HBASE集群,但是另一个集群怎么也访问不了,上网查了一下,发现thrift有两套HBASE的接口--thrift和thrift2,而且两套接口并不兼容. ? 用thrift2的接口

通过usb访问mtp设备(ubuntu12.04) (转载)

转自:http://robert.penz.name/658/howto-access-mtp-devices-via-usb-on-ubuntu-12-04/ A friend asked me how he can access his Nexus 7 device via USB on his Ubuntu 12.04 notebook. With Android versions below 4.0 that was simple as the device registered as

golang gin解决跨域访问

package middleware import ( "github.com/gin-gonic/gin" "net/http") func Cors() gin.HandlerFunc { return func(c *gin.Context) { method := c.Request.Method c.Header("Access-Control-Allow-Origin", "*") c.Header("A

在Golang中使用C语言代码实例

转自:http://www.jb51.net/article/56720.htm cgo 使得在 Golang 中可以使用 C 代码. Hello World 为了有一个较为直观的了解,我们来看一个简单的例子,创建文件 main.go: package main /* #include <stdio.h> void sayHi() { printf("Hi"); } */ import "C" func main() { C.sayHi() } 执行程序

Golang教程：Map

什么是 map? Map 是 Go 中的内置类型,它将键与值绑定到一起.可以通过键获取相应的值. 如何创建 map? 可以通过将键和值的类型传递给内置函数 make 来创建一个 map.语法为:make(map[KeyType]ValueType).(译者注:map 的类型表示为 map[KeyType]ValueType)例如: personSalary := make(map[string]int) 上面的代码创建了一个名为 personSalary 的 map.其中键的类型为 string

GoLang基础数据类型---字典

Map 是 Go 中的内置类型,它将键与值绑定到一起.可以通过键获取相应的值. 如何创建 map? 可以通过将键和值的类型传递给内置函数 make 来创建一个 map.语法为:make(map[KeyType]ValueType).(译者注:map 的类型表示为 map[KeyType]ValueType)例如: personSalary := make(map[string]int) 上面的代码创建了一个名为 personSalary 的 map.其中键的类型为 string,值的类型为 in

electron/nodejs实现调用golang函数

https://www.jianshu.com/p/a3be0d206d4c 思路 golang 支持编译成c shared library, 也就是系统中常见的.so(windows下是dll)后缀的动态链接库文件. c++可以调用动态链接库,所以基本思路是golang开发主要功能, c++开发插件包装golang函数,实现中转调用 对于类型问题, 为了方便处理, 暴露的golang函数统一接受并返回字符串, 需要传的参数都经过json编码, 返回值亦然. 这里实现了3种调用方式, 同步调用,

远程主机访问sshd服务

##########功能:让远程主机可以通过网络访问sshd服务,开始一个安全shell ##########ssh 远程主机用户@远程主机ip[[email protected] ~]# ssh [email protected]The authenticity of host '172.25.0.11 (172.25.0.11)' can't be establishedA key fingerprint is eb:24:0e:07:96:26:b1:04:c2:37:0c:78:2d:b