PHP 0.9.0 operation HBase through thrift

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

Use thrift and PHP to read and write data in HBase to recent projects, so under the arrangement of related classes, do the next test.

Now I use the HBase operation mode mainly has the following several:

1.HBase Shell, Is the implementation of shell to view the data in HBase, through a command such as   configuration; count'xxxx', scan'xxxx' and so on.

2 by Native Java Api, himself a package RESTfull Api, through the provision of Api (HTTP) to HBase

3 using Thrift serialization, Thrift support C++, PHP, Python language, suitable for operation of HBase heterogeneous systems of other, this just try

4 using HBasExplorer, before writing a graphical client to operate HBase, 

5. Hive/Pig , This is not really used.

At present the main speaker of the Thrift in third ways, this is a Facebook open source out, is the official website http://thrift.apache.org/  .


Download and install and start, please see the article content

Check whether the successful running...


Use the PHP class file operations Hbase, generating the class file, please see the production method of reference in the text, but the method to generate my own test Bug, namespace generated class file is empty, but generated from the official source code library is the namespace Hbase, so there is need to pay attention to them.

I debugging a driver class file, posted on GitHub, you need to download and use. 

https://github.com/xinqiyang/buddy/tree/master/Vender/thrift

The next test, reference   test class here, write a test, and debug.

<?php

/***
Thrift Test Class by xinqiyang

*/

ini_set('display_error', E_ALL);

$GLOBALS['THRIFT_ROOT'] = './lib';


/* Dependencies. In the proper order. */
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Transport/TTransport.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Protocol/TProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Protocol/TBinaryProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Transport/TBufferedTransport.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Type/TMessageType.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Factory/TStringFuncFactory.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/StringFunc/TStringFunc.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/StringFunc/Core.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Type/TType.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Exception/TException.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Exception/TTransportException.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Thrift/Exception/TProtocolException.php';





/* Remember these two files? */
require_once $GLOBALS['THRIFT_ROOT'].'/Types.php';
require_once $GLOBALS['THRIFT_ROOT'].'/Hbase.php';




use Thrift\Protocol\TBinaryProtocol;
use Thrift\Transport\TSocket;
use Thrift\Transport\TSocketPool;
use Thrift\Transport\TFramedTransport;
use Thrift\Transport\TBufferedTransport;
use Hbase\HbaseClient;


//define host and port
$host = '192.168.56.56';
$port = 9090;
$socket = new Thrift\Transport\TSocket($host, $port);

$transport = new TBufferedTransport($socket);
$protocol = new TBinaryProtocol($transport);
// Create a calculator client
$client = new HbaseClient($protocol);
$transport->open();



//echo "Time: " . $client -> time();

$tables = $client->getTableNames();
sort($tables);

foreach ($tables as $name) {

	echo $name."\r\n";
}

//create a fc and then create a table
$columns = array(
	new \Hbase\ColumnDescriptor(array(
			'name' => 'id:',
			'maxVersions' => 10
		)),
	new \Hbase\ColumnDescriptor(array(
			'name' => 'name:'
		)),
	new \Hbase\ColumnDescriptor(array(
			'name' => 'score:'
		)),
);

$tableName = "student";



/*
try {
    $client->createTable($tableName, $columns);
} catch (AlreadyExists $ae) {
    var_dump( "WARN: {$ae->message}\n" );
}
*/

// get table descriptors
$descriptors = $client->getColumnDescriptors($tableName);
asort($descriptors);
foreach ($descriptors as $col) {
	var_dump( "  column: {$col->name}, maxVer: {$col->maxVersions}\n" );
}

//set clomn



//add update column data

$time = time();

var_dump($time);

$row = '2';
$valid = "foobar-".$time;



$mutations = array(
	new \Hbase\Mutation(array(
			'column' => 'score',
			'value' => $valid
		)),
);


$mutations1 = array(
	new \Hbase\Mutation(array(
			'column' => 'score:a',
			'value' => $time,
		)),
);


$attributes = array (

);



//add row, write a row
$row1 = $time;
$client->mutateRow($tableName, $row1, $mutations1, $attributes);

echo "-------write row $row1 ---\r\n";


//update row
$client->mutateRow($tableName, $row, $mutations, $attributes);


//get column data
$row_name = $time;
$fam_col_name = 'score:a';
$arr = $client->get($tableName, $row_name, $fam_col_name, $attributes);

// $arr = array
foreach ($arr as $k => $v) {
	// $k = TCell
	echo " ------ get one : value = {$v->value} , <br>  ";
	echo " ------ get one : timestamp = {$v->timestamp}  <br>";
}

echo "----------\r\n";

$arr = $client->getRow($tableName, $row_name, $attributes);
// $client->getRow return a array
foreach ($arr as $k => $TRowResult) {
	// $k = 0 ; non-use
	// $TRowResult = TRowResult
	var_dump($TRowResult);
}


echo "----------\r\n";
/******
  //no test
  public function scannerOpenWithScan($tableName, \Hbase\TScan $scan, $attributes);

  public function scannerOpen($tableName, $startRow, $columns, $attributes);
  public function scannerOpenWithStop($tableName, $startRow, $stopRow, $columns, $attributes);
  public function scannerOpenWithPrefix($tableName, $startAndPrefix, $columns, $attributes);
  public function scannerOpenTs($tableName, $startRow, $columns, $timestamp, $attributes);
  public function scannerOpenWithStopTs($tableName, $startRow, $stopRow, $columns, $timestamp, $attributes);
  public function scannerGet($id);
  public function scannerGetList($id, $nbRows);
  public function scannerClose($id);
*/


echo "----scanner get ------\r\n";
$startRow = '1';
$columns = array ('column' => 'score', );


//

$scan = $client->scannerOpen($tableName, $startRow, $columns, $attributes);

//$startAndPrefix = '13686667';
//$scan = $client->scannerOpenWithPrefix($tableName,$startAndPrefix,$columns,$attributes);

//$startRow = '1';
//$stopRow = '2';
//$scan = $client->scannerOpenWithStop($tableName, $startRow, $stopRow, $columns, $attributes);



//$arr = $client->scannerGet($scan);

$nbRows = 1000;

$arr = $client->scannerGetList($scan, $nbRows);

var_dump('count of result :'.count($arr));

foreach ($arr as $k => $TRowResult) {
	// code...
	//var_dump($TRowResult);
}

$client->scannerClose($scan);

//close transport
$transport->close();

  

Where the operating createTable, Insert Row, Get Table, Update Row, Scan Table these common, be familiar with.


The actual operation of the time, need to pay attention to:

The version of 1.php, need to support namespace, so it needs the support of more than 5.3

Installing thrift 2 PHP extension, looks like this has no practical use, or use the documents related to the PHP, who can write an extension is good. Do not know whether can improve performance.

3 the operation of scan, start/stop test, prefix Scan, the feeling is still possible.

4 sense PHP namespace is poor, how to do..\ segmentation the feeling is so not tunnel......

Then, if you have time, do some other operations, and pressure test, and the deployment in the cluster.

Useful Thrift welcome, thanks to this article written by hguisu (see article), so that we can as soon as possible entry.


Updated content:

20130517   In the group launched the Thrift that when the write operation, or unstable, serious overtime phenomenon, for the operation, need to optimize operation of the PHP class. I feel operations or write too complex.




Reference article




Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download

Posted by Bishop at October 27, 2013 - 11:01 PM