Perform CRUD over HBase using HBaseTestingUtility/EmbeddedHBase

by Vivek Mishra


EmbeddedHBase, Something similar to EmbeddedCassandraService. I have been looking for something like this since last year(March 2011) to incorporate the same in Kundera. HBaseTestingUtility really helped me to achieve continuous integration and also testing junits with in-memory HBase server. So here i am sharing some code snippets about how to configure and start EmbeddedHBase:

  • HBaseCli: Responsible for initializing and configuring HBaseTestingUtility.

public final class HBaseCli
{

/** The Constant logger. */
private static final Logger logger = LoggerFactory.getLogger(HBaseCli.class);

/** The utility. */
private static HBaseTestingUtility utility;

private static Boolean isStarted = false;

}

  • Start cluster: (You can change settings for zookeeper port, timeout etc.)

/**
* Starts a new cluster.
*/
public static void startCluster()
{
if (!isStarted)
{
File workingDirectory = new File(“./”);
Configuration conf = new Configuration();
System.setProperty(“test.build.data”, workingDirectory.getAbsolutePath());
conf.set(“test.build.data”, new File(workingDirectory, “zookeeper”).getAbsolutePath());
conf.set(“fs.default.name”, “file:///”);
conf.set(“zookeeper.session.timeout”, “180000”);
conf.set(“hbase.zookeeper.peerport”, “2888”);
conf.set(“hbase.zookeeper.property.clientPort”, “2181”);
try
{
conf.set(HConstants.HBASE_DIR, new File(workingDirectory, “hbase”).toURI().toURL().toString());
}
catch (MalformedURLException e1)
{
logger.error(e1.getMessage());
}

Configuration hbaseConf = HBaseConfiguration.create(conf);
utility = new HBaseTestingUtility(hbaseConf);
try
{
MiniZooKeeperCluster zkCluster = new MiniZooKeeperCluster(conf);
zkCluster.setClientPort(2181);
zkCluster.setTickTime(18000);
zkCluster.startup(utility.setupClusterTestBuildDir());
utility.setZkCluster(zkCluster);
utility.startMiniCluster();
utility.getHbaseCluster().startMaster();
}
catch (Exception e)
{
logger.error(e.getMessage());
throw new RuntimeException(e);
}
isStarted = true;
}
}

This should get Zookeeper, Hadoop(datanode, namenode, tasktracker etc.) and HBase master server started.

Create table

public static void createTable(String tableName)
{
try
{
if (!utility.getHBaseAdmin().tableExists(tableName))
{
utility.createTable(tableName.getBytes(), tableName.getBytes());
}
else
{
logger.info(“Table:” + tableName + ” already exist:”);
}
}
catch (IOException e)
{
logger.error(e.getMessage());
}
}

Add Column

public static void addColumn(String tableName, String columnFamily)
{
try
{
utility.getHBaseAdmin().disableTable(tableName);
utility.getHBaseAdmin().addColumn(tableName, new HColumnDescriptor(columnFamily));
utility.getHBaseAdmin().enableTable(tableName);
}
catch (InvalidFamilyOperationException ife)
{
logger.info(“Column family:” + columnFamily + ” already exist!”);
}
catch (IOException e)
{
logger.error(e.getMessage());
}
}

Stop Cluster:

public static void stopCluster()
{
try
{
if (utility != null)
{
utility.shutdownMiniCluster();
utility = null;
}
}
catch (IOException e)
{
logger.error(e.getMessage());
}
}

Happy Programming🙂