Getting Hive metastore to work with HDInsight Emulator.

For some reason, Hive metastore.exe (which runs as a service) did not create the metastore db that is required for Hive to store table definitions. 

Hive reported the following exception at startup in the log files. 

"set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.

org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out"

Because of this error, "Show databases" command did not work and I could not use Hive with HDInsight emulator. 

After hours of investigation, I made the following edits to "C:\hdp\hive-0.13.0.2.1.3.0-1981\conf\hive-site.xml". As you can see, I changed the settings to EmbeddedDriver instead of ClientDriver. 

  javax.jdo.option.ConnectionURL

 

  jdbc:derby:;databaseName=metastore_db;create=true

  JDBC connect string for a JDBC metastore

  javax.jdo.option.ConnectionDriverName

 

  org.apache.derby.jdbc.EmbeddedDriver

  Driver class name for a JDBC metastore

Then I stopped the default metastore.exe service. 

And used the following command to start metastore

"Hive.cmd --service metastore" 

with the initial directory set to hive home.

This created a folder named "metadata_db" and finally hive started successfully!! 

By default, the metastore service listens for requests on port 9083.

You can confirm that the service is running, by executing netstat command as shown below.  

netstat -ano | find "9083"