Big Data

Hive NoClassDefFoundError auxiliary path issue

Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error- 1 2 add jar /xxx/hive-customserde.jar; add jar /xxx/solr-solrj.jar;add jar /xxx/hive-customserde.jar; add jar /xxx/solr-solrj.jar; Above commands will add resource to hive class path but suppose your custom…

Lanuch Hbase Mapreduce Job

Lanuch Hbase Mapreduce Job Add all the required jars to a variable separated by(,) like jar=a,b in my case I want to supply all hbase jars to job 1 2 3 4 5 jar=/home/hadoop/Desktop/techsquids/repos/techsquids/map-reduce/target/map-reduce-1.0-SNAPSHOT.jar classpath=$jar for f in $HBASE_HOME/lib/*.jar;do classpath="${classpath},$f" donejar=/home/hadoop/Desktop/techsquids/repos/techsquids/map-reduce/target/map-reduce-1.0-SNAPSHOT.jar classpath=$jar for f in $HBASE_HOME/lib/*.jar;do classpath="${classpath},$f" done Next prepare…

Hbase tips and tricks

Hbase tips and tricks irbrc file-irbrc configuration to save all command history of all hbase shell invocations. minimal configuration of irbrc- more ~/.irbrc require ‘irb/ext/save-history’ IRB.conf[:SAVE_HISTORY] = 100 IRB.conf[:HISTORY_FILE] = "#{ENV[‘HOME’]}/.irb_history" Kernel.at_exit do IRB.conf[:AT_EXIT].each do |i| i.call end endmore ~/.irbrc require ‘irb/ext/save-history’ IRB.conf[:SAVE_HISTORY] = 100 IRB.conf[:HISTORY_FILE] = "#{ENV[‘HOME’]}/.irb_history" Kernel.at_exit do…