bigdata

Debugging custom libraries hive update logging to console

Debugging custom libraries hive update logging to console. When launch Hive cli change logging set root logging or your library logging to DEBUG or INFO and print to console- hive –hiveconf hive.root.logger=<INFO|DEBUG>,consolehive –hiveconf hive.root.logger=<INFO|DEBUG>,console Check Mapreduce jobs logs- http://jobtracker:<job tracker port e.g. 50030>/jobdetails.jsp?jobid=<job id> then go to map or reduce…

Hive NoClassDefFoundError auxiliary path issue

Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error- 1 2 add jar /xxx/hive-customserde.jar; add jar /xxx/solr-solrj.jar;add jar /xxx/hive-customserde.jar; add jar /xxx/solr-solrj.jar; Above commands will add resource to hive class path but suppose your custom…

Hbase tips and tricks

Hbase tips and tricks irbrc file-irbrc configuration to save all command history of all hbase shell invocations. minimal configuration of irbrc- more ~/.irbrc require ‘irb/ext/save-history’ IRB.conf[:SAVE_HISTORY] = 100 IRB.conf[:HISTORY_FILE] = "#{ENV[‘HOME’]}/.irb_history" Kernel.at_exit do IRB.conf[:AT_EXIT].each do |i| i.call end endmore ~/.irbrc require ‘irb/ext/save-history’ IRB.conf[:SAVE_HISTORY] = 100 IRB.conf[:HISTORY_FILE] = "#{ENV[‘HOME’]}/.irb_history" Kernel.at_exit do…