好不容易把Hive裝完了,結果一執行Hive的查詢語句運行MapReduce程序立馬報錯。。。
log詳細信息如下:
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 1; number of reducers: 0
2017-10-21 21:54:15,503 null map = 0%, reduce = 0%
2017-10-21 21:54:44,930 null map = 100%, reduce = 0%
Ended Job = job_1508590228879_0004 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1508590228879_0004_m_000000 (and more) from job job_1508590228879_0004
Exception in thread "Thread-19" java.lang.IllegalArgumentException: Does not contain a valid host:port authority: localat org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)at org.apache.hadoop.hive.ql.exec.JobTrackerURLResolver.getURL(JobTrackerURLResolver.java:42)at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:209)at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:92)at java.lang.Thread.run(Thread.java:744)
Execution failed with exit status: 2
Obtaining error information
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
看了網上的許多方法,有讓在hive-site日志下加入這個屬性:
<property><name>hive.exec.show.job.failure.debug.info</name><value>false</value>
</property>
然而并沒有效果。剛好最近開十九大,整個VPN被封完了,也沒法翻墻找答案,只能在內網大海撈針(答案千篇一律)。然而皇天不負有心人,最終總算在一個算是能訪問的國外論壇中找到了答案,(同樣的問題,十幾個回答中找到了唯一正確的解決方法。。。)。至于原因,我還沒搞懂(根據這個屬性的設置,個人猜測應該通過添加這個設置的屬性值,將程序源碼執行中要拋出的異常給忽略了)。
廢話少說直接上解決方案:
直接再hive-site下添加如下屬性:
<property><name>mapreduce.jobtracker.address</name><value>ignorethis</value>
</property>
這樣直接就OK了。
就因為一個這樣的小問題就花了我快一天的學習時間,真TM要命。。。
希望后能為后來者提供下幫助。