[hive on tez] ROOT_INPUT_INIT_FAILURE 문제

2016. 2. 24. 22:20

일단 이런 조치는 해뒀다.

1) "$HADOOP_CLASSPATH" 에 tez라이브러리 패스 잡아둠

2) /etc/hadoop/conf/hadoop-env.sh 에도 라이브러리 패스 잡아둠

3) tez-site.xml 의 파일도 hdfs상 존재한다.

근데...이런 오류가 난다.

hive> set hive.execution.engine=tez;

hive> select count(*) from sample.user;

Query ID = user_20160224221313_71c7f4c8-dcb4-4fc6-b035-2cdd66caa8d1

Total jobs = 1

Launching Job 1 out of 1

Status: Running (Executing on YARN cluster with App id application_1456316700186_0002)

--------------------------------------------------------------------------------

VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED

--------------------------------------------------------------------------------

Map 1 FAILED -1 0 0 -1 0 0

Reducer 2 KILLED 1 0 0 1 0 0

--------------------------------------------------------------------------------

VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 0.22 s

--------------------------------------------------------------------------------

Status: Failed

Vertex failed, vertexName=Map 1, vertexId=vertex_1456316700186_0002_1_00, diagnostics=[Vertex vertex_1456316700186_0002_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: user initializer failed, vertex=vertex_1456316700186_0002_1_00 [Map 1], java.lang.ArithmeticException: / by zero

at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:122)

at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)

at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)

at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)

at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

]

Vertex killed, vertexName=Reducer 2, vertexId=vertex_1456316700186_0002_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1456316700186_0002_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]

DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask

일단, 오류가 나고 있는 버전은 'Hive 1.1.0-cdh5.5.1' 이다. (일단 과거에 성공했던건 Hive 1.1.0-cdh5.4.4 여서 jar를 바꿔서 테스트 했는데 잘된다)

우선, HiveSplitGenerator 이 클래스에서 오류인거 같으니 찾아보자. 다행인건 웹에서 조회하는 서비스가 있다.

http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hive/hive-exec/1.1.0/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java/

   @Override

   public List<Event> initialize() throws Exception {

     InputInitializerContext rootInputContext = getContext();

....

      int totalResource = rootInputContext.getTotalAvailableResource().getMemory();

      int taskResource = rootInputContext.getVertexTaskResource().getMemory();

      int availableSlots = totalResource / taskResource;

122번째 라인을 보면 taskResource가 0으로 만들어 지는것이 있나보다.

쫒아가서 getContext()를 가보면 리턴타입을 보니

org.apache.tez.runtime.api.InputInitializerContext;

ㄴ (getVertextTaskResource) org.apache.hadoop.yarn.api.records

이거다.

근데 저부분은 hive 코드인거 같고 tez에서 taskResource를 기본으로 뭔가 줄수 있는건 없나?

---------------- 추가글 ---------------

삽질하며 찾던중

http://stackoverflow.com/questions/34144978/hive-with-tez-out-of-memory-error

이런글을 찾았다.

hive.tez.container.size 설정을 바꿔주니 돌아간다. 관련된 값에 대해 찾아봐야할듯 하다.

hive> SET hive.tez.container.size=3020;

hive> set hive.execution.engine=tez;

hive> select count(*) from sample.user;

Query ID = user_20160229142020_d507d2e5-b55a-47a3-9ba1-f8dba9b315f6

Total jobs = 1

Launching Job 1 out of 1

Status: Running (Executing on YARN cluster with App id application_1456320477213_0056)

--------------------------------------------------------------------------------

VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED

--------------------------------------------------------------------------------

Map 1 .......... SUCCEEDED 1 1 0 0 0 0

Reducer 2 ...... SUCCEEDED 1 1 0 0 0 0

--------------------------------------------------------------------------------

VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 6.49 s

--------------------------------------------------------------------------------

Time taken: 12.434 seconds, Fetched: 1 row(s)

hive>

저작자표시 비영리 변경금지

정선생의 블로그

[hive on tez] ROOT_INPUT_INIT_FAILURE 문제

+ Recent posts

티스토리툴바