tez 에서 UNION ALL 을 이용할때,
테이블중 하나의 조건중 결과가 0건이라면 이런 오류가 난다.
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: NullPointerException null
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:231)
hive 상위버전으로 올라가면 버그가 해결될지 모르겠지만,
우선 이 버그를 회피하려면 vectorized 옵션을 off 해야한다.
set hive.vectorized.execution.enabled = false;
1. CDH 버전체크
우선 설치된 CDH 버전확인부터 !
빌드할때 pom.xml 을 수정해야 하니 hadoop과 pig 버전을 체크하자.
$ hadoop version
Hadoop 2.6.0-cdh5.10.0
Subversion http://github.com/cloudera/hadoop -r 307b3de961d083f6e8ee80ddba589f22cacd3662
Compiled by jenkins on 2017-01-20T20:12Z
Compiled with protoc 2.5.0
From source with checksum 9ee56c2ef46d6f9fae2f8b199c0e2c9
This command was run using /home1/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/jars/hadoop-common-2.6.0-cdh5.10.0.jar
$ pig version
2017-03-07 14:13:44,750 [main] INFO org.apache.pig.Main - Apache Pig version 0.12.0-cdh5.10.0 (rexported) compiled Jan 20
2.6.0-cdh5.10.0 , 0.12.0.-cdh5.10.0 이라는걸 알았다.
2. 소스다운로드 / POM 수정
http://tez.apache.org 에서 다운로드 받을수 있다. 다운받고 tar.gz 파일을 풀고 pom 파일을 수정해야한다.
CDH 배포버전은 기본 버전과는 구성이 다를수 있다. 그래서 CDH 용 버전과 리파지토리에 맞춰야한다.
...
<properties>
......
<hadoop.version>2.6.0-cdh5.10.0</hadoop.version> <!-- 버전 수정-->
<pig.version>0.12.0-cdh5.10.0</pig.version> <!-- 버전 수정-->
...
</properties>
...
<repositories>
<repository> <!-- 추가 -->
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
...
</repositories>
<pluginRepositories>
<pluginRepository> <!-- 추가 -->
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</pluginRepository>
....
</pluginRepositories>
3. 소스파일 수정
빌드하기전에 필요한건 javac , maven, 그리고 protobuf 2.5 버전이다. protobuf 는 서버간 콜렉션을 넘긴다고 해야하나? 아무튼 데이터 교환관련한건데 구글에서 만든걸로 알고 있다. (유사한걸로 페이스북에서 미는게 thrift 로 알고 있다)
protobuf 의 경우는 [이글] 을 참고하자
근데, CDH 환경용도로 빌드하면 기본적으로 실패가 난다. 아주 짜증나지만, CDH에서 패키징할때 나름대로 버전짜집기(?)를 하다보니 우리도 이런 짜집기를 해줘야한다고 해야할까?
3.1. JobContextImpl 관련 오류
아마도 버전차이로 인해 나오는 문제가 아닐가 싶은데, 추상클래스에서 구현이 안된 메소드가 존재하기 때문이다.
그래서 method 하나를 추가해주자 (하단 노란 박스 내용추가)
[ERROR] /home1/mc-jeong/tez-084-cdh5.10.0/apache-tez-0.8.4-src/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java:[58,8] org.apache.tez.mapreduce.hadoop.mapreduce.JobContextImpl is not abstract and does not override abstract method userClassesTakesPrecedence() in org.apache.hadoop.mapreduce.JobContext
[INFO] 1 error
* apache-tez-0.8.4-src/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java
// 이 메소드를 추가해준다
@Override
public boolean userClassesTakesPrecedence() {
return getJobConf().getBoolean(MRJobConfig.MAPREDUCE_JOB_USER_CLASSPATH_FIRST, false);
}
3.2 ShuffleHandler 관련오류
이번엔 또 tez-ext-service-test 쪽에서 실패가 난다. (구버전에선 오류안났었음)
해당 java 파일을 뒤져보면 netty 에서 request.getHeader() , response.setHeader(name, obj) 를 할때 문제가 되는걸로 보인다.
netty 쪽 홈페이지의 api 스펙과 github 코드를 찾아보니 header 정보를 다루는 메소드 사용법이 달라진걸로 보인다.
바로 header에 get(), set() 하지 않고, header() 정보를 받고 그이후 get() set() 하는 경유하는 구조로 변경되었다.
아마도 header 관련 책임을 별도의 클래스로 빼려고 해서 그런게 아닐까 싶다.
대충 10개정도 부분이 있을텐데, 노란색 박스를 참고해서 메소드 call 하는걸 치환하도록 하자.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project tez-ext-service-tests: Compilation failure: Compilation failure:
[ERROR] /home1/mc-jeong/tez-084-cdh5.10.0/apache-tez-0.8.4-src/tez-ext-service-tests/src/test/java/org/apache/tez/shufflehandler/ShuffleHandler.java:[370,18] cannot find symbol
[ERROR] symbol: method getHeader(java.lang.String)
[ERROR] location: variable request of type org.jboss.netty.handler.codec.http.HttpRequest
[ERROR] /home1/mc-jeong/tez-084-cdh5.10.0/apache-tez-0.8.4-src/tez-ext-service-tests/src/test/java/org/apache/tez/shufflehandler/ShuffleHandler.java:[372,22] cannot find symbol
[ERROR] symbol: method getHeader(java.lang.String)
* tez-084-cdh5.10.0/apache-tez-0.8.4-src/tez-ext-service-tests/src/test/java/org/apache/tez/shufflehandler/ShuffleHandler.java
// 메소드호출방법을 수정해야한다.
request.getHeader(...) -> request.headers().get(...)
response.setHeader(...) -> response.headers().set(...)
4. 빌드완료
자.. 다시 빌드 시도하자
mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ tez-docs ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /home1/mc-jeong/tez-084-cdh5.10.0/apache-tez-0.8.4-src/docs/target/tez-docs-0.8.4-tests.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] tez ................................................ SUCCESS [ 1.666 s]
[INFO] hadoop-shim ........................................ SUCCESS [ 1.840 s]
[INFO] tez-api ............................................ SUCCESS [ 4.768 s]
[INFO] tez-common ......................................... SUCCESS [ 0.400 s]
[INFO] tez-runtime-internals .............................. SUCCESS [ 0.782 s]
[INFO] tez-runtime-library ................................ SUCCESS [ 1.955 s]
[INFO] tez-mapreduce ...................................... SUCCESS [ 0.988 s]
[INFO] tez-examples ....................................... SUCCESS [ 0.171 s]
[INFO] tez-dag ............................................ SUCCESS [ 3.902 s]
[INFO] tez-tests .......................................... SUCCESS [ 0.762 s]
[INFO] tez-ext-service-tests .............................. SUCCESS [ 0.597 s]
[INFO] tez-ui ............................................. SUCCESS [02:04 min]
[INFO] tez-ui2 ............................................ SUCCESS [04:29 min]
[INFO] tez-plugins ........................................ SUCCESS [ 0.014 s]
[INFO] tez-yarn-timeline-history .......................... SUCCESS [ 0.391 s]
[INFO] tez-yarn-timeline-history-with-acls ................ SUCCESS [ 0.240 s]
[INFO] tez-history-parser ................................. SUCCESS [ 32.685 s]
[INFO] tez-tools .......................................... SUCCESS [ 0.015 s]
[INFO] tez-perf-analyzer .................................. SUCCESS [ 0.013 s]
[INFO] tez-job-analyzer ................................... SUCCESS [ 0.748 s]
[INFO] tez-javadoc-tools .................................. SUCCESS [ 0.132 s]
[INFO] hadoop-shim-impls .................................. SUCCESS [ 0.013 s]
[INFO] hadoop-shim-2.6 .................................... SUCCESS [ 0.108 s]
[INFO] tez-dist ........................................... SUCCESS [ 31.058 s]
[INFO] Tez ................................................ SUCCESS [ 0.017 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:57 min
[INFO] Finished at: 2017-03-07T15:32:53+09:00
[INFO] Final Memory: 114M/1580M
[INFO] ------------------------------------------------------------------------
hive orc 로 사용할때 struct 형태로 구성이 가능하다.
하지만 join 시도시 오류가 난다.
버그로 보이는데 map side join 으로 유도할때 문제가 발생되는듯 하다.
일단 오류를 회피하려면 map join 으로 유도한되게 하면 되긴한다.
https://issues.apache.org/jira/browse/HIVE-11051
위 링크가 유사한 오류에 대한 리포팅으로 보인다. 1.3.0 2.0.0 에서는 fix 되었다는데, 내가 쓰는버전은 아니니 아래와 같이 일단 회피.
* 해결방법
set hive.auto.convert.join=false;
* 에러메시지
INFO : Map 1: 26(+2)/28 Map 5: 337(+33)/370 Map 6: 1(+1)/2 Reducer 3: 0(+0,-415)/377 Reducer 4: 0/208
ERROR : Status: Failed
ERROR : Vertex re-running, vertexName=Map 5, vertexId=vertex_1475821062280_208694_1_01
ERROR : Vertex re-running, vertexName=Map 6, vertexId=vertex_1475821062280_208694_1_00
ERROR : Vertex re-running, vertexName=Map 1, vertexId=vertex_1475821062280_208694_1_02
ERROR : Vertex re-running, vertexName=Map 5, vertexId=vertex_1475821062280_208694_1_01
ERROR : Vertex re-running, vertexName=Map 6, vertexId=vertex_1475821062280_208694_1_00
ERROR : Vertex re-running, vertexName=Map 1, vertexId=vertex_1475821062280_208694_1_02
ERROR : Vertex re-running, vertexName=Map 5, vertexId=vertex_1475821062280_208694_1_01
ERROR : Vertex re-running, vertexName=Map 6, vertexId=vertex_1475821062280_208694_1_00
ERROR : Vertex failed, vertexName=Reducer 3, vertexId=vertex_1475821062280_208694_1_03, diagnostics=[Task failed, taskId=task_1475821062280_208694_1_03_000165, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task: attempt_1475821062280_208694_1_03_000165_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1047)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:858)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:718)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:786)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:397)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:638)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:670)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:748)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:306)
... 23 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldData(StandardStructObjectInspector.java:170)
at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator._evaluate(ExprNodeFieldEvaluator.java:93)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.populateCachedDistributionKeys(ReduceSinkOperator.java:430)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:336)
... 28 more
], TaskAttempt 1 failed, info=[Error: Failure while running task: attempt_1475821062280_208694_1_03_000165_1:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1047)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:858)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:718)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:786)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:397)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:638)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:670)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:748)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:306)
... 23 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldData(StandardStructObjectInspector.java:170)
at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator._evaluate(ExprNodeFieldEvaluator.java:93)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.populateCachedDistributionKeys(ReduceSinkOperator.java:430)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:336)
... 28 more
], TaskAttempt 2 failed, info=[Error: Failure while running task: attempt_1475821062280_208694_1_03_000165_2:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1047)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:858)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:718)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:786)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:397)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:638)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:670)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:748)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:306)
... 23 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldData(StandardStructObjectInspector.java:170)
at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator._evaluate(ExprNodeFieldEvaluator.java:93)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.populateCachedDistributionKeys(ReduceSinkOperator.java:430)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:336)
... 28 more
], TaskAttempt 3 failed, info=[Error: Failure while running task: attempt_1475821062280_208694_1_03_000165_3:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:270)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"_col0":"12개월","_col1":"306960"},"value":{"_col0":6}}
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:318)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1047)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:858)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:718)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:786)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:397)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:638)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:670)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:748)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:306)
... 23 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;
at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldData(StandardStructObjectInspector.java:170)
at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator._evaluate(ExprNodeFieldEvaluator.java:93)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.populateCachedDistributionKeys(ReduceSinkOperator.java:430)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:336)
... 28 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:376, Vertex vertex_1475821062280_208694_1_03 [Reducer 3] killed/failed due to:OWN_TASK_FAILURE]
ERROR : Vertex killed, vertexName=Reducer 4, vertexId=vertex_1475821062280_208694_1_04, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:208, Vertex vertex_1475821062280_208694_1_04 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE]
ERROR : Vertex killed, vertexName=Map 1, vertexId=vertex_1475821062280_208694_1_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:2, Vertex vertex_1475821062280_208694_1_02 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE]
ERROR : Vertex killed, vertexName=Map 5, vertexId=vertex_1475821062280_208694_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:33, Vertex vertex_1475821062280_208694_1_01 [Map 5] killed/failed due to:OTHER_VERTEX_FAILURE]
ERROR : Vertex killed, vertexName=Map 6, vertexId=vertex_1475821062280_208694_1_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1475821062280_208694_1_00 [Map 6] killed/failed due to:OTHER_VERTEX_FAILURE]
ERROR : DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:4
Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=2)
