王炸科技头像

hadooptoomanyfiles异常处理

来源:
       

  今天hadoop集群任务执行失败了。报错信息如下2013-10-2608:00:03,229ERRORserver.TThreadPoolServerTThreadPoolServer.java:run182-Erroroccurredduringprocessingofmessage.atorg.apache.hadoop.hive.service.HiveServer$ThriftHiveProcess

  今天hadoop集群任务执行失败了。报错信息如下

  2013-10-2608:00:03,229ERRORserver.TThreadPoolServer(TThreadPoolServer.java:run(182))-Erroroccurredduringprocessingofmessage.atorg.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor(HiveServer.java:553)atorg.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:169)atjava.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)atjava.lang.Thread.run(Thread.java:662)atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:277)atorg.apache.hadoop.hive.service.HiveServer$HiveServerHandler.init(HiveServer.java:136)atorg.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor(HiveServer.java:550)...4moreatorg.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:199)atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:272)...6moreCausedby:java.lang.RuntimeException:java.io.FileNotFoundException:/home/hadoop/hadoop-0.20.205.0/conf/mapred-site.xml(Toomanyopenfiles)atorg.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1231)atorg.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1093)atorg.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)atorg.apache.hadoop.conf.Configuration.set(Configuration.java:438)atorg.apache.hadoop.hive.conf.HiveConf.setVar(HiveConf.java:762)atorg.apache.hadoop.hive.conf.HiveConf.setVar(HiveConf.java:770)atorg.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:169)atjava.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)atjava.lang.Thread.run(Thread.java:662)Causedby:java.lang.RuntimeException:org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.RuntimeException:java.io.FileNotFoundException:/home/hadoop/hadoop-0.20.205.0/conf/core-site.xml(Toomanyopenfiles)atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:277)atorg.apache.hadoop.hive.service.HiveServer$HiveServerHandler.init(HiveServer.java:136)atorg.apache.hadoop.hive.service.HiveServer$ThriftHiveProcessorFactory.getProcessor(HiveServer.java:550)...4moreCausedby:org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.RuntimeException:java.io.FileNotFoundException:/home/hadoop/hadoop-0.20.205.0/conf/core-site.xml(Toomanyopenfiles)atorg.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizeProviderManager(HiveUtils.java:199)atorg.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:272)...6more

  debian系统解决方式

  ulimit-HSn32768

  原文地址:hadooptoomanyfiles异常处理,感谢原作者分享。