configuration - hadoop hbase cdh4 job fails to start with permission errors -
within cdh4 ecosystem, trying mapreduce job output hbase table. reason failing during adddependencyjars call of configuration setup.
from can tell hbase configuration not pick hadoop configuration (see warning output of job). have provided hdfs-site.xml, job configuration, job output stack trace, , file permissions.
any or insight how debug further appreciated.
hdfs-site.xml
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <!-- replication configuration --> <property> <name>dfs.permissions.enabled</name> <value>false</value> </property> <property> <name>dfs.permissions.superusergroup</name> <value>hadoop</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/var/hadoop/namenode</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>/var/hadoop/datanode</value> </property> </configuration> // configuration job
configuration conf = hbaseconfiguration.create(); job job = new job(conf); job.setjarbyclass(localcsvcdrhbasejob.class); job.setjobname("local cvs cdr venue session analysis hbase"); job.setmapoutputkeyclass(intwritable.class); job.setmapoutputvalueclass(venuesession.class); job.setmapperclass(venuemapper.class); job.setreducerclass(venuesessioncounthbasereducer.class); job.setinputformatclass(textinputformat.class); job.setoutputformatclass(tableoutputformat.class); fileinputformat.setinputpaths(job, new path(args[0])); tablemapreduceutil.inittablereducerjob("venue_session", venuesessioncounthbasereducer.class, job); tablemapreduceutil.adddependencyjars(job); job.waitforcompletion(true); the hbase classpath contains hadoop conf directory (etc/hadoop/conf).
:~ # sudo -u mapred hadoop_classpath=`hbase classpath` hadoop jar /home/mapred/cdr-hadoop-0.0.0-snapshot.jar net.thecloud.bi.cdr.jobs.localcsvcdrhbasejob /cdr-venue-sessions/2013-05-22.cdr.csv 13/08/08 11:03:12 warn conf.configuration: dfs.df.interval deprecated. instead, use fs.df.interval 13/08/08 11:03:12 warn conf.configuration: dfs.max.objects deprecated. instead, use dfs.namenode.max.objects 13/08/08 11:03:12 warn conf.configuration: hadoop.native.lib deprecated. instead, use io.native.lib.available 13/08/08 11:03:12 warn conf.configuration: dfs.data.dir deprecated. instead, use dfs.datanode.data.dir 13/08/08 11:03:12 warn conf.configuration: dfs.name.dir deprecated. instead, use dfs.namenode.name.dir 13/08/08 11:03:12 warn conf.configuration: fs.default.name deprecated. instead, use fs.defaultfs 13/08/08 11:03:12 warn conf.configuration: fs.checkpoint.dir deprecated. instead, use dfs.namenode.checkpoint.dir 13/08/08 11:03:12 warn conf.configuration: dfs.block.size deprecated. instead, use dfs.blocksize 13/08/08 11:03:12 warn conf.configuration: dfs.access.time.precision deprecated. instead, use dfs.namenode.accesstime.precision 13/08/08 11:03:12 warn conf.configuration: dfs.replication.min deprecated. instead, use dfs.namenode.replication.min 13/08/08 11:03:12 warn conf.configuration: dfs.name.edits.dir deprecated. instead, use dfs.namenode.edits.dir 13/08/08 11:03:12 warn conf.configuration: dfs.replication.considerload deprecated. instead, use dfs.namenode.replication.considerload 13/08/08 11:03:12 warn conf.configuration: dfs.balance.bandwidthpersec deprecated. instead, use dfs.datanode.balance.bandwidthpersec 13/08/08 11:03:12 warn conf.configuration: dfs.safemode.threshold.pct deprecated. instead, use dfs.namenode.safemode.threshold-pct 13/08/08 11:03:12 warn conf.configuration: dfs.http.address deprecated. instead, use dfs.namenode.http-address 13/08/08 11:03:12 warn conf.configuration: dfs.name.dir.restore deprecated. instead, use dfs.namenode.name.dir.restore 13/08/08 11:03:12 warn conf.configuration: dfs.https.client.keystore.resource deprecated. instead, use dfs.client.https.keystore.resource 13/08/08 11:03:12 warn conf.configuration: dfs.backup.address deprecated. instead, use dfs.namenode.backup.address 13/08/08 11:03:12 warn conf.configuration: dfs.backup.http.address deprecated. instead, use dfs.namenode.backup.http-address 13/08/08 11:03:12 warn conf.configuration: dfs.permissions deprecated. instead, use dfs.permissions.enabled 13/08/08 11:03:12 warn conf.configuration: dfs.safemode.extension deprecated. instead, use dfs.namenode.safemode.extension 13/08/08 11:03:12 warn conf.configuration: dfs.datanode.max.xcievers deprecated. instead, use dfs.datanode.max.transfer.threads 13/08/08 11:03:12 warn conf.configuration: dfs.https.need.client.auth deprecated. instead, use dfs.client.https.need-auth 13/08/08 11:03:12 warn conf.configuration: dfs.https.address deprecated. instead, use dfs.namenode.https-address 13/08/08 11:03:12 warn conf.configuration: dfs.replication.interval deprecated. instead, use dfs.namenode.replication.interval 13/08/08 11:03:12 warn conf.configuration: fs.checkpoint.edits.dir deprecated. instead, use dfs.namenode.checkpoint.edits.dir 13/08/08 11:03:12 warn conf.configuration: dfs.write.packet.size deprecated. instead, use dfs.client-write-packet-size 13/08/08 11:03:12 warn conf.configuration: dfs.permissions.supergroup deprecated. instead, use dfs.permissions.superusergroup 13/08/08 11:03:12 warn conf.configuration: topology.script.number.args deprecated. instead, use net.topology.script.number.args 13/08/08 11:03:12 warn conf.configuration: dfs.umaskmode deprecated. instead, use fs.permissions.umask-mode 13/08/08 11:03:12 warn conf.configuration: dfs.secondary.http.address deprecated. instead, use dfs.namenode.secondary.http-address 13/08/08 11:03:12 warn conf.configuration: fs.checkpoint.period deprecated. instead, use dfs.namenode.checkpoint.period 13/08/08 11:03:12 warn conf.configuration: topology.node.switch.mapping.impl deprecated. instead, use net.topology.node.switch.mapping.impl 13/08/08 11:03:12 warn conf.configuration: io.bytes.per.checksum deprecated. instead, use dfs.bytes-per-checksum exception in thread "main" java.io.ioexception: java.lang.runtimeexception: java.io.ioexception: permission denied @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.findorcreatejar(tablemapreduceutil.java:598) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.adddependencyjars(tablemapreduceutil.java:549) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.adddependencyjars(tablemapreduceutil.java:513) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.inittablereducerjob(tablemapreduceutil.java:456) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.inittablereducerjob(tablemapreduceutil.java:393) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.inittablereducerjob(tablemapreduceutil.java:363) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.inittablereducerjob(tablemapreduceutil.java:346) @ net.thecloud.bi.cdr.jobs.localcsvcdrhbasejob.main(localcsvcdrhbasejob.java:46) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) @ java.lang.reflect.method.invoke(method.java:597) @ org.apache.hadoop.util.runjar.main(runjar.java:208) caused by: java.lang.runtimeexception: java.io.ioexception: permission denied @ org.apache.hadoop.util.jarfinder.getjar(jarfinder.java:164) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) @ java.lang.reflect.method.invoke(method.java:597) @ org.apache.hadoop.hbase.mapreduce.tablemapreduceutil.findorcreatejar(tablemapreduceutil.java:595) ... 12 more caused by: java.io.ioexception: permission denied @ java.io.unixfilesystem.createfileexclusively(native method) @ java.io.file.checkandcreate(file.java:1704) @ java.io.file.createtempfile(file.java:1792) @ org.apache.hadoop.util.jarfinder.getjar(jarfinder.java:156) ... 17 more file permissions
:~ # ls -l /var/hadoop/ total 12 drwxrwxrwx 2 hdfs hdfs 4096 aug 8 09:23 datanode drwxrwxrwx 3 mapred hadoop 4096 aug 8 09:41 mapred drwxrwxrwx 3 hdfs hdfs 4096 aug 8 09:59 namenode hdfs permissions
:~ # hdfs dfs -ls -r / drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:36 /cdr-venue-sessions -rw-rw-rw- 3 hdfs hadoop 27014304 2013-08-08 09:36 /cdr-venue-sessions/2013-05-22.cdr.csv drwxrwxrwx - hbase hadoop 0 2013-08-08 10:10 /hbase drwxrwxrwx - hbase hadoop 0 2013-08-08 10:07 /hbase/.logs drwxrwxrwx - hbase hadoop 0 2013-08-08 10:06 /hbase/.oldlogs drwxrwxrwx - hbase hadoop 0 2013-08-08 10:10 /hbase/.tmp -rw-rw-rw- 3 hbase hadoop 38 2013-08-08 10:06 /hbase/hbase.id -rw-rw-rw- 3 hbase hadoop 3 2013-08-08 10:06 /hbase/hbase.version drwxrwxrwx - hbase hadoop 0 2013-08-08 10:10 /hbase/venue_session -rw-rw-rw- 3 hbase hadoop 711 2013-08-08 10:10 /hbase/venue_session/.tableinfo.0000000001 drwxrwxrwx - hbase hadoop 0 2013-08-08 10:10 /hbase/venue_session/.tmp drwxrwxrwx - hbase hadoop 0 2013-08-08 10:10 /hbase/venue_session/5cd64eee2dea6b1464023f24eee3daf0 -rw-rw-rw- 3 hbase hadoop 246 2013-08-08 10:10 /hbase/venue_session/5cd64eee2dea6b1464023f24eee3daf0/.regioninfo drwxrwxrwx - hbase hadoop 0 2013-08-08 10:10 /hbase/venue_session/5cd64eee2dea6b1464023f24eee3daf0/values drwxrwxrwt - hdfs hadoop 0 2013-08-08 09:41 /tmp drwxrwxrwx - mapred hadoop 0 2013-08-08 09:41 /tmp/hadoop-mapred drwxrwxrwx - mapred hadoop 0 2013-08-08 09:41 /tmp/hadoop-mapred/mapred drwxrwxrwx - mapred hadoop 0 2013-08-08 10:06 /tmp/hadoop-mapred/mapred/system -rw-rw-rw- 3 mapred hadoop 4 2013-08-08 10:06 /tmp/hadoop-mapred/mapred/system/jobtracker.info drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:30 /user-venue-types drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:28 /var drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:28 /var/hadoop drwxrwxrwx - mapred hadoop 0 2013-08-08 09:28 /var/hadoop/mapred drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:27 /var/lib drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:27 /var/lib/hadoop-hdfs drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:27 /var/lib/hadoop-hdfs/cache drwxrwxrwx - mapred hadoop 0 2013-08-08 09:27 /var/lib/hadoop-hdfs/cache/mapred drwxrwxrwx - mapred hadoop 0 2013-08-08 09:27 /var/lib/hadoop-hdfs/cache/mapred/mapred drwxrwxrwt - mapred hadoop 0 2013-08-08 09:27 /var/lib/hadoop-hdfs/cache/mapred/mapred/staging drwxrwxrwx - hdfs hadoop 0 2013-08-08 09:30 /venues
permissions not easy in hadoop usually. couple of debugging points:
- please check user run job , user 'visible' on hadoop cluster.
- please check happens inside method , files being modified.
- assure required permissions met. if not, have option either disable hdfs permissions or 'represent' different user hadoop cluster.
these questions useful you:
Comments
Post a Comment