How to change address 'hadoop jar' command is connecting to? -


i have been trying start mapreduce job on cluster following command:

bin/hadoop jar myjar.jar mainclass /user/hduser/input /user/hduser/output

but following error on , on again, until connection refused:

13/08/08 00:37:16 info ipc.client: retrying connect server: localhost/127.0.0.1:54310. tried 5 time(s); retry policy retryuptomaximumcountwithfixedsleep(maxretries=10, sleeptime=1 seconds)

i checked netstat see if service listening correct port:

~> sudo netstat -plten | grep java tcp        0      0 10.1.1.4:54310          0.0.0.0:*               listen      10022      38365       11366/java tcp        0      0 10.1.1.4:54311          0.0.0.0:*               listen      10022      32164       11829/java

now notice service listening port 10.1.1.4:54310, ip of master, seems 'hadoop jar' command connecting 127.0.0.1 (the localhost, same machine) therefore doesn't find service. there anyway force 'hadoop jar' @ 10.1.1.4 instead of 127.0.0.1?

my namenode, datanode, jobtracker, tasktracker, ... running. checked datanode , tasktracker on slaves , seems working. can check webui on master , shows cluster online.

i expect problem dns related since seems 'hadoop jar' command finds correct port, uses 127.0.0.1 address instead of 10.1.1.4

update

configuration in core-site.xml

<configuration>  <property>   <name>hadoop.tmp.dir</name>   <value>/app/hadoop/tmp</value>   <description>a base other temporary directories.</description> </property>  <property>   <name>fs.default.name</name>   <value>hdfs://master:54310</value>   <description>the name of default file system.  uri   scheme , authority determine filesystem implementation.    uri's scheme determines config property (fs.scheme.impl) naming   filesystem implementation class.  uri's authority used   determine host, port, etc. filesystem.</description> </property>  </configuration>

configuration in mapred-site.xml

<configuration>  <property>   <name>mapred.job.tracker</name>   <value>master:54311</value>   <description>the host , port mapreduce job tracker runs   at.  if "local", jobs run in-process single map   , reduce task.   </description> </property>  </configuration>

configuration in hdfs-site.xml

<configuration>  <property>   <name>dfs.replication</name>   <value>1</value>   <description>default block replication.   actual number of replications can specified when file created.   default used if replication not specified in create time.   </description> </property>  </configuration>

although seemed dns issue, hadoop trying resolve reference localhost in code. deploying jar of else , assumed correct. upon further inspection found reference localhost , changed master, solving issue.


Comments

Post a Comment

Popular posts from this blog

image - ClassNotFoundException when add a prebuilt apk into system.img in android -

i'm in trouble add prebuilt apk system.img based taiwan mtk android phone platform, when push apk onto device,after reboot phone,the apk can installed succecessfully,but if add apk alps/out/target/product/$prj/system/app or vendor/mediatek/$prj/artifacts/out/target/product/$prj/system/app ,and produce system.img,the apk can't installed normally,the logcat say: a.lang.runtimeexception: unable instantiate activity componentinfo{com.sec.android.widgetapp.ap.hero.accuweather.widget/com.sec.android.widgetapp.ap.hero.accuweather.widget.weatherclock}: java.lang.classnotfoundexception: com.sec.android.widgetapp.ap.hero.accuweather.widget.weatherclock. can tell me what's problem ? when package manager installs apps part of system partition different installing apps on data partition. looks app using libraries somewhere else. when install on data lib directory created in apps data directory , symlinked /data/app-lib/. when apps installed on system it's expecte...
Read more

I need to import mysql 5.1 to 5.5? -

i have mysql 5.1 database , need import localhost running 5.5. how import 5.5 or downgrade mysql? install phpmyadmin 5.1 os: http://www.wikihow.com/install-phpmyadmin-on-your-windows-pc or https://www.digitalocean.com/community/articles/how-to-install-and-secure-phpmyadmin-on-ubuntu-12-04 then login, click on database, hit export, , export in sql format (quick fine). import .sql file 5.5 server (after installing phpmyadmin) logging in clicking database , hit import. hope helps!
Read more

Java, Hibernate, MySQL - store UTC date-time -

i saw here on few related questions (like this one , of course, this one )... essentially, want store date-time utc, , let application user choose time zone wants display date-time in. since seems date-time fields affected underlying jdbc driver, wonder if acceptable way go storing utc date-time: set both mysql , application server machine utc time zone (no need separate) both mysql , jvm should pick underlying system time settings (if not instructed otherwise) use datetime table columns on mysql side use java.util.date corresponding mapping on hibernate side (i guess java.sql.timestamp used too) let application worry interpreting date-time fields - i.e. let user choose preferred time zone is ok? edit clarify - here meant refer timestamps created strictly on server (e.g.date-time of record creation). application server instantiates date objects (new date() equals current date-time on server, , time zone agnostic). if client user wants supply date searching/filte...
Read more