1.好久没用log4j了,转到logback好多年了,hbase程序运行时,报缺少log4j配置,那么,就转去logback吧(以下的XXX表示版本号)。
原先lib包里面有log4j-XXX.jar、slf4j-api-XXX.jar、slf4j-log4j12-XXX.jar,干掉log4j-XXX.jar和slf4j-log4j12-XXX.jar,加入jcl-over-slf4j-XXX.jar,log4j-over-slf4j-XXX.jar、logback-classic-XXX.jar、logback-core-XXX.jar,至于原先的slf4j-api这个包,可以顺手替换成新的,然后,加入一个logback.xml到classpath下
<?xml version="1.0" encoding="UTF-8"?> <!-- For assistance related to logback-translator or configuration --> <!-- files in general, please contact the logback user mailing list --> <!-- at http://www.qos.ch/mailman/listinfo/logback-user --> <!-- --> <!-- For professional support please see --> <!-- http://www.qos.ch/shop/products/professionalSupport --> <!-- --> <configuration scan="true" scanPeriod="120 seconds" debug="false"> <appender name="FileApp" class="ch.qos.logback.core.rolling.RollingFileAppender"> <!--See also http://logback.qos.ch/manual/appenders.html#RollingFileAppender--> <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>WARN</level> </filter> <File>log/HBaseClient.log</File> <encoder> <pattern>%d [%t] %-5p %c - %m%n</pattern> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <maxIndex>10</maxIndex> <FileNamePattern>log/HBaseClient.log.%i</FileNamePattern> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <MaxFileSize>1024KB</MaxFileSize> </triggeringPolicy> </appender> <appender name="ConApp" class="ch.qos.logback.core.ConsoleAppender"> <encoder> <pattern>%d [%t] %-5p %c - %m%n</pattern> </encoder> </appender> <root level="ERROR"> <appender-ref ref="ConApp"/> <appender-ref ref="FileApp"/> </root> </configuration>
这样,log4j和系统的日志系统就被logback接管了,至于为啥不用log4j,用logback,这个。。。自己去看吧,有空我可能会写点啥,logback有点小bug,就是有的linux系统下会找不到配置文件,以前修复过,不知道新版本有处理掉没。
2.Could not locate executable null\bin\winutils.exe in the Hadoop binaries
老掉牙的问题了,系统变量设置HADOOP_HOME,我还是不想去设置环境变量,还是一行代码来得快
System.setProperty("hadoop.home.dir", "G:/hadoop/hadoop-2.4.1");
3.程序运行正常,但是当log级别改为Info时,突然发现了这么一条
2014-08-04 16:19:22,942 [main-SendThread(Master:2222)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server Master/192.168.117.128:2222. Will not attempt to authenticate using SASL (java.lang.SecurityException: 无法定位登录配置)
看到个Exception,什么情况?觉得哪里不对,但是程序又是一切正常的。
搜索了一通,发现好多一样的问题,但是貌似和我的没啥关系,别人是程序跑不通,我是程序正常得很。
有说是zookeeper版本不一致的,我看了下hadoop和hbase中的确实有区别,一个zookeeper-3.4.5.jar,一个zookeeper-3.4.6.jar,但是我总觉得不对,新旧怎么可能会不兼容啊。那就直接统一了试试,连客户端程序的都统一了,结果那个info信息还是存在,就和不换是一样的,果然不是这里的问题!
有说是配置和hosts的,我这里都没这问题的,都配置过了的。加参数?我在配置中都写好的,直接打印一看就知道有了。
这只是一个info信息,很奇怪为啥是个info,照说info级别的都是不影响啥的信息,程序确实是很正常的跑,但是看到这个信息就是不舒服。
自己查代码去算了,根据日志,是在ClientCnxn.java中出现的信息,好吧,查看源码
private void logStartConnect(InetSocketAddress addr) { String msg = "Opening socket connection to server " + addr; if (zooKeeperSaslClient != null) { msg += ". " + zooKeeperSaslClient.getConfigStatus(); } LOG.info(msg); }
汗,还得换去查ZooKeeperSaslClient.java,继续吧
public ZooKeeperSaslClient(final String serverPrincipal) throws LoginException { /** * ZOOKEEPER-1373: allow system property to specify the JAAS * configuration section that the zookeeper client should use. * Default to "Client". */ String clientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client"); // Note that ‘Configuration‘ here refers to javax.security.auth.login.Configuration. AppConfigurationEntry entries[] = null; RuntimeException runtimeException = null; try { entries = Configuration.getConfiguration().getAppConfigurationEntry(clientSection); } catch (SecurityException e) { // handle below: might be harmless if the user doesn‘t intend to use JAAS authentication. runtimeException = e; } catch (IllegalArgumentException e) { // third party customized getAppConfigurationEntry could throw IllegalArgumentException when JAAS // configuration isn‘t set. We can reevaluate whether to catch RuntimeException instead when more // different types of RuntimeException found runtimeException = e; } if (entries != null) { this.configStatus = "Will attempt to SASL-authenticate using Login Context section ‘" + clientSection + "‘"; this.saslClient = createSaslClient(serverPrincipal, clientSection); } else { // Handle situation of clientSection‘s being null: it might simply because the client does not intend to // use SASL, so not necessarily an error. saslState = SaslState.FAILED; String explicitClientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY); if (explicitClientSection != null) { // If the user explicitly overrides the default Login Context, they probably expected SASL to // succeed. But if we got here, SASL failed. if (runtimeException != null) { throw new LoginException("Zookeeper client cannot authenticate using the " + explicitClientSection + " section of the supplied JAAS configuration: ‘" + System.getProperty(Environment.JAAS_CONF_KEY) + "‘ because of a " + "RuntimeException: " + runtimeException); } else { throw new LoginException("Client cannot SASL-authenticate because the specified JAAS configuration " + "section ‘" + explicitClientSection + "‘ could not be found."); } } else { // The user did not override the default context. It might be that they just don‘t intend to use SASL, // so log at INFO, not WARN, since they don‘t expect any SASL-related information. String msg = "Will not attempt to authenticate using SASL "; if (runtimeException != null) { msg += "(" + runtimeException + ")"; } else { msg += "(unknown error)"; } this.configStatus = msg; this.isSASLConfigured = false; } if (System.getProperty(Environment.JAAS_CONF_KEY) != null) { // Again, the user explicitly set something SASL-related, so they probably expected SASL to succeed. if (runtimeException != null) { throw new LoginException("Zookeeper client cannot authenticate using the ‘" + System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") + "‘ section of the supplied JAAS configuration: ‘" + System.getProperty(Environment.JAAS_CONF_KEY) + "‘ because of a " + "RuntimeException: " + runtimeException); } else { throw new LoginException("No JAAS configuration section named ‘" + System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") + "‘ was found in specified JAAS configuration file: ‘" + System.getProperty(Environment.JAAS_CONF_KEY) + "‘."); } } } }
根据上面的代码,可知问题在于红色部分,再注意一下上面的注释,我靠,果然只是一个普通的说明文字,但是,尼玛能不能不在info信息里面搞个Exception提示出来???
好吧,那确实是一个info信息,是无关紧要的info信息,其实只是说明了。。。。。由于客户端没用SASL,所以不使用SASL进行验证。。。
能不能换个好点的说明文字?我这些操作都是多余的了!人家就那样显示的,唉。看得不爽也没办法了,难道为了这点东西还去改源码不成?够汗的。
解决办法,客户端添加SASL信息,然后服务端也改掉,添加验证,这个官方文档http://hbase.apache.org/book.html#security有说明了,直接看去就是了,这里不说明了。
唉,处女座追求完美的心,果然是不变的。。。加上,看不到烦心的文字,舒坦了。。
Window中调试HBase问题小结,布布扣,bubuko.com