htmlunit out of memoery error:java heap space

程序            WebClient webClient = new WebClient(BrowserVersion.CHROME);
		webClient.getOptions().setJavaScriptEnabled(false);
		webClient.getOptions().setCssEnabled(false);
		HtmlPage page;
		try {
			page = webClient.getPage("http://www.taolife.com.cn/");
			System.out.print(page.asText());
		} catch (FailingHttpStatusCodeException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (MalformedURLException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

bug

1Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.lang.Class.getDeclaredFields0(Native Method)
	at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
	at java.lang.Class.getField0(Class.java:2975)
	at java.lang.Class.getField(Class.java:1701)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.configureConstants(JavaScriptEngine.java:612)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.init(JavaScriptEngine.java:271)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.access$000(JavaScriptEngine.java:105)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$1.run(JavaScriptEngine.java:173)
	at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:628)
	at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:513)
	at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.initialize(JavaScriptEngine.java:184)
	at com.gargoylesoftware.htmlunit.WebClient.initialize(WebClient.java:1008)
	at com.gargoylesoftware.htmlunit.WebWindowImpl.setEnclosedPage(WebWindowImpl.java:135)
	at com.gargoylesoftware.htmlunit.html.FrameWindow.setEnclosedPage(FrameWindow.java:105)
	at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:221)
	at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:199)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:272)
	at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:160)
	at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:476)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:350)
	at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:184)
	at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:122)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1945)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:227)
	at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:483)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:350)
	at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:184)
	at com.gargoylesoftware.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:122)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1945)
	at com.gargoylesoftware.htmlunit.html.HtmlPage.initialize(HtmlPage.java:227)
	at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:483)
	at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:350)

修改运行参数:-Xms1024m -Xmx1024m,依旧报错。

修改

webClient.getOptions().setJavaScriptEnabled(false);

程序正常运行;

分析:

估计htmunit采用的javascript解析器存在问题,陷入死循环耗尽内存;

因项目时间问题,暂时采集true方案,回头用内存监控工具 jconsole或者eclipse memoery analyzer找出是哪个模块出问题。

时间: 2024-11-09 10:05:39

htmlunit out of memoery error:java heap space的相关文章

Error: Java heap space

在chd中 的hive中执行 (select count (*))  或者mr程序都报Error: Java heap space 4.io.sort.mb 的作用  排序所使用的内存数量.  默认值:100M,需要与mapred.child.java.opts相配 默认:-Xmx200m.  不能超过mapred.child.java.opt设置,否则会OOM. 调整mapred.child.java.opts的大小没有效果,调整set mapreduce.task.io.sort.mb=10

HIVE Error : Java heap space 解决方案

Java heap space问题一般解决方案: 设置 set io.sort.mb=10; 排序所使用的内存数量,默认值是100M,和mapred.child.java.opts相对应,opts默认:-Xmx200m,则mb不能超过200M,否则会OOM. 设置 set hive.map.aggr=true; 是否在 Map 端进行聚合,默认为True,会在map端进行部分数据端聚合操作. (可选)hive.groupby.mapaggr.checkinterval =100000; Map端

Hadoop on Mac with IntelliJ IDEA - 5 解决java heap space问题

本文讲述在CentOS 6.5中提交作业到hadoop 1.2.1于reduce阶段遇到Error: java heap space错误导致作业重新计算的解决过程.解决办法适用Linux.Mac OS X 和Windows操作系统. 环境:Mac OS X 10.9.5, IntelliJ IDEA 13.1.4, Hadoop 1.2.1 Hadoop放在虚拟机中,宿主机通过SSH连接,IDE和数据文件在宿主机.IDEA自身运行于JDK 1.8,IDEA工程及Hadoop使用JDK 1.6.

Java heap space cdh 5.11.1

在执行hive count 查询的时候报错:Error: Java heap space 解决办法是 set io.sort.mb=10; 执行hadoop的Exeample的时候报错,也是java heap space 的问题 Diagnostic Messages for this Task:Error: Java heap space FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.

Five Steps to Avoiding Java Heap Space Errors

来自:https://www.mapr.com/blog/how-to-avoid-java-heap-space-errors-understanding-and-managing-task-attempt-memory#.VMWvNDGUfXY Keeping these five steps in mind can save you a lot of headaches and avoid Java heap space errors. Calculate memory needed. C

排查sqoop报错:Error running child : java.lang.OutOfMemoryError: Java heap space

报错栈: 2017-06-16 19:50:51,002 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: 1=1 AND 1=1 2017-06-16 19:50:51,043 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: 1=1 AND 1=1 2017-06-16 19:50:51,095 INFO [mai

eclipse里报:An internal error occurred during: "Building workspace". Java heap space(内存溢出)

当在eclipse中的web工程中增加了extjs4,出现An internal error occurred during: "Building workspace".Java heap space 错误. 解决办法1:把extjs4的工程安装包不要放到web工程中或者建议直接用myeclipse直接导入的exjts4包,就没问题. 解决办法2:导入ExtJS包,这样会卡死eclipse的.去到工程根目录下,找到.project,用记事本打开,把两处删除掉: 第一处: <bui

MyEclipse for Spring启动时报错&quot;An internal error occurred during: &#39;Updating indexes&#39;.Java heap space&quot;的解决办法

问题 MyEclipse for Spring在启动时,报如下错误:An internal error occurred during: 'Updating indexes'.Java heap space 解决办法 对于这种问题,可以采用禁止MyEclipse的updating indexes的方法来解决. Window -> Preferences -> Myeclipse -> Maven4Myeclipse,禁用Download repository index updates

解决Scala异常处理java.lang.OutOfMemoryError: Java heap space error

需求:百万.千万.4千万级日志对设备进行除重环境:设备内存64G,scala单机版运行shell文件日志:20G 48000000.log4.0G 10000000.log396M 1000000.log 代码如下 帮助 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 import scala.io.Source import scala.collection.mutable.ArrayBuffer var te