Hi,

I'm trying to get NHtmlUnit working to do some basic web scraping on a website. It works fine on the local end, but when deployed onto a server (Windows Azure) I get the following error

'Address family not supported by protocol family: connect'

when it tries to execute .getPage on a WebClient.

Any idea why?

Greatly appreciate any help, thanks.

With the same browser setting, HtmlUnit took 8 seconds to navigate, but with NHtmlUnit it took 220 seconds,
Any ideas? Am i wrong somewhere?

Here is my setting:

webClient = new WebClient(BrowserVersion.CHROME);
webClient.Options.JavaScriptEnabled = true;
webClient.Options.CssEnabled = false;
webClient.Options.AppletEnabled = false;
webClient.Options.Timeout = 30000;
webClient.Options.RedirectEnabled = true;
webClient.Options.ThrowExceptionOnFailingStatusCode = false;
webClient.Options.ThrowExceptionOnScriptError = false;

Navigate

page = WebBrowserHelper.Instance.navigate("http://vnraovat.net/register");

Thanks,

I am trying to use the AlertHandler as follows:

        java.util.List collectedAlerts = new ArrayList();
        webClient.WObj.setAlertHandler(new CollectingAlertHandler(collectedAlerts));

The problem I am having is that I getting this error in my Weclient Object –

webClient.AlertHandler' threw an exception of type 'System.InvalidOperationException
{"Operation is not valid due to the current state of the object."}

What am I missing.

Sam

In java, HtmlTextArea.SetText() will set content of textarea. But in .NET, there are no SetText() method in HtmlTextArea.
I am using NHtmlUnit 2.15, how can I set content of textarea?
Do I miss something?

Thanks,

I have read the code but can't find out how to submit
How to submit form with input type submit ?
I am use C#

Thanks

I've had this code for 2+ years - with the most recent Windows 10 upgrade (thanks Microsoft) I was getting some odd IKVM errors, so I finally upgraded to the latest IKVM & NHtmlUnit, and now this line below throws the above error.

Task task = Task.Run(() => webClient.GetHtmlPage(uri.ToString()), cts.Token);

Is this a bug in NHtmlUnit?
Thanks!

This project won't receive any form of updates (at least not from me) until IKVM is ported to .NET 5 (or compatible). See windward-studios/ikvm8#3 and ikvmnet/ikvm#20 for updates on that front.

I've tried a simple piece of code that interacts with Uber's new OAuth 2.0 flow:

var url = "https://login.uber.com/oauth/authorize?response_type=code&client_id=MYCLIENTID&redirect_uri=http%3a%2f%2flocalhost%3a2497%2fLogin%2fCallback"
var webClient = new WebClient(BrowserVersion.CHROME);

webClient.Options.JavaScriptEnabled = true;
webClient.Options.ActiveXNative = true;
webClient.Options.CssEnabled = true;

var page = webClient.GetHtmlPage(url);

However, when it runs, I get the following exception:

An exception of type 'com.gargoylesoftware.htmlunit.ScriptException' occurred in HtmlUnit.dll but was not handled in user code

Additional information: DomNode has not been set for this SimpleScriptable: com.gargoylesoftware.htmlunit.javascript.host.html.HTMLFormElement

Based on a cursory look, it seems that this may be a problem with the underlying HtmlUnit assembly? I'd appreciate it if anyone could provide some input. Thanks!

So I tried to call WebRequest.WObj.setRequestParameters but can't figure out what to pass as arguments since com.gargoylesoftware.htmlunit.WebRequest is expecting java.Util.List

Attempting to set HttpMethod in a WebRequest requires creating a new HttpMethod object via the wrapper constructor, passing the native enum value to it. Couldn't it be made easier, hard-coding the equivalent values into the wrapper?

To have a publicly available build on Linux, we should set up a build on Travis. I would do this myself if I could, but only an administrator of @HtmlUnit are allowed to create builds, so we need help from @asashour or someone else with the rights to do it.

IKVM 7.4 Release Candidate 0 has been released. An upgrade to the latest version would be nice.

I'm trying to write a functional test for an MVC5 + Web API project where I log in (via web forms), and then make a POST call directly to the API route. When I make the call to client.GetPage, I get an InvalidOperationException with this message: No creator found for com.gargoylesoftware.htmlunit.UnexpectedPage

This was reported against HtmlUnit itself on SO about five years ago.

var client = new WebClient(BrowserVersion.CHROME);
 var method = new HttpMethod(com.gargoylesoftware.htmlunit.HttpMethod.POST);
var json = JsonConvert.SerializeObject(inputObject);

var webRequest = new WebRequest(new java.net.URL("http://localhost/site/api/game"), new HttpMethod(method))
{
    RequestBody = "=" + json // WebAPI expects this. Don't ask me why.
};
var page = client.GetPage(webRequest);

I can see the call hit the server side (including the response object which comes back) -- it's not throwing an exception there, only on the client. The response is pure JSON:

{
    "Data": [
        {
            "Id": "00000000-0000-0000-0000-000000000000",
            "Name": "FT-FunctionalTestGame",
            "OwnerId": 3
        }
    ],
    "Errors": null
}

I try to process web pages with mathml using mathjax via nhtmlunit

In java (using htmlunit) everything is fine.
I use the next snippet:

WebClient webClient = new WebClient(BrowserVersion.CHROME);
HtmlPage page = webClient.getPage(new File(mathml).toURI().toURL());
webClient.waitForBackgroundJavaScript(300000 * 1000);
String pageAsXml = page.asXml();

and the next html - https://pastebin.com/hvJZmtnH
The result is quite good : https://pastebin.com/kHzrM3rJ

However it's not so good in C# (NHtmlUnit):

The code i use:

WebClient webClient = new WebClient(BrowserVersion.CHROME);
webClient.Options.JavaScriptEnabled = true;
webClient.Options.Timeout = 30000;
webClient.Options.RedirectEnabled = true;
HtmlPage page = webClient.GetHtmlPage(mathml);
webClient.WaitForBackgroundJavaScript(50000000);
string pageAsXml = page.AsXml();

The result: https://pastebin.com/E5uXfUZ1

As you can see the javascript wasn't processed.

Please confirm that it's impossible to process such scripts via NHtmlUnit or point me out how to fix the issue.

If I try to right-click on a Telerik RadTreeNode using NHtmlUnit, nothing happens. Please help. I can send you an example project if you send me the email to send it to.

I'm right-clicking on 'Amazing Weekend Package' in the attached screenshot. It normally brings up a context menu.

homePage.GetFirstByXPath("//span[text()='Clinical']/parent::a") returns non wrapped HtmlAnchor

For example I want to implement an AttachmentHandler. Strightforwardly I tought I only need to implement IAttachmentHanlder, but it also seems I'll need to derive ObjectWrapper in order to implement WrappedObject. But ObjectWrapper has a one-argument c'tor.

So how should I implement such interfaces in my code?

    class AttachmentHandler : ObjectWrapper, NHtmlUnit.Attachment.IAttachmentHandler
    {
        public void HandleAttachment(IPage p)
        {
            Console.WriteLine("HAHAHAHAHA *****************************************************");
        }
    }

see security issue
https://nvd.nist.gov/vuln/detail/CVE-2020-5529

including NuGet

To have a publicly available build on Windows, we should set up a build on AppVeyor. I would do this myself if I could, but only an administrator of @HtmlUnit are allowed to create builds, so we need help from @asashour or someone else with the rights to do it.

System.MissingMethodException was unhandled by user code
HResult=-2146233069
Message=Method not found: 'Void java.util.HashMap.__()'.
Source=HtmlUnit
StackTrace:
at com.gargoylesoftware.htmlunit.Cache..ctor()
at com.gargoylesoftware.htmlunit.WebClient..ctor(BrowserVersion browserVersion)
at NHtmlUnit.WebClient..ctor(BrowserVersion browserVersion)
at [our_project].FWWebClient.ClientWrapper..ctor(BrowserVersion version) in [our_project]\FWWebClient\ClientWrapper.cs:line 14
at [our_project].FWWebClient.WebClientPool..ctor() in [our_project]\FWWebClient\WebClientPool.cs:line 16
at [our_project]...
InnerException:

This error occurs after IKVM.NET update from 7.4.5196 to at least 8.0.5449 (the next one).
Using this version of IKVM.NET is critical for us because otherwise NHtmlUnit doesn' work with HTTPS in some cases due to 'Could not generate DH keypair' error at com.gargoylesoftware.htmlunit.WebClient.download(WebWindow requestingWindow, String target, WebRequest request, Boolean isHashJump, String description). As far as we can see, this is due to an obsolete Java version, that is, IKVM.NET 7.4.5196 is too old. (Indeed, IKVM.NET 8.0.5449 is more than a year old).
Could you please include the IKVM 8.0.5449 support?

Thanks in advance,
Regards, L.B.

So I hit an exception while getting page using WebClient. But the exception information is nearly useless. I believe the .Net wrapper somehow wraps the exception but discarded the useful diagnostic information.

This severely limits the diagnosability. Please fix.

System.InvalidOperationException was unhandled by user code
  HResult=-2146233079
  Message=Operation is not valid due to the current state of the object.
  Source=NHtmlUnit
  StackTrace:
       at NHtmlUnit.ObjectWrapper.CreateWrapper[T](Object wrappedObject)
       at NHtmlUnit.WebClient.GetPage(String url)
       at TestProject.TestProject.TryDownloadOne(DownloadSpec spec, FlexUri uri, String& content) in c:\Users\myuser\Documents\Visual Studio 2013\Projects\TestProject\TestProject\TestProject.cs:line 101
       at TestProject.TestProject.DownloadAll() in c:\Users\myuser\Documents\Visual Studio 2013\Projects\TestProject\TestProject\TestProject.cs:line 52
       at TestProjectTestCases.UnitTest1.TestMethod1() in c:\Users\myuser\Documents\Visual Studio 2013\Projects\TestProject\TestProjectTestCases\UnitTest1.cs:line 19
  InnerException: 

I have no idea why, but neither Travis nor AppVeyor is running our builds, so even though #37 is merged, tagged and released, there won't be an automatic push of the .nupkg to NuGet.org. We should just delete appveyor.yml and .travis.yml and set up auto-publishing to NuGet.org with GitHub Actions instead.

Hello,

I am facing error while connecting website running with TLS1.2 protocol only.

following is the error:

Received fatal alert: handshake_failure

at org.apache.http.impl.execchain.RetryExec.execute(HttpRoute route, HttpRequestWrapper request, HttpClientContext context, HttpExecutionAware execAware)

 at org.apache.http.impl.execchain.RedirectExec.execute(HttpRoute route, HttpRequestWrapper request, HttpClientContext context, HttpExecutionAware execAware)

 at org.apache.http.impl.client.InternalHttpClient.doExecute(HttpHost , HttpRequest , HttpContext )

 at org.apache.http.impl.client.CloseableHttpClient.execute(HttpHost target, HttpRequest request, HttpContext context)

 at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(WebRequest request)

 at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebRequest , Int32 )

 at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebRequest webRequest)

 at com.gargoylesoftware.htmlunit.WebClient.getPage(WebWindow webWindow, WebRequest webRequest)

 at com.gargoylesoftware.htmlunit.WebClient.getPage(URL url)

 at com.gargoylesoftware.htmlunit.WebClient.getPage(String url)

 at NHtmlUnit.WebClient.GetPage(String url)

 at NHtmlUnit.WebClient.GetHtmlPage(String url)

Dear All

I like HtmlUnit, it’s fast once started and accurate. However my team use it for smoke testing and like to deploy the tools to test with the package. The IKVM makes this very heavy. Has there ever any thought into totally porting this project into .net code and not relying on the IKVM runtime and the supporting DLLs? Currently the project when built is 53 meg, weighty you would agree.

Thanks