Hello, wondering if it's on your roadmap to have JSON and Image Extraction features?

{ "response": { "title": null, "content": " \nThe third characteristic of web page writing ....\n ", "source": "https://www.york.ac.uk/teaching/cws/wws/webpage4.html", "images": [ { "src": "https://www.york.ac.uk/teaching/cws/wws/toofar.gif", "width": null, "height": null, "alt": "some alt" } ] }, "status": "success" }

Hi,

I'm currently using boilerpipe.
I'm trying to switch to this new package, GoLang based which looks maintained.

I've downloaded the code,
did go get for the relevant packaages, and did go build main.go

then ran
main -http :8080

when navigating there and trying any url i get

Get https://dealbook.nytimes.com/2006/09/12/big-names-pull-cash-from-venture-capital-fund-over-political-contributions?_r=4: stopped after 10 redirects

Any idea?

Thanks.

When I run this:
$ go install github.com/jlubawy/go-boilerpipe/cmd/boilerpipe

I get:

can't load package: package github.com/jlubawy/go-boilerpipe/cmd/boilerpipe: cannot find package "github.com/jlubawy/go-boilerpipe/cmd/boilerpipe" in any of:
        /usr/local/go/src/github.com/jlubawy/go-boilerpipe/cmd/boilerpipe (from $GOROOT)
        /Users/bodell/go/src/github.com/jlubawy/go-boilerpipe/cmd/boilerpipe (from $GOPATH)

I'm a Go newbie. Maybe I need to do more to set it up on my machine?

can't load package: package http:/github.com/jlubawy/go-boilerpipe/cmd/boilerpipe: cannot find package "http:/github.com/jlubawy/go-boilerpipe/cmd/boilerpipe" in any of:

Is there no interface for extracting HTML,
I need to get html. Is there an excuse?

I haven't dug into the code but when I try to parse websites with nested anchor elements I get the input contains nested <a> elements source link.

Why does this cause an issue, and is there any workarounds for this?

Hi There!
What would be best way to use it in the project where I have bunch of web pages in []byte form.

Something like

b, _ := boilerpipe.ExtractFromBytes([]byte{"Some single html page here"}) 
fmt.Println(b.Content)

I did some "cowboy shortcut"

func boiler(b []byte) string {
	r := bytes.NewReader(b)
	doc, err := boilerpipe.ParseDocument(r)
	if err != nil {
		log.Fatal(err)
	}
	boilerpipe.ArticlePipline.Process(doc)
	return doc.Content()
}

Works good so far:-)

First
Second

When I tested the code with these two url, I faced with a problem.
The problem is these two page contains same contents but the code returns different results.
I also tested with base boilerpipe library with Java and Python also but they produce correct results.

If you want me to share some images.. I can.

Thank you

○ go get github.com/jlubawy/go-boilerpipe/cmd/boilerpipe

github.com/jlubawy/go-boilerpipe

gocode/src/github.com/jlubawy/go-boilerpipe/actions.go:173:2: undefined: atom.Menuitem