alexandres / lexvec Goto Github PK

Sorry if this is a silly question, though can you provide any references to assist in making use of these output vectors in Go?

Hello! Is there a way to convert all the files of embeddings in bin format?

Hi Alexandre,

Is there a way to embed you application as lib in Go for POS Tagger actions within a Go Web Application?

Thanks!

I'm experimenting with building lexvec and running it.

• go get github.com/alexandres/lexvec
• go build

./lexvec -threads 64 -corpus 59m.txt -output lexvectors
.....
cds total: 67097075.384447

calculating ppmi matrix

create vectors

running lexvec
panic: nan

goroutine 102 [running]:
main.learn(0xc420210a20, 0xc447a48b40, 0xc424326d70, 0x5f7720, 0xc42132e570, 0xc424648000, 0x0, 0xc4a465a000, 0x12c, 0x12c, ...)
/var/lib/jenkins/workspace/lexvec/lexvec.go:319 +0x4a0
main.main.func3(0xc4200ce330, 0xc4200ce370, 0xc4200ce310, 0xc4200ce2a0, 0xc6294e9980, 0xc449fb4200, 0x40, 0x40, 0xc449fb4000, 0x40, ...)
/var/lib/jenkins/workspace/lexvec/lexvec.go:1022 +0x1929
created by main.main
/var/lib/jenkins/workspace/lexvec/lexvec.go:1056 +0x2f22

when i install this package,i used make,one error occurred.pls help

CC="clang" CGO_CFLAGS="-Ofast -std=gnu99 -march=native" go build --ldflags '-extldflags "-static"' -o lexvec
# github.com/alexandres/lexvec
/usr/local/Cellar/go/1.11.2/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)

make: *** [optimal] Error 2

got this error with model=2.
can i get model=2 vectors by the outputs produced in model=0, i.e by adding position context vectors to get W_hat_pos and then add to W?

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x4ca33f]

goroutine 1 [running]:
main.mergeContextVectors()
.../lexvec/model.go:117 +0x15f
main.saveVectors()
.../lexvec/model.go:65 +0x455
main.main()
.../lexvec/main.go:211 +0x13ef

Requesting an updated version of LexVec with Common Crawl 2017,
with 3 million most frequent words to be competitive with GloVe.
Bonus: Common Crawl 2015~2017 to get 100B tokens (like word2vec)