The act-scio2 from mnemonic-no

KeyError when trying to run scio-analyze on stdin

I'm currently trying to get act-scio2 to work a number of text documents, following the README instructions. The installation of the various packages went well, but I do get an error when running scio-analyze on some stdin input like in the provided example:

$ echo "The companies in the Bus; Finanical, Aviation and Automobile industry are large." | scio-analyze --beanstalk=
[2020-06-24 12:00:19] app=scio-analyze level=INFO msg=Waiting for work on stdin
[2020-06-24 12:00:19] app=scio-analyze level=ERROR msg=Got LookupError. If nltk data is missing, run scio-nltk-download, which should download all nltk data to ~/nltk_data.
[2020-06-24 12:00:19] app=scio-analyze level=ERROR msg=Task exception was never retrieved
future: <Task finished coro=<async_main() done, defined at /nr/samba/user/plison/anaconda3/lib/python3.7/site-packages/act/scio/analyze.py:98> exception=KeyError('threat_actor')>
Traceback (most recent call last):
  File "/nr/samba/user/plison/anaconda3/lib/python3.7/site-packages/act/scio/analyze.py", line 128, in async_main
    await task
  File "/nr/samba/user/plison/anaconda3/lib/python3.7/site-packages/act/scio/analyze.py", line 75, in analyze
    await asyncio.gather(*tasks)
  File "/nr/samba/user/plison/anaconda3/lib/python3.7/site-packages/act/scio/plugins/threatactor_pattern.py", line 28, in analyze
    ini['threat_actor']['alias'] = os.path.join(self.configdir, ini['threat_actor']['alias'])
  File "/nr/samba/user/plison/anaconda3/lib/python3.7/configparser.py", line 958, in __getitem__
    raise KeyError(key)
KeyError: 'threat_actor'

Any idea how to fix this?

Performance of NLP modules

This is more a suggestion than an actual issue, but I see that much of the processing time is spent on running NLP modules from NLTK. I would actually recommend to drop NLTK altogether -- it's now a quite outdated piece of software, and it was never developed for processing large document bases (it was actually primarily meant for educational purposes, as part of NLP courses).

I am a big fan of spacy (www.spacy.io). It's very fast, reliable, contains all the standard NLP modules you might need (tokeniser, POS tagger, lemmatiser, NER, parser, etc.) and has a developer-friendly interface. The accuracy of these NLP modules is also definitely better than NLTK (spacy relies on deep learning models). Plus, spacy it now supports a dozen languages or so. Give it a try and you will never look back at NLTK :-)

mnemonic-no / act-scio2 Goto Github PK

act-scio2's People

Contributors

Stargazers

Watchers

Forkers

act-scio2's Issues

KeyError when trying to run scio-analyze on stdin

Performance of NLP modules

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent