Restructure entry point about nlp-cube HOT 4 CLOSED

I think we should switch from optparse to argparse in the near future. Also, for tokenization and (optional) sentence-splitting tasks, we need separate params for train/dev/test raw-text files (eg. --raw-train-file) to be able to evaluate tok and ss at train-time. I'll look into it.

from nlp-cube.

So the argparse suggestion is pretty clear. But I have to ask: should we create separate entry points for train, test (which I would actually rename to "run" or "process") and runserver ?

from nlp-cube.

What I think we should do is thinking like the users and not like the developers. If I was a user, I'd like an entry point with something like:
-train
-task = tok / lemma / etc. (which task should we create a model for)
-train&dev conllu files (test is ok if we want to test also, but optional)
-dev txt raw-file (so we can train the tok and ss), test txt raw-file (optional, if we want to get results on test)
-other params like config (which should have ALL config sections for ALL tasks), base folder path, batch size, etc

-run (run is better than test)
-a config file OR a parameter that specifies the task chain, like : ss,tok,tag,lemma,parse so we can run only required components
-raw text file (if ss/tok are specified in the processing chain) OR conllu file if not
-output conllu file
-other params like a config file having ALL config sections in a single file), base to load model, batch size, etc.

Also we should have a whitespace tokenizer (dumb tokenizer) for people that already have their input tokenized (this should be the default if no tokenizer is specified in the task pipeline).

from nlp-cube.

We could have two entry points to drop the train/run parameters.
Also argparse works for python>=2.7

from nlp-cube.