Create a package hieararchy about neuralmonkey HOT 15 CLOSED

I have already started working on parts of this issue (1 and 2).

from neuralmonkey.

First thing we should do is to move all the python code in directory neuralmonkey, so the python package is separated from all the helper scripts and has correct name even if you clone the repository into a directory with different name (eg. mmmt). But I'm not sure if this will not break anything. Maybe we should just proceed and see what happens, @jindrahelcl, @jlibovicky?

from neuralmonkey.

It should not break anything, but..

I would do this as the last step when all the files that should belong inside a package will be inside one. Then, moving all the packages into another one should be easier.

I would leave out only train.py and run.py to be in the root directory of the package. Other non-related scripts, such as estimate_scheduled_sampling.py would go to a tools dir.

from neuralmonkey.

I guess train.py and run.py should be merged into one neuralmonkey executable provided by the package. I see no harm in having modules directly under neuralmonkey, eg. neuralmonkey.evaluation.

from neuralmonkey.

putting __init__.py in the root directory and not doing anything else and push it to the repository is really not the way to achieve this. :)

from neuralmonkey.

I agree. That's why I opened this. But it's a (temporary) way to run tests/python/test_vocabulary.py, without the extremely ugly relative import from higher directory.

from neuralmonkey.

No no no it is a way to break the code from running because if you put __init__.py to the root, every package changes its name to neuralmonkey.*

from neuralmonkey.

OK. We need to do that anyway. Why don't we just put everything (I mean all the python source code) inside neuralmonkey directory in the repo and correct all the imports?

from neuralmonkey.

As I said, I would wait for this change until we have optimized the inner structure. I need to be able to run this. If you want to do it, start a branch and do it locally, then test it (and try also actually run it) and then merge it. Otherwise, I will do this gradually over this and next week.

from neuralmonkey.

What's the difference in optimizing the inner structure after we move from root to neuralmonkey? I could do it locally and run it, but since we have no common tests, will that mean that you will be able to run it too?

from neuralmonkey.

Yeah, it should. I don't use code that is not commited, so if you manage to run it on some small data (preferably using the translation-example.ini), then it will work here as well.

The difference is nearly none, but I am working on it side-by-side with other things. If every name of every class changes, I have to change all the ini files I use (not mentioning fixing all the bugs), which is not what I want to spend my time with right now.

from neuralmonkey.

If I understand things correctly, translation-example.ini does not work in master (who's the troll now?), so it's not really possible to test my changes with it. Can you please make it work?

from neuralmonkey.

Try it now, i updated the dataset series names

from neuralmonkey.

@tomasmcz How am I supposed to run the training script now?

this:
$ python -u neuralmonkey/train.py experiments/tiny.bpe.ini

gives

Traceback (most recent call last):
File "neuralmonkey/train.py", line 15, in
from neuralmonkey.checking import check_dataset_and_coders
ImportError: No module named neuralmonkey.checking

the same goes for the case i run the train.py directly from the neuralmonkey dir

from neuralmonkey.

python -u -m neuralmonkey.train whatever.ini

We should probably document this somewhere. Or maybe find a better way to run this. I don't really understand Python packages, so I don't know how is this usually done.

from neuralmonkey.