derang (گرفته شده از دِرنگنما), diacritics creation for Persian/Farsi language
python libs/processor.py --config config/alef/alef.json
python libs/processor.py --config configs/alef.json datasets/sentences1.txt datasets/sentences2.txt --output-dir datasets/processed-data
python libs/processor.py --config configs/alef.json datasets/sentences1.txt datasets/sentences2.txt --output-dir datasets/processed-data
python libs/processor.py --config configs/alef.json datasets/sentences1.txt --output-dir datasets/processed-data --n-val 1 --n-test 1
run the following command to check list of input/target symbols, and how is the input/target symbol to id mapping.
python libs/encoder.py
run the following to start the training process, preprocessing using processor.py should have been done earlier
python libs/trainer.py --config configs/alef.json --accelerator cpu
inference, using the latest model saved under the logs/alef/lightning_logs/
python libs/inference.py --config configs/alef.json --text "سلام سلام"