tugstugi / mongolian-bert Goto Github PK

$ python3 datasets/dl_and_preprop_mn_wiki.py

downloading https://dumps.wikimedia.org/mnwiki/20181220/mnwiki-20181220-pages-articles-multistream.xml.bz2...
1MB [00:00, 1081.56MB/s]
Traceback (most recent call last):
  File "wikiextractor/WikiExtractor.py", line 3238, in <module>
    main()
  File "wikiextractor/WikiExtractor.py", line 3228, in main
    args.compress, args.processes)
  File "wikiextractor/WikiExtractor.py", line 2847, in process_dump
    for line in input:
  File "/usr/local/lib/python3.6/fileinput.py", line 250, in __next__
    line = self._readline()
  File "/usr/local/lib/python3.6/fileinput.py", line 364, in _readline
    return self._readline()
  File "/usr/local/lib/python3.6/bz2.py", line 219, in readline
    return self._buffer.readline(size)
  File "/usr/local/lib/python3.6/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/local/lib/python3.6/_compression.py", line 103, in read
    data = self._decompressor.decompress(rawblock, size)
OSError: Invalid data stream
tmp_mn_wiki/*/*.txt
done!