susyustc / mathtranslate Goto Github PK

View Code? Open in Web Editor NEW

980.0 980.0 65.0 32.58 MB

translate scientific papers in latex, especially arxiv papers

Home Page: https://github.com/SUSYUSTC/MathTranslate

License: Apache License 2.0

Python 83.45% Shell 0.48% TeX 2.06% kvlang 6.32% Batchfile 0.20% CSS 0.58% JavaScript 0.23% HTML 6.68%

chinese english equations language latex math paper pdf python symbols translate translation

mathtranslate's People

Stargazers

Watchers

mathtranslate's Issues

using web, there is an issue:Invalid Arxiv ID.

但是这篇文章确实存在：

mis-translate latex commands to Chinese

使用 mathpix 将 PDF 转换成 latex 后，通过 Web Server 上传 zip 进行翻译，结果排版格式会有问题。

\rule{\linewidth}{0.2pt} is bad

Step to reproduce the behavior

http://mathtranslate.xyz/
Arxiv ID: 2302.06873
result
PDF generation failed. Please try on Overleaf.
I am not familiar with the syntax of latex, please help me solve it.
Minimum steps to reproduce

0-main.tex

\documentclass{article}

\begin{document}

\begingroup
\renewcommand\thefootnote{}\footnote{
\noindent
\rule{\linewidth}{0.2pt}
}\addtocounter{footnote}{-1}\endgroup

\end{document}

translate_tex 0-main.tex --debug --nocache
cat 0-main.tex

\documentclass{article}
\usepackage{xeCJK}
\usepackage{amsmath}

\begin{document}

  

   \begingroup   
    \renewcommand       \thefootnote{}       \footnote{   \noindent   
    \rule{\linewidth}     { 0.2点   }   }       \addtocounter{footnote}     { 
-1   }     \endgroup     


  
\end{document}

cat text_old


Paragraph 0





Paragraph 1





Paragraph 2


 XMATHX_4 
 XMATHX_5 XMATHX_0 XMATHX_1 XMATHX_2 XMATHX_3 XMATHX_6 


Paragraph 2


-1


Paragraph 2



 XMATHX_2 
 XMATHX_0 XMATHX_1 



Paragraph 2


0.2pt

cat text_new



Paragraph 0






Paragraph 1






Paragraph 2


XMATHX_4
 XMATHX_5 XMATHX_0 XMATHX_1 XMATHX_2 XMATHX_3 XMATHX_6


Paragraph 2



-1


Paragraph 2


XMATHX_2
 XMATHX_0 XMATHX_1


Paragraph 2


0.2点

cat objs



Paragraph 0




Paragraph 1




Paragraph 2


obj 0
 \thefootnote{} 
obj 1
 \footnote{
\noindent
\rule{\linewidth}{0.2pt}
} 
obj 2
 \addtocounter{footnote} 
obj 3
 {-1} 
obj 4
 \begingroup 
obj 5
 \renewcommand 
obj 6
 \endgroup 


Paragraph 2




Paragraph 2


obj 0
 \rule{\linewidth} 
obj 1
 {0.2pt} 
obj 2
 \noindent 


Paragraph 2

RecursionError: maximum recursion depth exceeded while calling a Python object

case:https://arxiv.org/abs/1904.09237

74%|████████████████████████████▊ | 34/46 [00:00<00:00, 257412.16it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 256, in translate_full_latex
latex_translated_paragraphs = list(tqdm.auto.tqdm(executor.map(self.worker, latex_original_paragraphs), total=len(latex_original_paragraphs)))
File "/usr/local/lib/python3.7/site-packages/tqdm/std.py", line 1166, in iter
for obj in iterable:
File "/usr/local/Cellar/[email protected]/3.7.12/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
yield fs.pop().result()
File "/usr/local/Cellar/[email protected]/3.7.12/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/usr/local/Cellar/[email protected]/3.7.12/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/local/Cellar/[email protected]/3.7.12/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 203, in worker
raise e
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 193, in worker
latex_translated_paragraph = self.translate_paragraph_latex(latex_original_paragraph)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 172, in translate_paragraph_latex
latex_translated_paragraph = self.translate_text_in_paragraph_latex_and_leading_brace(latex_original_paragraph)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 167, in translate_text_in_paragraph_latex_and_leading_brace
latex_translated_paragraph = self.translate_text_in_paragraph_latex(latex_original_paragraph)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 144, in translate_text_in_paragraph_latex
result += self._translate_text_in_paragraph_latex(split) + ' ' + sep + ' '
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate.py", line 119, in _translate_text_in_paragraph_latex
text_original_paragraph = process_text.split_too_long_paragraphs(text_original_paragraph)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/process_text.py", line 47, in split_too_long_paragraphs
par2 = split_too_long_paragraphs('.'.join(lines[position:]))
File "/usr/local/lib/python3.7/site-packages/mathtranslate/process_text.py", line 47, in split_too_long_paragraphs
par2 = split_too_long_paragraphs('.'.join(lines[position:]))
File "/usr/local/lib/python3.7/site-packages/mathtranslate/process_text.py", line 47, in split_too_long_paragraphs
par2 = split_too_long_paragraphs('.'.join(lines[position:]))
[Previous line repeated 982 more times]
File "/usr/local/lib/python3.7/site-packages/mathtranslate/process_text.py", line 42, in split_too_long_paragraphs
first_words = [get_first_word(line) for line in lines]
File "/usr/local/lib/python3.7/site-packages/mathtranslate/process_text.py", line 42, in
first_words = [get_first_word(line) for line in lines]
File "/usr/local/lib/python3.7/site-packages/mathtranslate/process_text.py", line 26, in get_first_word
words = line.split(' ')
RecursionError: maximum recursion depth exceeded while calling a Python object

pip安装时的警告

pip安装时生成了如下警告，知悉后即可关闭此issue：

  DEPRECATION: mtranslate is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

macOS的版本GUI无法打开

提示信息：未能打开文稿“MathTranslate_MacOS”。文本编码Unicode (UTF-8)不适用。
我的macOS版本是13.1 (22C65)

有部分论文无法正常翻译，翻译出的内容很少

2012.07421v3

翻译出来只保留了少部分英文，大部分内容都丢失了

HTTP Error 503+PermissionError: [WinError 32] +PermissionError: [WinError 5]+RecursionError

here are some error:

HTTP Error 503
PermissionError: [WinError 32]
PermissionError: [WinError 5]
RecursionError

This is the result of executing:

translate_arxiv 2205.15510
The current mathtranslate is latest
Start
engine google
language from en
language to zh-CN

arxiv number: 2205.15510

temporary directory C:\Users\29927\AppData\Local\Temp\tmpyucy8gow
trying to download from https://arxiv.org/e-print/2205.15510
main tex files found:
.\main.tex
.\MOB-ML for dipole\main.tex
Processing .\main
Cache is found
It is a full latex document
  0%|                                                                                           | 0/49 [00:00<?, ?it/s]Error found in Parapragh 1
Content
  \date{\today}
  \begin{abstract}
This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree--Fock computations.
A molecular-orbital-based (MOB) pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of molecular orbitals (MOs).
The dipole MOB features consist of the energy MOB features and their responses to electric fields.
An interpretable and rotationally equivariant Gaussian process regression (GPR) with derivatives algorithm is introduced to learn the dipole moment more efficiently.
The proposed problem setup, feature design, and ML algorithm are shown to provide highly-accurate models for both dipole moment and energies on water and fourteen small molecules.
To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset.
The application of local scalable GPR with Gaussian mixture model unsupervised clustering (GMM/GPR) scales up MOB-ML to a large-data regime while retaining the prediction accuracy.
In addition, compared with literature results, MOB-ML provides the best test MAEs of 4.21 mDebye and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.
\end{abstract}
  2%|█▋                                                                                 | 1/49 [00:00<00:27,  1.78it/s]
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate_arxiv.py", line 143, in main
    success = translate_dir('.', options)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate_arxiv.py", line 75, in translate_dir
    translate_single_tex_file(file_path, file_path, options.engine, options.l_from, options.l_to, options.debug)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 255, in translate_single_tex_file
    text_final = latex_translator.translate_full_latex(text_original)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 227, in translate_full_latex
    raise e
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 220, in translate_full_latex
    latex_translated_paragraph = self.translate_paragraph_latex(latex_original_paragraph)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 163, in translate_paragraph_latex
    latex_translated_paragraph = self.translate_text_in_paragraph_latex_and_leading_brace(latex_original_paragraph)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 158, in translate_text_in_paragraph_latex_and_leading_brace
    latex_translated_paragraph = self.translate_text_in_paragraph_latex(latex_original_paragraph)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 137, in translate_text_in_paragraph_latex
    result += self._translate_text_in_paragraph_latex(split) + ' ' + sep + ' '
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 120, in _translate_text_in_paragraph_latex
    text_translated_paragraph = self.translate_paragraph_text(text_original_paragraph)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 95, in translate_paragraph_text
    parts_translated.append(self.translator.translate(part))
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 55, in translate
    raise e
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 49, in translate
    result = self.try_translate(text)
  File "C:\Program Files\Python310\lib\site-packages\mathtranslate\translate.py", line 41, in try_translate
    return self.translator.translate(text, self.language_to, self.language_from)
  File "C:\Program Files\Python310\lib\site-packages\mtranslate\core.py", line 80, in translate
    raw_data = urllib.request.urlopen(request).read()
  File "C:\Program Files\Python310\lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Program Files\Python310\lib\urllib\request.py", line 525, in open
    response = meth(req, response)
  File "C:\Program Files\Python310\lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
  File "C:\Program Files\Python310\lib\urllib\request.py", line 563, in error
    return self._call_chain(*args)
  File "C:\Program Files\Python310\lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
  File "C:\Program Files\Python310\lib\urllib\request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: Service Unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\shutil.py", line 622, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'C:\\Users\\29927\\AppData\\Local\\Temp\\tmpyucy8gow'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\tempfile.py", line 843, in onerror
    _os.unlink(path)
PermissionError: [WinError 5] 拒绝访问。: 'C:\\Users\\29927\\AppData\\Local\\Temp\\tmpyucy8gow'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\shutil.py", line 622, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'C:\\Users\\29927\\AppData\\Local\\Temp\\tmpyucy8gow'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\tempfile.py", line 843, in onerror
    _os.unlink(path)
PermissionError: [WinError 5] 拒绝访问。: 'C:\\Users\\29927\\AppData\\Local\\Temp\\tmpyucy8gow'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\shutil.py", line 622, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'C:\\Users\\29927\\AppData\\Local\\Temp\\tmpyucy8gow'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\tempfile.py", line 843, in onerror
    _os.unlink(path)
PermissionError: [WinError 5] 拒绝访问。: 'C:\\Users\\29927\\AppData\\Local\\Temp\\tmpyucy8gow'

......
.....
......
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Program Files\Python310\lib\tempfile.py", line 846, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Program Files\Python310\lib\tempfile.py", line 855, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 624, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Program Files\Python310\lib\tempfile.py", line 846, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Program Files\Python310\lib\tempfile.py", line 855, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 624, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Program Files\Python310\lib\tempfile.py", line 846, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Program Files\Python310\lib\tempfile.py", line 855, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 624, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Program Files\Python310\lib\tempfile.py", line 846, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Program Files\Python310\lib\tempfile.py", line 855, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 624, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Program Files\Python310\lib\tempfile.py", line 846, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Program Files\Python310\lib\tempfile.py", line 855, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Program Files\Python310\lib\shutil.py", line 624, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Program Files\Python310\lib\tempfile.py", line 840, in onerror
    resetperms(path)
  File "C:\Program Files\Python310\lib\tempfile.py", line 832, in resetperms
    _os.chflags(path, 0)
RecursionError: maximum recursion depth exceeded while calling a Python object

But actually I'm already an administrator

中文怎么翻译成英文？

skip \inpu{xx.tex} in case of file non-existence

        # assert os.path.exists(filename) 
        if not os.path.exists(filename): 
            # 从内容中移除对应的 \input 命令
            content = content[:begin] + content[end:]
            continue

the entire paper not translated

case: 1802.05642

some paper can not process

Executing command: translate_arxiv 2206.05982
The current mathtranslate is latest
Start
engine google
language from en
language to zh-CN

arxiv number: 2206.05982

temporary directory /var/folders/vt/qvr9908s2p16x02gpkty4c6h0000gn/T/tmplorc7_cg
trying to download from https://arxiv.org/e-print/2206.05982
main tex files found:
./demo.tex
./supplementary.tex
./main.tex
merging ./section/abstract.tex
merging ./section/introduction.tex
merging ./section/related_work.tex
merging ./section/main.tex
merging ./section/datasets.tex
merging ./section/experiments.tex
merging ./tables/all_tables.tex
merging ./tables/main_content}
Traceback (most recent call last):
  File "/Users/****/bin/translate_arxiv", line 8, in <module>
    sys.exit(main())
  File "/Users/****/lib/python3.10/site-packages/mathtranslate/translate_arxiv.py", line 154, in main
    raise e
  File "/Users/****/lib/python3.10/site-packages/mathtranslate/translate_arxiv.py", line 146, in main
    success = translate_dir('.', options)
  File "/Users/****/lib/python3.10/site-packages/mathtranslate/translate_arxiv.py", line 58, in translate_dir
    process_file.merge_complete(tex)
  File "/Users/****/lib/python3.10/site-packages/mathtranslate/process_file.py", line 27, in merge_complete
    assert os.path.exists(filename)
AssertionError

Some arxiv article can't be successfully translate

version
mathtranslate 2.4.4
log
log.txt
I have successfully translate some article. in this log I try to translate one article I once successed(2211.07864), it worked without any error. and then I try to translate the arxiv 2306.04098 with debug option. It return me with an error about proxy error. However , when I try with debug option, It shows that this may not because of the proxy. Is this a bug or I used it in a wrong way?

mis translation to Chinese, niche bib usage

case:
1905.05067v1

do not translate words from a given vocabulary

e.g., do not translate LLM to 法学硕士. Leave it as LLM.

e.g., do not Transformer LLM to 变压器. Leave it as Transformer.

Feature Request: support for ChatGPT translation

Translation doesn't work.

A great idea for a program, but unfortunately, despite all my efforts, I could not get it to work on Arch Linux. Neither method worked. Meanwhile, the demo version works great. Unfortunately, you cannot select the translation language there. Could you check the code or download the docker image from your working demo version? Thank you in advance. I will be looking forward to your answer.

Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

http://mathtranslate.xyz/ 翻译的输入是2302.00539
报错

Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

mis-translate \bib item to Chinese

case: 2311.12023

tikzstyle should not be translated

case: 2306.13549

\tikzstyle{my-box}    =[ 矩形，绘制=隐藏绘制，圆角，文本不透明度=1，最小高度=1.5em，最小宽度=5em，内部间隔=2pt，对齐=居中，填充不透明度=.5，线宽=0.8pt , ]
\tikzstyle{leaf}    =[我的盒子，最小高度=1.5em，填充=hidden-pink!80，文本=黑色，对齐=左，字体=    \tiny   ，内部xsep=2pt，内部ysep=4pt，线宽=0.8pt，]

Critical issue C:/Users permission denied

the same in administrative mode
and the same in MacOS version

trying to download from https://arxiv.org/e-print/2307.09288 Cannot download source, maybe network issue or wrong link 正确的地址应该是：https://arxiv.org/pdf/2307.09288.pdf

urllib.error.URLError: <urlopen error [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。>

translate_arxiv 2206.01838
The current mathtranslate is latest
Start
engine google
language from en
language to zh-CN

arxiv number: 2206.01838

temporary directory C:\Users\25122\AppData\Local\Temp\tmpzhsmq8r2
main tex files found:
.\neurips_2022.tex
merging .\envs.tex
merging .\body/Introduction.tex
merging .\body/Preliminaries.tex
merging .\body/KnowledgeDistillation.tex
merging .\tables/rand-format.tex
merging .\tables/zeroshot-pretrained-format.tex
merging .\tables/better_teacher_format.tex
merging .\body/IMP.tex
merging .\tables/structured.tex
merging .\tables/unstructured.tex
merging .\body/Conclusion.tex
merging .\body/appendix.tex
merging .\tables/hparam-mnli.tex
merging .\tables/hparam-qqp.tex
merging .\tables/hparam-qnli.tex
merging .\tables/hparam-sst.tex
merging .\body/related_work.tex
Processing .\neurips_2022
Cache is found
It is a full latex document
0%| | 0/76 [00:00<?, ?it/s]Error found in Parapragh 1
Content
\maketitle
1%|█ | 1/76 [00:21<26:20, 21.07s/it]
Traceback (most recent call last):
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
self.connect()
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 938, in connect
(self.host,self.port), self.timeout, self.source_address)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\socket.py", line 727, in create_connection
raise err
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\socket.py", line 716, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\Scripts\translate_arxiv.exe_main.py", line 7, in
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate_arxiv.py", line 176, in main
raise e
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate_arxiv.py", line 166, in main
success = translate_dir('.', options)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate_arxiv.py", line 91, in translate_dir
translate_single_tex_file(file_path, file_path, options.engine, options.l_from, options.l_to, options.debug, options.nocache)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 265, in translate_single_tex_file
text_final = latex_translator.translate_full_latex(text_original, nocache=nocache)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 237, in translate_full_latex
raise e
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 228, in translate_full_latex
latex_translated_paragraph = self.translate_paragraph_latex(latex_original_paragraph)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 165, in translate_paragraph_latex
latex_translated_paragraph = self.translate_text_in_paragraph_latex_and_leading_brace(latex_original_paragraph)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 160, in translate_text_in_paragraph_latex_and_leading_brace
latex_translated_paragraph = self.translate_text_in_paragraph_latex(latex_original_paragraph)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 137, in translate_text_in_paragraph_latex
result += self._translate_text_in_paragraph_latex(split) + ' ' + sep + ' '
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 120, in _translate_text_in_paragraph_latex
text_translated_paragraph = self.translate_paragraph_text(text_original_paragraph)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 95, in translate_paragraph_text
parts_translated.append(self.translator.translate(part))
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 55, in translate
raise e
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 49, in translate
result = self.try_translate(text)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 41, in try_translate
return self.translator.translate(text, self.language_to, self.language_from)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mtranslate\core.py", line 80, in translate
raw_data = urllib.request.urlopen(request).read()
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 525, in open
response = self._open(req, data)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 543, in _open
'_open', req)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1345, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。>

想问一下这个项目和nougat的匹配程度怎么样

nougat复读蛮严重的，这个项目有做一些后验的处理去除掉nougat输出的源文件的复读部分和一些latex公式错误的部分吗

submodule: extract text from pdf and translate

    1. It is simply a PDF file (cannot translate)

报错RecursionError: maximum recursion depth exceeded while calling a Python object

translate_arxiv.exe 2305.13048 -eng tencent
The current mathtranslate is latest
Start
engine tencent
language from en
language to zh
threads 1

arxiv number: 2305.13048

temporary directory C:\Users\25122\AppData\Local\Temp\tmprmub0lge
main tex files found:
.\main.tex
merging .\acc_figures.tex
Processing .\main
Cache is found
It is not a full latex document
  0%|                                                                                          | 0/297 [00:00<?, ?it/s]Error found in Parapragh 110
Content
Tasks:
¥begin  {itemize}
    ¥item LAMBADA‾¥cite  {LAMBADAdataset}  . A benchmark dataset that evaluates the model's contextual reasoning and language comprehension abilities by presenting context-target pairs, where the objective is to predict the most probable target token.
    ¥item PIQA‾¥cite  {Bisk2020}  . A benchmark for the task of physical common sense reasoning, which consists of a binary choice task that can be better understood as a set of two pairs, namely (Goal, Solution).
    ¥item HellaSwag ‾¥cite  {HellaSwag2019}   A novel benchmark for commonsense Natural Language Inference (NLI) which is build by adversarial filtering against transformer models.
    ¥item Winogrande ‾¥cite  {Wino2020}   A dataset designed to evaluate the acquisition of common sense reasoning by neural language models, aiming to determine whether we are accurately assessing the true capabilities of machine common sense.
    ¥item StoryCloze‾¥cite  {StoryCloze2016}   A benchmark to present a novel approach to assess comprehension of narratives, narrative generation, and script acquisition, focusing on commonsense reasoning.
    ¥item ARC Challenge ‾¥cite  {ARC2018}   A dataset designed for multiple-choice question answering, encompassing science exam questions ranging from third grade to ninth grade.
    ¥item ARC Easy An easy subset of ARC.
    ¥item HeadQA ‾¥cite  {HeadQA2020}   A benchmark consisting of graduate-level questions encompassing various fields such as medicine, nursing, biology, chemistry, psychology, and pharmacology.
    ¥item OpenBookQA ‾¥cite  {OpenBookQA2018}   A QA dataset to evaluate human comprehension of a subject by incorporating open book facts, scientific knowledge, and perceptual common sense, drawing inspiration from open book exams.
    ¥item SciQ ‾¥cite  {SciQ2017}   A multiple-choice QA dataset which was created using an innovative approach to gather well-crafted multiple-choice questions that are focused on a specific domain.
    ¥item TriviaQA ‾¥cite  {TriviaQA2017}   A QA-IR dataset which is constituted of triples of questions, answers, supporting evidence, and independently collected evidence documents, with an average of six documents per question for reliable sources.
    ¥item ReCoRD ‾¥cite  {ReCord}   A benchmark for evaluating commonsense reasoning in reading comprehension by generating queries from CNN/Daily Mail news articles and requiring text span answers from corresponding summarizing passages.
    ¥item COPA ‾¥cite  {COPA2011}   A dataset to evaluate achievement in open-domain commonsense causal reasoning.
    ¥item MMMLU ‾¥cite  {MMMLU2021}   A multi-task dataset for 57 tasks containing elementary mathematics, US history, computer science, law, etc.
¥end  {itemize}
 37%|████████████████████████████▉                                                 | 110/297 [00:00<00:00, 1188.54it/s]
Error found in Parapragh 110
Content
¥begin  {table*}  [!]
¥centering
¥small
¥begin  {tabular}    {llllllllll}
        ¥toprule
¥textbf  {Model}   & ¥textbf  {Params}   & ¥textbf  {PIQA}   & ¥textbf  {StoryCloze}   & ¥textbf  {HellaSwag}   & ¥textbf  {WinoGrande}   & ¥textbf  {ARC-e}   & ¥textbf  {ARC-c}   & ¥textbf  {OBQA}   ¥¥
 & B & acc & acc & acc¥\_norm & acc & acc & acc¥\_norm & acc¥\_norm ¥¥
¥midrule
RWKV-4 &  0.17 & ¥TBF  {65.07}   & ¥TBF  {58.79}   & ¥TBF  {32.26}   & 50.83 & ¥TBF  {47.47}   & ¥TBF  {24.15}   & ¥TBF  {29.60}   ¥¥
Pythia & 0.16      & 62.68 & 58.47 & 31.63 & ¥TBF  {52.01}   & 45.12 & 23.81 & 29.20 ¥¥
GPT-Neo & 0.16     & 63.06 & 58.26 & 30.42 & 50.43 & 43.73 & 23.12 & 26.20 ¥¥
¥midrule
RWKV-4 &  0.43 & ¥TBF  {67.52}   & ¥TBF  {63.87}   & ¥TBF  {40.90}   & 51.14       & ¥TBF  {52.86}   & 25.17       & ¥TBF  {32.40}   ¥¥
Pythia & 0.40      & 66.70       & 62.64      & 39.10      & ¥TBF  {53.35}   & 50.38       & ¥TBF  {25.77}   & 30.00 ¥¥
GPT-Neo & 0.40     & 65.07       & 61.04       & 37.64      & 51.14     & 48.91       & 25.34 & 30.60 ¥¥
¥midrule
RWKV-4 & 1.5 & ¥TBF  {72.36}   & ¥TBF  {68.73}   & ¥TBF  {52.48}     & 54.62      & ¥TBF  {60.48}   & ¥TBF  {29.44}               & ¥TBF  {34.00}   ¥¥
Pythia & 1.4 & 71.11       & 67.66       & 50.82        & ¥TBF  {56.51}   & 57.74 & 28.58             & 30.80 ¥¥
GPT-Neo & 1.4 & 71.16      & 67.72        & 48.94       & 54.93       & 56.19       & 25.85             & 33.60 ¥¥
¥midrule
RWKV-4 & 3.0 & ¥TBF  {74.16}   & ¥TBF  {70.71}   & ¥TBF  {59.89}   & 59.59       & ¥TBF  {65.19}           & ¥TBF  {33.11}   & ¥TBF  {37.00}   ¥¥
Pythia & 2.8 & 73.83       & ¥TBF  {70.71}   & 59.46       & ¥TBF  {61.25}   & 62.84   & 32.25 & 35.20 ¥¥
GPT-Neo & 2.8 & 72.14      & 69.54       & 55.82        & 57.62      & 61.07         & 30.20       & 33.20 ¥¥
¥midrule
RWKV-4 & 7.4 & ¥TBF  {76.06}   & 73.44       & 65.51      & 61.01       & ¥TBF  {67.80}   & ¥TBF  {37.46}           & ¥TBF  {40.20}   ¥¥
Pythia & 6.9 & 74.54       & 72.96       & 63.92      & 61.01       & 66.79       & 35.07               & 38.00 ¥¥
GPT-J & 6.1  & 75.41        & ¥TBF  {74.02 }  & ¥TBF  {66.25}  & ¥TBF  {64.09}   & 66.92 & 36.60                    & 38.20 ¥¥
¥midrule
RWKV-4 & 14.2    & ¥TBF  {77.48}              & ¥TBF  {76.06}     & ¥TBF  {70.65}         & 63.85       & 70.24       & ¥TBF  {38.99}         & ¥TBF  {41.80}   ¥¥
GPT-level  $^*$   & 14.2 & 76.49           & 74.97   & 68.72       & ¥TBF  {65.14}        & ¥TBF  {70.77}          & 37.99       & 39.27 ¥¥
¥midrule
Pythia (c.f.)  & 11.8    & 75.90           & 74.40   & 67.38       & 64.72       & 69.82        & 36.77       & 38.80 ¥¥
GPT-NeoX (c.f.)  & 20.6  & 77.69     & 76.11 & 71.42 & 65.98    & 72.69 & 40.44 & 40.20 ¥¥
¥bottomrule
¥end  {tabular}
¥centering
¥caption  {¥label{tab:commonsense_reasoning_results}
Zero-Shot Performance of the model on Common Sense Reasoning Tasks.   $^*$   Interpolation of Pythia and GPT-Neo models
}
¥end  {table*}
Traceback (most recent call last):
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 246, in translate_full_latex
    latex_translated_paragraphs = list(tqdm.auto.tqdm(executor.map(self.worker, latex_original_paragraphs), total=len(latex_original_paragraphs)))
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\tqdm\std.py", line 1180, in __iter__
    for obj in iterable:
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 598, in result_iterator
    yield fs.pop().result()
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 435, in result
    return self.__get_result()
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 201, in worker
    raise e
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 191, in worker
    latex_translated_paragraph = self.translate_paragraph_latex(latex_original_paragraph)
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 170, in translate_paragraph_latex
    latex_translated_paragraph = self.translate_text_in_paragraph_latex_and_leading_brace(latex_original_paragraph)
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 165, in translate_text_in_paragraph_latex_and_leading_brace
    latex_translated_paragraph = self.translate_text_in_paragraph_latex(latex_original_paragraph)
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 142, in translate_text_in_paragraph_latex
    result += self._translate_text_in_paragraph_latex(split) + ' ' + sep + ' '
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\translate.py", line 117, in _translate_text_in_paragraph_latex
    text_original_paragraph = process_text.split_too_long_paragraphs(text_original_paragraph)
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\process_text.py", line 47, in split_too_long_paragraphs
    par2 = split_too_long_paragraphs('.'.join(lines[position:]))
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\process_text.py", line 47, in split_too_long_paragraphs
    par2 = split_too_long_paragraphs('.'.join(lines[position:]))
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\process_text.py", line 47, in split_too_long_paragraphs
    par2 = split_too_long_paragraphs('.'.join(lines[position:]))
  [Previous line repeated 982 more times]
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\process_text.py", line 42, in split_too_long_paragraphs
    first_words = [get_first_word(line) for line in lines]
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\process_text.py", line 42, in <listcomp>
    first_words = [get_first_word(line) for line in lines]
  File "C:\Users\25122\AppData\Local\Programs\Python\Python37\lib\site-packages\mathtranslate\process_text.py", line 26, in get_first_word
    words = line.split(' ')
RecursionError: maximum recursion depth exceeded while calling a Python object

latex objects substitution fail

2311.00636

trying to download from https://arxiv.org/e-print/2307.09288

Tecent Engine

How can I solve the problem when I use tecent engine and it reports like this:

tencentcloud.common.exception.tencent_cloud_sdk_exception.TencentCloudSDKException: [TencentCloudSDKException] code:RequestLimitExceeded message:您当前每秒请求 6 次，超过了每秒频率上限 5，请稍后重试。 requestId:bc0e30fe-037b-48ce-85a7-0376e0cff672

Typo: LaTeX

Feature Request: support DeepL and OpenAI for translation

mac没有软件显示在application，命令行运行后无事发生

translate_arxiv 1603.09320 报错

论文地址：https://arxiv.org/abs/1603.09320

Windows11
miniconda3
确定 Temp下可以访问和删除文件；

注1：每次运行前 Temp下的tmptm74l2n0/ 删除；

注2：别的 arxiv 论文没有问题

报错如下：

translate_arxiv 1603.09320

The current mathtranslate is latest
Start
engine google
language from en
language to zh-CN

arxiv number: 1603.09320

temporary directory E:\Users\XXXXX\AppData\Local\Temp\tmptm74l2n0
Traceback (most recent call last):
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 622, in _rmtree_unsafe
os.rmdir(path)
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'E:\Users\XXXXX\AppData\Local\Temp\tmptm74l2n0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 843, in onerror
_os.unlink(path)
PermissionError: [WinError 5] 拒绝访问。: 'E:\Users\XXXXX\AppData\Local\Temp\tmptm74l2n0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 622, in _rmtree_unsafe
os.rmdir(path)
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'E:\Users\XXXXX\AppData\Local\Temp\tmptm74l2n0'

During handling of the above exception, another exception occurred:

// 还有很多这种打印，紧接着

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\Users\ProgramData\miniconda3\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "E:\Users\ProgramData\miniconda3\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "E:\Users\ProgramData\miniconda3\Scripts\translate_arxiv.exe_main.py", line 7, in
File "E:\Users\ProgramData\miniconda3\lib\site-packages\mathtranslate\translate_arxiv.py", line 140, in main
with tempfile.TemporaryDirectory() as temp_dir:
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 869, in exit
self.cleanup()
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 873, in cleanup
self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 750, in rmtree
return _rmtree_unsafe(path, onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 624, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 846, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 750, in rmtree
return _rmtree_unsafe(path, onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 624, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 846, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 750, in rmtree
return _rmtree_unsafe(path, onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 624, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 846, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)

// 还有很多这样打印，最后

File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 846, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 750, in rmtree
return _rmtree_unsafe(path, onerror)
File "E:\Users\ProgramData\miniconda3\lib\shutil.py", line 624, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 840, in onerror
resetperms(path)
File "E:\Users\ProgramData\miniconda3\lib\tempfile.py", line 832, in resetperms
_os.chflags(path, 0)
RecursionError: maximum recursion depth exceeded while calling a Python object

MacOS上的一点小建议

MacOS上编译带中文的.tex文件时，对ctex库要加上一句\usepackage[fontset=mac]{ctex}，是否可以在运行程序时自动补上，不用后续手动添加。

## bugfix: ValueError: ZIP does not support timestamps before 1980

case: 2107.00630v6

fix:

## bugfix: ValueError: ZIP does not support timestamps before 1980
import time
def zipdir(dir, output_path):
    # ziph is zipfile handle
    zipf = zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED)
    for file in loop_files(dir):
        rel_path = os.path.relpath(file, dir)

        # 检查并修复文件时间戳
        timestamp = os.path.getmtime(file)
        if timestamp < time.mktime((1980, 1, 1, 0, 0, 0, 0, 0, 0)):
            os.utime(file, (time.mktime((1980, 1, 1, 0, 0, 0, 0, 0, 0)), time.mktime((1980, 1, 1, 0, 0, 0, 0, 0, 0))))

        zipf.write(file, arcname=rel_path)
    zipf.close()

def zipdir0(dir, output_path):
    # ziph is zipfile handle
    zipf = zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED)
    for file in loop_files(dir):
        rel_path = os.path.relpath(file, dir)
        zipf.write(file, arcname=rel_path)

all the \input subdocs are NOT translated

case: 2307.08423

all the \input subdocs are NOT translated

临时存储文件位置

windows下对 C 盘的某些位置读写的时候会导致PermissionError，必须得用管理员打开才可以。

可否更改下cache的位置，比如 D 盘的某些位置

(PermissionError应该就是这个问题，或者可以判断下系统，win的话放d盘？)

(U◕‿‿◕U)

cant run the program

i tried the CLI and it said:"The term 'translate_tex' is not recognized as the name of a cmdlet, function, script file, or operable program"
WEB: Cant change language
APP:Error: mathtranslate_log is not find

\input nested

def merge_complete(tex):
    '''
    for replace all \input commands by the file content
    '''
    path = f'{tex}.tex'
    dirname = os.path.dirname(path)
    encoding = get_file_encoding(path)
    content = open(path, encoding=encoding).read()
    content = remove_tex_comments(content)
    pattern_input = re.compile(r'\\input{(.*?)}')
    while True:
        result = pattern_input.search(content)
        if result is None:
            break
        begin, end = result.span()
        match = result.group(1)
        filename = os.path.join(dirname, match)
        if os.path.exists(f'{filename}.tex'):
            filename = f'{filename}.tex'
        print('merging', filename)
        assert os.path.exists(filename)
        encoding = get_file_encoding(filename)
        new_content = open(filename, encoding=encoding).read()
        new_content = remove_tex_comments(new_content)
        content = content[:begin] + new_content + content[end:]
    print(content, file=open(path, "w", encoding='utf-8'))

函数解析
该函数的主要目标是将LaTeX文档中的\input命令替换为对应文件的内容，也就是说，它将所有被\input的文件内容合并到主tex文件中。

函数定义
函数merge_complete(tex)接收一个参数tex，这个参数应该是一个不包含扩展名的LaTeX文件名。
函数实现
首先，函数通过os.path.dirname(path)获取主tex文件的目录名，然后通过get_file_encoding(path)获取文件的编码方式。

然后，函数打开主tex文件，并读取其内容。同时，它使用remove_tex_comments(content)函数移除了文件中的所有LaTeX注释。

接着，函数使用正则表达式re.compile(r'\input{(.*?)}')匹配所有的\input命令。

在一个无限循环中，函数搜索第一个\input命令。如果找不到\input命令，就跳出循环。

如果找到了\input命令，函数就获取该命令的位置（begin和end），以及\input命令中的文件名（match）。

函数检查\input命令中的文件是否存在。如果存在，就打开该文件，并读取其内容。同时，它使用remove_tex_comments(new_content)函数移除了文件中的所有LaTeX注释。

最后，函数将\input命令替换为对应文件的内容，然后继续查找下一个\input命令。

当所有的\input命令都被替换后，函数将合并后的内容写入主tex文件。

注意事项
这个函数假设所有的\input命令都在主tex文件的同一目录或其子目录中。

这个函数不会处理\input命令中的相对路径。例如，如果\input命令中的文件名是../other.tex，函数将无法正确地找到该文件。

这个函数不会处理嵌套的\input命令。例如，如果一个被\input的文件中还包含\input命令，这个函数将无法正确地处理这种情况。

这个函数不会处理\include命令，这是LaTeX中另一种包含文件的命令。

翻译出来的文档无法正常显示参考文献

paragraphs not translated

case: 2311.09263

windows版本gui出现错误

下载文件后，以管理员运行MathTranslate出错：
protocol ( ssl.c:1131)')
urlib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:1131)
The above exception was the direct cause of the following exception:
urllib3.exceptions.ProxyError: ('Unable to connect to proxy', SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol ( ssl.c:1131)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "requests\adapters.py", line 486, in send
File "urllib3\connectionpool.py", line 844, in urlopen
File "urllib3\util\retry.py", line 515, in increment
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "MathTranslate. py", line 44, in
File "kivy\app.py", line 955, in run
File "kivy\app.py", line 925, in _run_prepare
File"MathTranslate.py", line 27, in build
File "IndexPage. py", line 13, in init
File "mathtranslate\update.py", line 5, in get_latest_version
File "requests\api.py", line 73, in get
File "requests\api.py", line 59, in request
File "requests\sessions.py", line 589, in request
File "requests\sessions.py", line 703, in send
File "requests\adapters.py", line 513, in send
requests.exceptions.ProxyError: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/mathtranslate/json (Caused by ProxyError('Unable to connect to proxy', SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)'))))

Error found in Parapragh 63

translate_arxiv.exe 2307.08621v4 -eng google
The current mathtranslate is latest
Start
engine google
language from en
language to zh-CN
threads auto

arxiv number: 2307.08621v4

temporary directory C:\Users\25122\AppData\Local\Temp\tmphwijv6j4
trying to download from https://arxiv.org/e-print/2307.08621v4
main tex files found:
.\main.tex
merging .\settings.tex
merging .\math_commands.tex
Processing .\main
It is a full latex document
  3%|██▊                                                                                | 3/90 [00:05<02:55,  2.01s/it]Error found in ParapraghError found in Parapragh  3232

Content
Content  \begin{table*}[t]
\begin{tabular}{lcccc}
\toprule
\bf Architectures      & \bf \tabincell{c}{Training  XMATHXBS  Parallelization} & \bf  Inference Cost & \bf \tabincell{c}{Long-Sequence  XMATHXBS  Memory Complexity} & \bf  Performance  XMATHXBS
\midrule
Transformer        & \ding{52} &   $O(N)$   &   $O(N^2)$   & \ding{52}\ding{52}  XMATHXBS
Linear Transformer & \ding{52} &   $O(1)$   &   $O(N)$   & \ding{56}  XMATHXBS
Recurrent NN       & \ding{56} &   $O(1)$   &   $O(N)$   & \ding{56}  XMATHXBS
RWKV               & \ding{56} &   $O(1)$   &   $O(N)$   & \ding{52}  XMATHXBS
H3/S4              & \ding{52} &   $O(1)$   &   $O(N \log N)$   & \ding{52}  XMATHXBS
Hyena              & \ding{52} &   $O(N)$   &   $O(N\log N)$   & \ding{52}  XMATHXBS
\our{}                & \ding{52} &   $O(1)$   &   $O(N)$   & \ding{52}\ding{52}  XMATHXBS
\bottomrule
\end{tabular}
\caption{Model comparison from various perspectives. \our{} achieves training parallelization, constant inference cost, linear long-sequence memory complexity, and good performance.}
\label{tbl:compare:aspects}
\end{table*}

  \paragraph{Inference}
The recurrent representation (  \Eqref{eq:ret:recurrent}  ) is employed during the inference, which nicely fits autoregressive decoding.
The   $O(1)$   complexity reduces memory and inference latency while achieving equivalent results.
  6%|████▌                                                                              | 5/90 [00:07<01:57,  1.38s/it]Error found in Parapragh 46

Failed to execute script 'MathTranslate'

Reproduce Command

sudo ./update # or ./update directly
sudo ./MathTranslate # or  ./MathTranslate directly

Issue Output

[202173] Failed to execute script 'MathTranslate' due to unhandled exception!

Target Release:

MathTranslate_Linux

Environment:

Linux 20.04 LTS
GPU 3090
NVIDIA Driver Version: 535.54.03
CUDA Version: 12.2

PDF 翻译

老哥们PDF的翻译有什么好法子吗？

get_file_encoding error

arxiv number: 1612.02523
temporary directory /var/folders/xz/b53crnzx71j3q8q7fh6qch400000gp/T/tmpmt9nyn04
This is a pure text file
main tex files found:
Traceback (most recent call last):
File "/usr/local/bin/translate_arxiv", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate_arxiv.py", line 221, in main
raise e
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate_arxiv.py", line 211, in main
success = translate_dir('.', options)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate_arxiv.py", line 111, in translate_dir
content = open(path, encoding=input_encoding).read()
File "/usr/local/Cellar/[email protected]/3.7.12/Frameworks/Python.framework/Versions/3.7/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 67479: invalid start byte

arxiv number: 1612.02523
temporary directory /var/folders/xz/b53crnzx71j3q8q7fh6qch400000gp/T/tmpwz89rjiv
download completed...
This is a pure text file
main tex files found:
Traceback (most recent call last):
File "/usr/local/bin/translate_arxiv", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate_arxiv.py", line 221, in main
raise e
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate_arxiv.py", line 211, in main
success = translate_dir('.', options)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/translate_arxiv.py", line 110, in translate_dir
input_encoding = get_file_encoding(path)
File "/usr/local/lib/python3.7/site-packages/mathtranslate/encoding.py", line 21, in get_file_encoding
if result['confidence'] < 0.9:
TypeError: '<' not supported between instances of 'NoneType' and 'float'

将 overleaf 编译功能抽取出来

https://www.bilibili.com/read/cv6547551

根据这篇文章的说法，overleaf 有开源版本，请问您之前调研过将它的编译 latex 到 pdf 的功能单独提取出来调用吗？

这样的好处是，将整个论文的调用过程全自动了，不需要手工上传zip；

如果您有调研过，能不能简单说明情况，比如哪里不可行或者遇到什么问题；我也可以尝试下。

susyustc / mathtranslate Goto Github PK

mathtranslate's People

Stargazers

Watchers

Forkers

mathtranslate's Issues

Reproduce Command

Issue Output

Target Release:

Environment:

Recommend Projects

Recommend Topics

Recommend Org