对于生产环境部署, 能否使用llama.cpp/fastllm.cpp等纯C++框架运行TexTeller模型? 希望能提供便于部署的q4/q8模型和actions/dockerfile之类部署batch文件

在windows端需要上传文件才能识别，比较繁琐。

https://github.com/microsoft/ArxivFormula

想问下[2024-05-02] Support mixed Chinese English formula recognition.具体是指什么样的功能或识别能力。测试如果是公式里带有中文字符并不能正常识别出来，看HF模型的tokenizer_config.json、vocab.jsont也没有更新中文的词。

Windows端截图工具和Snipaste不支持直接拖放, 仅支持粘贴图片. 之前尝试修改web.py使得客户端同时支持拖放和粘贴, 但修改后拖放功能失效, 仅能进一步修改使其较好的支持粘贴功能. 希望能同时支持拖放上传图片和粘贴.

本地对web.py进行的修改如下, 仅实现了从剪切板粘贴图片:

from streamlit_paste_button import paste_image_button
paste_result = paste_image_button( label="📋 Paste an image", background_color="#FF0000", hover_background_color="#380909", errors='raise')

if paste_result.image_data is not None:
    temp_dir = tempfile.mkdtemp()
    png_file_path = os.path.join(temp_dir, 'image.png')

    if paste_result.image_data:
        img = paste_result.image_data

    img.save(png_file_path, format='PNG')
    file = open(png_file_path,'br')
    buffered=io.BytesIO(file.read())
    img_base64 = base64.b64encode(buffered.getvalue()).decode()

以下是报错信息：

RuntimeError: stack expects a non-empty TensorList
Traceback:
File "E:\anaconda\envs\TeXTeller310\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 535, in _run_script
exec(code, module.dict)
File "C:\Users\啦啦啦\TexTeller\src\web.py", line 180, in
TexTeller_result = inference(
File "C:\Users\啦啦啦\TexTeller\src\models\ocr_model\utils\inference.py", line 27, in inference
pixel_values = torch.stack(imgs)

我尝试了扫描版pdf里的公式识别，正确率不太行

首先非常感谢分享该开源项目与模型，目前发现在本地调试中，部分自测的公式图片识别异常，存在复读问题，请帮忙分析下（在该项目提供的部分训练数据集中测试正常）：

识别结果为：
[\begin{array}{c}\Delta\underline{L}_{\text{\tiny{\it{j}}}\text{\tiny{\it{i}}} \text{\tiny{\it{j}}}\text{\tiny{\it{i}}}\text{\tiny{\it{j}}}\text{\tiny{\it{i}}} \text{\tiny{\it{j}}}\text{\tiny{\it{i}}}\text{\tiny{\it{j}}}\text{\tiny{\it{i}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}} \text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{ \it{j}}}\text{\it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{ \it{j}}}\text{\tiny{\it{j}}}\text{\it{ \it{ \it{j}}}\text{\it{ \it{ \it{j}}}\text{\tiny{\it{ \it{ \it{j}}}\text{\it{ \it{j}}}\text{\it{ \it{ \it{ \it{j}}}\text{\it{ \it{ \it{ }}\text{ \it{ \it{ }}\text{ \it{ \it{ }}\text{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \it{ \bm{ \it}}}\text{ \bm{ \it{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \bm{ \

识别结果为：
[\bar{b}\bar{=}\frac{\Delta\bar{T_{i}}}{\bar{k_{i}}\bar{=}\bar{k_{i}}\bar{=} \bar{1}\bar{1}\bar{1}\bar{1}\bar{0}\bar{0}\bar{0}\bar{0}\bar{+}\bar{1}\bar{1} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{1}\bar{1} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0} \bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\bar{0}\

另外请帮忙确认下目前在huggingface上提供的模型版本已经是TexTeller 2.0吗（基于7.5M数据训练）？

作者您好，当我训练自己的时候的时候，能够训练python -m models.tokenizer.train 并保存了自己的tokenizer
当我通过python -m models.ocr_model.train.train，并未添加您训练好的模型，是通过您给的一些数据集进行训练
训练后不知道应该如何保存成作者您huggingface上这样的权重：https://huggingface.co/OleehyO/TexTeller/tree/main。
我想将保存下来的权重后加载您写的web进行重复使用
请问我是需要添加怎样的操作才能做到你这样吗？

非常感谢你的工作！拿 https://gregorygundersen.com/blog/2019/12/23/random-fourier-features/ 里的若干公式试了试，单行公式总体还可以，但多行公式大概 30--40% 预测的公式有语法错误、或有行缺失，我尝试将多行公式拆成单行来截图，会好很多。另外即使是单行公式也有时有 hallucination 问题，例如

$\mathbb R^D$ 变成了 $\mathbb S^2$。用的 NUM_BEAM=4。

我使用的latex引擎是tectonic, 对于不准备安装TeXLive的用户来说, 能否设法在web端使用katex渲染输出结果?

你好，首先非常感谢你在Latex OCR方面所做的贡献，我下载过您提供的550K数据集，经过数据清洗，大概有1000个token，其中绝大多数我感觉已经造成数据集的冗余，也许剔除掉更好，然而这并不简单，我想经过mathpix api数据进行清洗或许是个可行的方案，对您提供的550K数据集我正在做这方面的工作，从结果上来看，600个token就可以完全覆盖，但我仍然缺乏最够的向550K这样高质量多行复杂印刷体数据（同时我正在构造手写体和扫描体），或许我们可以合作，我可以提供mathpix清洗后的数据（mathpix大概可以有600万的额度）如果对此感兴趣，欢迎联系

There is no such function in the functional module of utils
i swapped it to img_inf_transform and training works, wanted to see if this is right

分享文件：train_dataset(1).zip
链接：https://pan.xunlei.com/s/VNunHL_PrRCgxrdFretqBDcMA1#
提取码：bw55
复制这段内容后打开手机迅雷App，查看更方便

这个怎么用来训练表格结构识别？

试用了混合识别，但没有一张图片能够正确识别，文字的加入甚至导致公式识别正确率大幅下滑，请问是否应该放弃使用多模型合作，而是将文字识别和公式识别训练为一个模型？