Giter Club home page Giter Club logo

Comments (5)

AemaH avatar AemaH commented on July 24, 2024

找到原因了(捂脸)
原来ctc_2019里面的callbacks 把那个封装的evaluate封装函数,记得昨晚十几分钟跑完那个CNN_2019的文件的训练部分后,结果用这个来测试,结果跑了两个点还没跑完,又是被这个支配了吗?(惊恐脸)

顺带说一下修改方法:把callbacks里面的Evaluate()注释掉就好了,也就是不用准确率回调函数。

from captcha_break.

ypwhs avatar ypwhs commented on July 24, 2024

我这里跑起来没有问题,你可以尝试更新版本,pip install -U tensorflow-gpu

from captcha_break.

ypwhs avatar ypwhs commented on July 24, 2024

顺带插一句隔壁的题外话,在TensorFlow相关的ipynb文件中,您对于模型的evaluate都是编写了一个evalute()函数或者类,这个评估函数算起来总是十分的慢,比如CNN_2019文件中,训练模型总共加起来不过十多分钟,结果在运行这个函数的时候,已经修改epoch为1了,依旧快两个小时,还没有结束的预兆。
按照该函数的意思,应该是遍历全部的可能组合结果对于该识别器进行评估?,在开头设置的时候,我加入了小写字母的考虑,难道是这个原因,导致的计算时间猛增?

Originally posted by @AemaH in #29 (comment)

准确率回调函数

from tqdm import tqdm

def evaluate(model, batch_size=128, steps=20):
    batch_acc = 0
    valid_data = CaptchaSequence(characters, batch_size, steps)
    for [X_test, y_test, _, _], _ in valid_data:
        y_pred = base_model.predict(X_test)
        shape = y_pred.shape
        out = K.get_value(K.ctc_decode(y_pred, input_length=np.ones(shape[0])*shape[1])[0][0])[:, :4]
        if out.shape[1] == 4:
            batch_acc += (y_test == out).all(axis=1).mean()
    return batch_acc / steps
from tensorflow.keras.callbacks import Callback

class Evaluate(Callback):
    def __init__(self):
        self.accs = []
    
    def on_epoch_end(self, epoch, logs=None):
        logs = logs or {}
        acc = evaluate(base_model)
        logs['val_acc'] = acc
        self.accs.append(acc)
        print(f'\nacc: {acc*100:.4f}')

这里的 steps=20,速度很快,不会遍历所有情况。

from captcha_break.

AemaH avatar AemaH commented on July 24, 2024

好的,我明天会再好好再查看一下里面的过程,十分感谢您的回复,给您添麻烦了 感恩

from captcha_break.

ypwhs avatar ypwhs commented on July 24, 2024

可以考虑的 debug 方向:

  • 在 evaluate 函数的 valid_data 外面包一层 tqdm,显示计算分数的进度
  • 把代码复制到 Pycharm 里,进入 Debug 模式,调试代码,在evaluate 函数里下断点,看哪一步会出问题
  • 随时监控显存容量,watch -n 1 nvidia-smi,看看是不是显存爆了

我注意到你修改了图片尺寸,并且在跑 CNN 的时候一代需要十几分钟,而我本地跑只需要2分钟,所以你的显卡的显存可能不大,在显存爆掉的时候,有可能会卡很久。

我这里跑 ctc_2019.ipynb 所需的显存是 4822MiB,你可以作为参考。 evaluate 函数的 batch_size 默认是 128,你可以适当减小 batch_size,增加 steps,以适应较小显存,同时保持比较准确的分数估计。

如果无法定位问题,我这里也无法复现,可以考虑使用 pytorch 代码。

from captcha_break.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.