Giter Club home page Giter Club logo

Comments (7)

chengduoZH avatar chengduoZH commented on May 10, 2024

这里只返回了seq_pool之后的结果,确实跟huggingface/transformers里面的返回结果不一样,我们内部讨论一下是否要做出一致的,稍后答复。

一个临时的解决方案是您那边将hidden_cache放到返回值列表中(这里的hidden_cache是encoder的最后一层的state,维度为[batch, seq_len, hidden_size]):

 hidden_cache = self.encoder(hidden_states=hidden_cache, 
                             attention_mask=extended_attention_masks, 
                             return_type=ReturnType.turbo_transformers, 
                             output=hidden_cache) 
  
 self.seq_pool = SequencePool(PoolingMap[pooling_type]) 
 output = self.seq_pool(input_tensor=hidden_cache, 
                        return_type=return_type, 
                        output_tensor=output) 
 return  convert_returns_as_type(hidden_cache, return_type), output 

另外,如果您的代码中同时用到了BertModelWithPooler,在BertModelWithPooler里面也需要改一下:

        encoder_output = self.bertmodel(
            inputs,
            attention_masks,
            token_type_ids,
            position_ids,
            pooling_type,
            hidden_cache,
            output=None,
            return_type=ReturnType.turbo_transformers)
        pooler_output = self.pooler(encoder_output[1], return_type, pooler_output)
        return pooler_output, convert_returns_as_type(encoder_output,
                                                      return_type)

from turbotransformers.

shizhh avatar shizhh commented on May 10, 2024

@chengduoZH 赞!测试了下,在我的机器上,cpu环境下速度提升了4倍。另外问一下,会不会提供GPU的版本?

from turbotransformers.

feifeibear avatar feifeibear commented on May 10, 2024

@chengduoZH 赞!测试了下,在我的机器上,cpu环境下速度提升了4倍。另外问一下,会不会提供GPU的版本?

已经有GPU版本了呀,另外你要要hidden_state就给加一下也不麻烦

from turbotransformers.

shizhh avatar shizhh commented on May 10, 2024

cpu环境下速度提升了4倍

更正下,是因为线程数设置了4;如果线程数为1,速度提升没那么明显,10%左右吧

from turbotransformers.

shizhh avatar shizhh commented on May 10, 2024

已经有GPU版本了呀,另外你要要hidden_state就给加一下也不麻烦

@feifeibear 什么时候提供conda的安装方式呢

from turbotransformers.

feifeibear avatar feifeibear commented on May 10, 2024

已经有GPU版本了呀,另外你要要hidden_state就给加一下也不麻烦

@feifeibear 什么时候提供conda的安装方式呢

已经加了你的hidden_state返回需求到master,什么情况下必须一个conda包进行部署?gpu的wheel包满足不了需求么?

from turbotransformers.

reyoung avatar reyoung commented on May 10, 2024

Hidden需求已经满足了。

使用 #19 这个issue来跟踪conda 安装包的开发进度。

from turbotransformers.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.