Comments (2)
我用这个模型做一个批量的处理,基本每个样本处理一次,都要重新load一次同义词表,这样巨慢,请问有什么好的方法可以只加载一次同义词表嘛?
def aug(df,create_num=3,change_rate=0.3): smw = Similarword(create_num=create_num, change_rate=change_rate) rs1 = smw.replace(df) for i in rs1: res = i return resdf_9['content_seg_new']= df_9['content_seg'].apply(lambda x:aug(x))load :D:\anaconda3\lib\site-packages\nlpcda\data\同义词.txt done load :D:\anaconda3\lib\site-packages\nlpcda\data\同义词.txt done load :D:\anaconda3\lib\site-packages\nlpcda\data\同义词.txt done load :D:\anaconda3\lib\site-packages\nlpcda\data\同义词.txt done ...
pandas的apply是并行的啊。当然这样了。你把smw = Similarword(...)放全局,或者当成参数传入。
from nlpcda.
赞
from nlpcda.
Related Issues (20)
- Simbert参数问题 HOT 3
- 希望能添加的新功能 HOT 1
- 请问使用simbert生成的时候报错是什么原因 HOT 1
- 关于:if t > self.create_num * self.loop_t / self.change_rate HOT 1
- 词表使用中文名称,使用pycharm debug时报编码错误,是否可以用英文命名?
- 请问你是有对simbert进一步改进吗? HOT 3
- NER类数据增强
- 请问一下simbert怎么生成不相似的文本 HOT 1
- 使用百度翻译报错,什么原因呢 HOT 2
- ImportError: cannot import name 'Simbert' from 'nlpcda' HOT 4
- 简化了一版
- Simbert将不能正常使用,除非你安装keras后端:tensorflow .... type object 'AutoRegressiveDecoder' has no attribute 'wraps' HOT 3
- Simbert不能正常使用,除非你安装:bert4keras、tensorflow HOT 1
- KeyError: 'CUDA_VISIBLE_DEVICES' HOT 1
- 第三方包报错! HOT 2
- 使用除了simbert之外的方法也报错 HOT 1
- 不能使用NER HOT 1
- 如果能支持定义stopwords 就更好了 HOT 1
- 近音字替换 HOT 1
- pls
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nlpcda.