Comments (2)
Hi Philip,
Thanks for your interest, the ConvBERT base model is trained on a TPU v3-8 (which has 8 cores). Our code is based on TensorFlow 1.x and the current codebase does not support multi GPU training. You'd better use TPU for training large models if you use this codebase.
from convbert.
Ok. Thanks @zihangJiang !
Closing this again.
from convbert.
Related Issues (20)
- 用自己的数据预训练 各种nah loss 问题 HOT 3
- 关于mixed-attention推理速度的问题 HOT 2
- 请问你提供的预训练模型是中文预训练模型 还是英文 是基于什么进行训练的 细节可以稍微介绍下吗 HOT 1
- 请问有pytorch版本发布吗? HOT 1
- What's the essential difference between ConvBert and LSRA? HOT 3
- The exact English pretraining data and Chinese pretraining data that are exact same to the BERT paper's pretraining data.
- Train on GPU instead of TPU - differnt distribution strategies HOT 2
- Where is the chinese convbert model?
- 请问有计划开源中文的模型吗 HOT 1
- Please update your citation bib HOT 1
- Is ConvBertModel autoregressive?
- UnboundLocalError: local variable 'seq_length' referenced before assignment
- 请问使用tpu还是gpu训练 HOT 7
- Pytorch version HOT 2
- 预测性能 HOT 4
- 疑惑 HOT 1
- 关于预训练的问题 HOT 1
- span light conv疑惑 HOT 1
- 这个预训练代码不就是ELECTRA那套? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from convbert.