Comments (1)
Related Issues (20)
- [QUESTION] Using segformer segmentation models
- [QUESTION] Question about Mixtral compatibility with Megatron-LM core0.7.0
- [BUG] megatron.training not found HOT 7
- [QUESTION] How to time the code
- [BUG] pipeline_paralle is not available when pp_size > 2 HOT 1
- [BUG] RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead. HOT 2
- [QUESTION]when pretraining bert,meet bug:cuBLAS Error: the requested functionality is not supported HOT 3
- [QUESTION] Gloo connectFullMesh failed when the number of nodes setting "export GLOO_SOCKET_IFNAME=bond4" exceeds 60
- [QUESTION] OSError: [Errno 28] No space left on device HOT 4
- [QUESTION] --overlap-grad-allreduce failing as gradients coming through as None in param hook HOT 2
- [BUG] @jit_fuser fails with Unknown type constructor Sequence HOT 6
- [BUGS] Pipeline Parallelism fails/hangs with Megatron Core example HOT 2
- [QUESTION] Why is TELayerNormColumnParallelLinear used instead of TEColumnParallelLinear in gpt_layer_specs HOT 2
- [QUESTION] Why does the tokenizer of mamba-2-hybrid have two ids for the token 'Yes'? id 24639 and id 7298 HOT 1
- [QUESTION] Has standalone_embedding_stage been supported yet in core? HOT 1
- [QUESTION] Sample idx, bin files in public domain for trying out pretrain_gpt.py? HOT 2
- [QUESTION] Getting tools/preprocess_data.py to work is painful
- [REGRESSION] MoEs are obtaining higher loss than they should during training HOT 10
- [BUG]Question about helpers.cpp in version core_v0.7.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from megatron-lm.