Comments (12)
@williamFalcon interesting article and I agree that in all the cases user needs to learn the API of the library. Features provided by your package are interesting and some of them are also indirectly supported by Ignite without imposing the package. For example, if user would like to use horovod with pytorch and fp16 instead of nvidia/apex, he could code it and insert into the libraries frame.
Similalry, for the reproducibility, as far as I understand the package is coupled with test-tube
and maybe this can be a limitation if user would like to use mlflow, polyaxon or similiar...
Anyway, I'm not familiar with your API in details.
Concerning your tables "HPC" and "Debugging" in Ignite column I would add the "checks" for tensorboard (link), Learning-rate warm-up and actually we can schedule any optimizer parameter (not only lr), Gradient inspection with tensorboard, "Nan gradient inspection" - we have TerminateOnNan handler, "Train/Validate on subset of data" it is up to user to define and it is posible too in ignite.
Other features:
- custom events : e.g. custom epoch as a number of iterations
- out-of-box metrics for classification, segmentation, regression
So if you could include this, it will be cool :)
And definitly, if we could have a guy like you to communicate about Ignite :)
from lightning.
On a side-note, @vfdev-5 any interest in just merging ignite with Lightning? You guys could bring in some of the things you've been working on into Lightning.
That way we can all focus on fixing the reproducibility issue.
from lightning.
@williamFalcon I would prefer to speak about merging lighting with Ignite :)
As the latter is older and used by at least 105 projects according to github (e.g https://medium.com/huggingface/how-to-build-a-state-of-the-art-conversational-ai-with-transfer-learning-2d818ac26313).
If you are interested, we can discuss about this in pytorch slack in pytorch-ignite channel in details.
from lightning.
Hello, thank you such explanation, maybe you may consider a table with features in rows and framework in columns and check-list if the framework has it or not...
Btw, I was a bit puzzled by the very detail granulity of your releases, don't you think that the 4th (e.g. 0.3.6.8) is too much? I would have rather higher number on the 3rd place and also I would say that each commit does not have to be separate release... :)
from lightning.
I think that you can drop the sub-minor releases and keep only the minor... :)
from lightning.
Makes sense. In keeping with allowing as much flexibility as possible for researchers, let's add support for any logger they want to use.
See #47
from lightning.
not yet. working on those. however, the graphic in the README should make this very clear.
At a base level, lightning implements distributed GPU, multi-node, 16-bit, auto-checkpoint and resubmit for cluster training. The learning curve with lightning is also lower since it doesn’t make unnecessary wrappings.
Further, lightning is also very thoroughly documented and well tested as is ignite.
Basically, if you want to retain 99% of the control and are ok not having to deal with your own training loop, checkpointing, distributing, etc... then lighting is for you. It’s designed to give academics the most control without oversimplifying or making a whole new library someone has to learn.
from lightning.
Yeah, that was the development stage. We were getting key features in-place. It’s stable now, we’ll batch more features and do bigger releases.
Wanted to get rid of the major issues to have good first adoption
from lightning.
@williamFalcon I would be also curious to compare both libraries and understand the idea behind the creation of pytorch-lightning
as an alternative to existing frameworks.
from lightning.
@vfdev-5 @Borda
https://medium.com/@_willfalcon/pytorch-lightning-vs-pytorch-ignite-vs-fast-ai-61dc7480ad8a
(@vfdev-5 if i missed anything about ignite, lmk, happy to edit!)
from lightning.
I would also tend to use mlflow
or polyaxon
intead test-tube
since they seem to have much wider/larger support... :)
from lightning.
No worries, i prefer to keep both libraries separate then. Researchers can pick whichever makes sense for them :)
from lightning.
Related Issues (20)
- Trainer does not wait for neptune logger completion and logger connection stays open unless explicitly closed HOT 1
- Validation does not produce any output in PyTorch Lightning using my UNetTestModel
- Unable to extend FSDPStrategy to HPU accelerator HOT 7
- SaveConfigCallback.save_config is conflict with DDP HOT 1
- Logging Documentation Does not Detail How to Access the Logged Values during the fit loop
- Apply the ignore of the save_hyperparameters function to args as well.
- Cannot run in SLURM Interactive Session
- Resume from mid steps inside an epoch
- `DDPStrategy` fails when using accelerators other than CUDA
- PyTorch Lightning with T5 Model - RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn HOT 1
- Script freezes when Trainer is instantiated
- Sanitize object params before they get logged from argument-free classes
- Support GAN based model training with deepspeed which need to setup fabric twice HOT 2
- IndexError: Pytorch-lightning CompositionalMetric require tensor.item() if dim=0 whether I did so
- Huge metrics jump between epochs && Step and epoch log not matched, when accumulate_grad_batches > 1
- Does `fabric.save()` save on rank 0? HOT 3
- Turn off hpc checkpoint saving in SLURM environment if trainer.fit(..., ckpt_path="last") HOT 3
- DDP strategy doesn't work for on_validation_epoch_end, always hang HOT 4
- TensorBoardLogger does not document .add_image() function
- Passing a dataloader to save_hyperparams hangs trainer.fit
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightning.