Comments (8)
benfred
commented on June 13, 2024
3
Alberto has reduced inference container to 17GB (uncompressed) from 27GB previously by minimizing system and python packages that are being installed, and is working on pulling from internal rapids image to further reduce sizes.
from merlin.
albert17
commented on June 13, 2024
3
Completed. We have now 3 inference containers
from merlin.
albert17
commented on June 13, 2024
Done:
- Minimized System Packages
- Minimized Python Packages
- Minimized Dependencies
Doing/To-DO:
- Reporting size per component
- Trying Tritonserver smaller base images: tf/pyt
- Rapids from internal image
- HugeCTR dependencies as optional
from merlin.
albert17
commented on June 13, 2024
| Component | Size (GB) | Size increase (GB) |
|---|---|---|
| Base | 12.4 | 0 |
| System Packages | 12.7 | 0.3 |
| Python Packages | 13.7 | 1 |
| Cmake | 14 | 0.3 |
| spdlog | 14 | 0 |
| Arrow | 14.3 | 0.3 |
| rmm | 14.3 | 0 |
| cuDF | 14.9 | 0.6 |
| Merlin Core | 14.9 | 0 |
| Merlin NVTabular | 15 | 0.1 |
| Merlin Transformers4Rec | 15.1 | 0.1 |
| Merlin Models | 15.1 | 0 |
| Merlin NVTabular Triton Backend | 15.1 | 0 |
| Hiredis | 15.1 | 0 |
| redis++ | 15.1 | 0 |
| RocksDB | 15.4 | 0.3 |
| LibRdKafka | 15.5 | 0.1 |
| Java | 15.9 | 0.4 |
| libhdfs | 17.1 | 1.2 |
| HugeCTR | 17.9 | 0.8 |
from merlin.
albert17
commented on June 13, 2024
Created 2 new images
- merlin-inference-tf: 12.6 GB
- merlin-inference-pyt: 13.4 GB
from merlin.
viswa-nvidia
commented on June 13, 2024
@albert17 , please check off the completed items at the top of this ticket in the to do . Is there anything blocking for 22.03 ?
from merlin.
viswa-nvidia
commented on June 13, 2024
👍
from merlin.
karlhigley
commented on June 13, 2024
🎉
from merlin.
Related Issues (20)
- [BUG] The multi-stage example is broken due to recent changes in systems HOT 1
- [RMP] Add support for ranking models in PyTorch
- POC on how to build a session-based recommendation pipeline that can deal with the item cold-start problem
- POC on how to build a session-based recommendation model that can be used to re-rank candidate items
- [Task] Centralize API Documentation in Merlin
- [RMP] Update Merlin Models TensorFlow API to Match PyTorch API
- [BUG] User cannot deploy Merlin image >=23.04 on Azure Databricks
- [QST] Where to get tensorflow2.10.1+nv22.12 source code ?
- [QST] How to serve merlin-tensorflow model in Triton Inference Server and convert it to ONNX? HOT 1
- Use MLflow Experiments with Merlin containers[QST] HOT 5
- [BUG] Merlin io - ModuleNotFoundError: No module named 'cudf._version' HOT 2
- [QST]Follow the example 'getting started movies' to execute an error. HOT 3
- [BUG] FileNotFoundError when apply Categorify after JoinExternal HOT 1
- [BUG] CUDA context error HOT 1
- [BUG]Unauthorized with docker build HOT 1
- [QST] Status: CUDA driver version is insufficient for CUDA runtime version HOT 1
- [QST] What is the best way of handling string UUIDs in Merlin? HOT 2
- [QST] Help w/ exporting Retrieval Model. HOT 5
- [QST] What is the best way of handling string UUIDs in Merlin? HOT 1
- [QST] How to do normal retrieval of candidates without starting a server HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from merlin.