Comments (5)
Thank you raising this issue.
A sample with int8x32 has been created for you to look at.
auto tensor_x = cudnn_frontend::TensorBuilder()
.setDim(4, x_dim_padded)
.setStrides(4, strideA_padded)
.setId('x')
.setAlignment(16)
.setDataType(CUDNN_DATA_INT8)
.setVectorCountAndDimension(vectorCount, vectorDimension)
.build();
Note: There are numerical issues with engine_id=0 and int8x32 vectorCount and has been added to errata.
from cudnn-frontend.
Thanks for reply.
I have referred to the samples with int8x32 from:
1、one sample you provided, conv_op with int8x32(x y w is int8x32)
2、one sample from one developer in github, conv_op+add_op+bias_op+activation_op(x y w z b is all int8x4; after modification, also run int8x32 type with inner imma api)
For project application, I need test the conv_op+scale_op+bias_op+activation_op case(same as using cudnnFusedOpsExecute), in which
datatype: x y w is all int8x32, scale and bias is float(fp32),
dim: x[N, IC, IH, IW], y[N, OC, OH, OW], z[OC, IC, KH, KW], scale[N, OC, 1, 1], bias[N, OC, 1, 1]
According to the cudnn8 developer guide manual, it supports Convolution_Pointwise flexibly when the compute capability is above 7.5.
Therefore, referred to the samples above, I have write my conv_op+scale_op+bias_op+activation_op sample(x y w is all int8x32, scale and bias is float), but can not find the supported engine for cudnnBackendExectue(), can you give a sample? Or can you correct the error in my code?
from cudnn-frontend.
Hi Zhao,
Apologies for the delayed response. We found we do not support the above data type combination because of an internal bug. We have a fix for this and will be part of our future 8.3.0 release and will supported through the run time fusion.
Thanks
from cudnn-frontend.
Hi Zhao,
We have fixed this issue in cudnn v8.3 and have released a sample ConvScaleBiasAct_int8 sample for the same. Let us know if it addresses your use case.
Thanks
from cudnn-frontend.
Hi @ZhaoJob hope the responses above answer your questions! I'm closing the issue for now. If you have additional questions, please feel free to open a new issue!
Thanks
from cudnn-frontend.
Related Issues (20)
- Why implicit_convolveNd_hhgemm consume too long HOT 3
- Inference result of deep learning model is all NAN HOT 4
- need default return value for cudnn_frontend::PointWiseDesc_v8::getPortCount() const HOT 1
- About cudnn backend
- Many samples don't work for me HOT 5
- INT8 sample didn't work? HOT 1
- Execute matmul op faild HOT 13
- Is dgrad+relu with fp32 supported? HOT 4
- question about the fusion_sample HOT 1
- error: ambiguous overload for ‘operator*’ in test_list.cpp HOT 2
- Update single header file for nlohmann json HOT 2
- What's the difference of flash attention implement between cudnn and Dao-AILab? HOT 19
- Cudnn Error InstanceNormalizationPlugin HOT 5
- CUDNN_FRONTEND_BUILD_UNIT_TESTS option is broken HOT 2
- Support use of external/system Catch2 installation HOT 2
- Support "make install" HOT 1
- Windows build error HOT 3
- identifier "geomlib::_NV_ANON_NAMESPACE::kEps" is undefined in device code HOT 2
- Why is graph::check_support really slow? HOT 4
- [ERROR] Exception CUDNN_BACKEND_TENSOR_DESCRIPTOR cudnnFinalize failed cudnn_status: CUDNN_STATUS_NOT_INITIALIZED HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cudnn-frontend.