Comments (3)
Looks like this is both a bug in the ToMixedPrecision
pass, and a limitation in the well-formed checker. The bug is that the current implementation of ToMixedPrecision
implicitly assumes that all tensors are float32 before the pass is applied. As a result, when the annotation MixedPrecisionPolicyKind::kNever
(that is, never change the dtype of this operator) is encountered for R.call_tir
, it casts the input/output of R.call_tir
to float32.
The limitation in the well-formed checker is that the callee in R.call_tir
doesn't get inspected to identify this issue when it first occurs, so it only is caught later by FuseTIR
. The StructuralEqual()
check looks like it was intended to catch a mismatched TIR buffer shape, but serendipitously caught the mismatched dtype between the output of fused_layer_norm_cast
and the input of conv2d_cast_relu
.
To fix this, I think the ToMixedPrecision
pass will need updated behavior for kNever
. Where currently, when an operator with mixed-precision policy of kNever
will cast all inputs to float32
, it should instead cast all inputs to the same dtype as they had prior to ToMixedPrecision
.
from tvm.
@Lunderberg Thanks for your deep investigation and quick fixing.
from tvm.
And should be fixed in #17263
from tvm.
Related Issues (20)
- [Bug] [Relax] InternalError: Check failed: last_sep < self->shape.size() Last output axis must contain at least one input axis HOT 3
- [Bug] [Relax] VMBuiltinLower expects bound value to be a ShapeExpr HOT 3
- [Bug] [Relax] Variable was used before its definition HOT 2
- [Bug] [Relax] Argument type mismatch: expected R.Tensor, given R.Tuple HOT 1
- [Bug] CreateModulePass has an inconsistent para_format between BindSymbolicVars and other pass HOT 3
- [Bug] [Relax] Expect a not null value of IRModule HOT 4
- [Bug] [Relax] Check failed: (it != this->var_arg_map_.end()) is false: Var is not defined HOT 3
- [Bug] [Relax] InternalError: Check failed: type_code_ == kTVMObjectHandle expected Object but got int HOT 4
- target.build.llvm is not enabled HOT 1
- [Bug] [Relax] Segfault error when parse the Relax IR HOT 4
- [Bug] output different results when inference twice with the same inputs HOT 3
- [Bug] Cannot add loops on top of the root block HOT 2
- [Bug] RemoveUnusedOutputs give unexpected results HOT 4
- [Bug] [Relax] Hug inference performance difference on cuda and cpu HOT 3
- [Bug] The entry value of attr should be integer. However Array is got HOT 2
- [Bug] TVM Cross Compilation with QNX for arm
- [Bug] VTA RPC Error
- [Bug] AssertionError in the LazyTransformParams
- [Bug] give inconsistent inference results after use ReorderTakeAfterMatmul
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tvm.