mainak431 / list-of-differentiable--ops-and-non-differentiable-ops--in-tensorflow Goto Github PK

I thought there was no gradient defined in TensorFlow for round?

Note for python tensorflow API users... to find the tensorflow equivalent class names, you can search for them in tensorflow/python/ops/array_ops.py, and look for which tensorflow operator it is getting mapped to for your arguments. Often this can be a constant, or not. In my case, where_v2 maps to select_v2, which is probably SelectV2, which shows up in the output.

$ rg Select
DIFFERENTIABLE LIST.txt
283:"Select"

README.md
308:"Select" \

DIFFERENTIABLE.TXT
220:"Select"
221:"SelectV2"

There are probably operators that are differentiable, or not, depending on the arguments.

I think currently the list of differentiable OPs is incorrect.

Take ArgMax as an example:

https://github.com/tensorflow/tensorflow/blob/87462bfac761435a46641ff2f10ad0b6e5414a4b/tensorflow/python/ops/math_grad.py#L37

It returns [None, None].
And I guess, that is why I'm still getting:
ValueError: No gradients provided for any variable
when I use a Lambda-Layer with tf.math.argmax within keras.
So my guess is that ArgMax is indeed not differentiable.

But maybe I'm just missing something?

Example code:

import tensorflow as tf
import tensorflow.keras.backend as K
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
from tensorflow.keras.optimizers import *

def ArgMax():
    return Lambda(lambda q:tf.math.argmax(q, 2))
inputs1 = Input(shape = (1,1,))
d = Dense(3)(inputs1)
out = ArgMax()(d)

model = Model(inputs = [inputs1], outputs = [out], name = "test")

model.summary()
model.compile(SGD(0.1),loss="mae")
model.fit(
        x = [np.ones((100,1,1))],
        y = [np.zeros((100,1))],
        batch_size = 100,
        verbose = 2,
        epochs = 10)

Alternatively, you could just focus on the list of known differentiable functions, and include a "since" flag when you detect a version that doesn't have AD.

Your script ran successfully over a tensorflow 1.14 installation from conda, by the way. I would include it, but if you do this, you probably want to automate checking out the versions and gathering all the info anyway.

Thanks!