Deep Replay - Generate visualizations as in my "Hyper-parameters in Action!" series!
Home Page: https://towardsdatascience.com/hyper-parameters-in-action-a524bf5bf1c
License: MIT License
I ran into this error while trying to fit my model and i havent been able to fix it:
AttributeError: 'function' object has no attribute 'encode'
I'm assuming that it doesn't work with tf.keras yet (or there are different bugs).
Is that correct or did I just do something stupid?
I ask this question because when I tried it with a simple model from a example I got some odd errors.
PS. Thank you; your article was really great, and I really like the visualizations, they are very helpful. Random question, are all the things you pointed out the same in Convolution layers as in your Dense examples, or do the visuals differ?
PSS. One thing I have trouble with is figuring out is which shapes/differences are problems and which are just harmless variations. If you had a cheat sheet explaining which things to look out for in each of the 4 graphs that would be great.
Note: This is not an bug/error report. I just thought I should provide more details.
With the simple model I got odd errors like this;
AttributeError Traceback (most recent call last)
<ipython-input-16-d1a685d6c63e> in <module>()
----> 1 replaydata = ReplayData(x_test, y_test, filename="filename", group_name="some group", model=model)
2
3 # Now we feed the data to the actual Replay object so we can build the visualizations
4 replay = Replay(replay_filename="filename", group_name="some group")
5
4 frames
/usr/local/lib/python3.6/dist-packages/h5py/_hl/base.py in _e(self, name, lcpl)
130 else:
131 try:
--> 132 name = name.encode('ascii')
133 coding = h5t.CSET_ASCII
134 except UnicodeEncodeError:
AttributeError: 'MeanMetricWrapper' object has no attribute 'encode'
Then, I changed the tf.keras to just Keras and it worked
(except the OOM exception on the GPU; ran in Google Colab).
Here is the whole Workbook: https://gist.github.com/Raukk/2b03bd8447433b83f862c7e7c437ec46
The model code (using tf.keras) is pasted below if you want to look into it.
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.BatchNormalization(input_shape=x_train.shape[1:]))
model.add(tf.keras.layers.Conv2D(64, (5, 5), padding='same', activation='elu'))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2,2)))
model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.BatchNormalization(input_shape=x_train.shape[1:]))
model.add(tf.keras.layers.Conv2D(128, (5, 5), padding='same', activation='elu'))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))
model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.BatchNormalization(input_shape=x_train.shape[1:]))
model.add(tf.keras.layers.Conv2D(256, (5, 5), padding='same', activation='elu'))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2,2)))
model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(256))
model.add(tf.keras.layers.Activation('elu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(10))
model.add(tf.keras.layers.Activation('softmax'))
model.summary()
Bug report
Run code from Part II — Weight Initializers, got this error.
InvalidArgumentError Traceback (most recent call last)
<ipython-input-5-12bc6e8cd13c> in <module>()
1 zv = replay.build_outputs(ax_zvalues,before_activation=True,
----> 2 exclude_outputs=True, include_inputs=False)
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/deepreplay/replay.py in build_outputs(self, ax, before_activation, layer_names, include_inputs, exclude_outputs, epoch_start, epoch_end)
468 inputs = [self.learning_phase, self.inputs] + weights
469 if before_activation:
--> 470 outputs.append(self._get_zvalues(inputs=inputs))
471 else:
472 outputs.append(self._get_activations(inputs=inputs))
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
2659 return self._legacy_call(inputs)
2660
-> 2661 return self._call(inputs)
2662 else:
2663 if py_any(is_tensor(x) for x in inputs):
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in _call(self, inputs)
2628 feed_symbols,
2629 symbol_vals,
-> 2630 session)
2631 fetched = self._callable_fn(*array_vals)
2632 return fetched[:len(self.outputs)]
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in _make_callable(self, feed_arrays, feed_symbols, symbol_vals, session)
2580 callable_opts.target.append(self.updates_op.name)
2581 # Create callable.
-> 2582 callable_fn = session._make_callable_from_options(callable_opts)
2583 # Cache parameters corresponding to the generated callable, so that
2584 # we can detect future mismatches and refresh the callable.
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/tensorflow/python/client/session.py in _make_callable_from_options(self, callable_options)
1478 """
1479 self._extend_graph()
-> 1480 return BaseSession._Callable(self, callable_options)
1481
1482
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/tensorflow/python/client/session.py in __init__(self, session, callable_options)
1439 else:
1440 self._handle = tf_session.TF_DeprecatedSessionMakeCallable(
-> 1441 session._session, options_ptr, status)
1442 finally:
1443 tf_session.TF_DeleteBuffer(options_ptr)
~/miniconda3/envs/rlgym/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
517 None, None,
518 compat.as_text(c_api.TF_Message(self.status.status)),
--> 519 c_api.TF_GetCode(self.status.status))
520 # Delete the underlying status object from memory otherwise it stays alive
521 # as there is a reference to status from this from the traceback due to
InvalidArgumentError: hidden_input_1:0 is both fed and fetched.
Enveriment
keras 2.2.0
tensorflow 1.8.0
Can you tell me what package versions of tensorflow and keras I should be using in order to run this code? Specifically, the example I've been trying to recreate is from your post here.
I see the requirements.txt says tensorflow==1.5.0 but that doesn't appear to be a distribution available through pip.
Hey
I am having this custom activation function which is not listed in the replay.ACTIVATIONS nor in the keras package
Although I am able to Train the Keras model with custom activation function, How do I generate the plot for custom activation function in deepreplay?
on running the below cell
replay = Replay(replay_filename='spambase_dataset.h5', group_name=group_name)
fs = replay.build_feature_space(ax_fs, layer_name='hidden', scale_fixed=False)
ph = replay.build_probability_histogram(ax_ph_neg, ax_ph_pos)
lh = replay.build_loss_histogram(ax_lh)
lm = replay.build_loss_and_metric(ax_lm, 'acc')
I get the following error
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-31-f06018e9e1b6> in <module>()
1
----> 2 replay = Replay(replay_filename='spambase_dataset.h5', group_name=group_name)
3
4 fs = replay.build_feature_space(ax_fs, layer_name='hidden', scale_fixed=False)
5 ph = replay.build_probability_histogram(ax_ph_neg, ax_ph_pos)
15 frames
/usr/local/lib/python3.6/dist-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
165 if fn is None:
166 raise ValueError('Unknown ' + printable_module_name +
--> 167 ':' + function_name)
168 return fn
169 else:
ValueError: Unknown activation function:swish
getting KeyError: 'samples' error when trying the sample code.
KeyError Traceback (most recent call last)
c:\Users..modelling.py in
19 m.compile(loss='binary_crossentropy', optimizer=SGD(lr=0.05), metrics=['acc'])
20
---> 21 m.fit(xx, yy, epochs=150, batch_size=16, callbacks=[replaydata])
~\Documents\Programs\Anaconda3\envs\base\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.traceback)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
~\Documents\Programs\Anaconda3\envs\base\lib\site-packages\deepreplay\callbacks.py in on_train_begin(self, logs)
84
85 self.group = self.handler.create_group(self.group_name)
---> 86 self.group.attrs['samples'] = self.params['samples']
87 self.group.attrs['batch_size'] = self.params['batch_size']
88 self.group.attrs['n_batches'] = (self.params['samples'] + self.params['batch_size'] - 1) // \
KeyError: 'samples'
Using coda,
deepreplay version is 0.1.2a2
keras version is 2.4.0
Hey How could assign this method on ResNet? I tried but it throws out the following error:
model.summary()
==================================================================================================
_________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 1, 1, 512) 2359808 activation_32[0][0]
__________________________________________________________________________________________________
add_16 (Add) (None, 1, 1, 512) 0 add_15[0][0]
conv2d_36[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 1, 1, 512) 2048 add_16[0][0]
__________________________________________________________________________________________________
activation_33 (Activation) (None, 1, 1, 512) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 1, 1, 512) 0 activation_33[0][0]
__________________________________________________________________________________________________
flatten_1 (Flatten) (None, 512) 0 average_pooling2d_1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 10) 5130 flatten_1[0][0]
==================================================================================================
from keras.callbacks import TensorBoard
tensorboard = TensorBoard(log_dir='./logs_ResNet34', histogram_freq=1,
write_graph=True, write_images=True)
# define model
model.fit(X_train, Y_train,
batch_size=batch_size,
epochs=nb_epoch,
validation_data=(X_test, Y_test),
shuffle=True,
callbacks=[lr_reducer, early_stopper, csv_logger, tensorboard, replaydata])
filename = 'hyperparms_in_action.h5'
group_name = 'part2'
replaydata = ReplayData(X_train, Y_train, filename=filename, group_name=group_name, model=model)
fig, ax = plt.subplots(1, 1, figsize=(5, 5))
fs = replay.build_feature_space(ax, layer_name='dense_1') #AssertionError: Only layers with 2-dimensional outputs are supported!
#fs = replay.build_decision_boundary(ax_fs) #NameError: name 'ax_fs' is not defined
fs.plot(epoch=60).savefig('feature_space_epoch60.png', dpi=120)
fs.animate().save('feature_space_animation.mp4', dpi=120, fps=5)How can I fix that?
Thank you for the job done, it's really impressive.
I have encountered a problem when working with custom metrics. Indeed, custom metrics are passed to the model as functions, not strings, which does not work with self.group.create_dataset(metric, shape=(self.n_epochs,), dtype='f')
Here is the error I get when running :
File "C:\Users\AppData\Local\Continuum\anaconda3\lib\site-packages\deepreplay\callbacks.py", line 62, in __init__ self.on_train_begin()
File "C:\Users\AppData\Local\Continuum\anaconda3\lib\site-packages\deepreplay\callbacks.py", line 109, in on_train_begin self.group.create_dataset(metric, shape=(self.n_epochs,), dtype='f')
File "C:\Users\AppData\Roaming\Python\Python36\site-packages\h5py\_hl\group.py", line 119, in create_dataset self[name] = dset
File "C:\Users\AppData\Roaming\Python\Python36\site-packages\h5py\_hl\group.py", line 284, in __setitem__ name, lcpl = self._e(name, lcpl=True)
File "C:\Users\AppData\Roaming\Python\Python36\site-packages\h5py\_hl\base.py", line 132, in _e name = name.encode('ascii')
AttributeError: 'function' object has no attribute 'encode'
Hi recently I faced a problem when I used this package as its Traceback is following:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-43-c3b5d8180301> in <module>()
----> 1 model.fit(X, y, epochs=50, batch_size=16, callbacks=[replay])
2 frames
/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
/usr/local/lib/python3.7/dist-packages/deepreplay/callbacks.py in on_train_begin(self, logs)
83 self.n_epochs = self.params['epochs']
84
---> 85 self.group = self.handler.create_group(self.group_name)
86 self.group.attrs['samples'] = self.params['samples']
87 self.group.attrs['batch_size'] = self.params['batch_size']
/usr/local/lib/python3.7/dist-packages/h5py/_hl/group.py in create_group(self, name, track_order)
63 name, lcpl = self._e(name, lcpl=True)
64 gcpl = Group._gcpl_crt_order if track_order else None
---> 65 gid = h5g.create(self.id, name, lcpl=lcpl, gcpl=gcpl)
66 return Group(gid)
67
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/h5g.pyx in h5py.h5g.create()
ValueError: Unable to create group (name already exists)I shared the google colab notebook which I could run on Aug 2, 2019, but now it threw out this KeyError: 'samples' in 2021. Please check the notebook and feel free to run it for quick debugging.
Following is the configuration and package versions in google colab:
matplotlib==3.2.2
matplotlib-inline==0.1.3
matplotlib-venn==0.11.6
numpy==1.19.5
pandas==1.1.5
pandas-datareader==0.9.0
pandas-gbq==0.13.3
pandas-profiling==1.4.1
scikit-learn==1.0.1
scipy==1.4.1
seaborn==0.11.2
sklearn-pandas==1.8.0
3.7.12
Python 3.7.12
This is the full code:
from keras.models import Sequential
from keras.layers import Dense
#from keras.optimizers import SGD
from tensorflow.keras.optimizers import SGD
from keras.initializers import glorot_normal, normal
model = Sequential()
model.add(Dense(input_dim=2,
units=2,
activation='sigmoid',
kernel_initializer=glorot_normal(seed=42),
name='hidden'))
model.add(Dense(units=1,
activation='sigmoid',
kernel_initializer=normal(seed=42),
name='output'))
model.compile(loss='binary_crossentropy', optimizer=SGD(learning_rate=0.05), metrics=['acc'])
from deepreplay.callbacks import ReplayData
from deepreplay.datasets.parabola import load_data
from deepreplay.replay import Replay
X, y = load_data()
replay = ReplayData(X, y, filename='hyperparms_in_action.h5', group_name='part1')
model.fit(X, y, epochs=50, batch_size=16, callbacks=[replay])I found the similar open issues here & here related to saving model via hdf5 file. I tried based on some suggestions 12195# and here to save the model with tf instead of h5 in replay_filename which was unsuccessful. There is lots of post in this regard in SoF
Any helps will be highly appreciated.
Sorry, there's a problem that has troubled me for a long time.
Why is the boundary of the transformed feature space linear? And why is the boundary of the original feature space nonlinear?
Is it your code that forces the boundaries of the transformed feature space to become linear boundaries?
Thank you so much.
How to use this library if I am using fit_generator to load images.
My datasets are huge, so this will be needed.
Thanks in advance.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.