The example from custom_series notebook doesn't work:

TypeError: init() got an unexpected keyword argument 'series_fmt'

Was series_fmt parameter in an older version?

Live loss plot work almost perfect. But in regression problems there is the interest in Live Regression Plot, that can show us how predicted curve tends to real test curve during the training epochs.

Is it hard for you to do this kind callback based on livelossplot callback class?

It would be very useful in case jupyter closes or training cell is rerun - plots will get lost.
Please consider adding this feature or maybe tell me how could I make it without manually modifying sources.

I am thinking what is the best amount of data displayed?
Maybe it is good to add other features - like the numerical value of current metrics, or maximal values.

Though, I don't want to make it too cluttered. So, I am interested in your feedback what is missing (or what you would consider more a distraction).

It the end of the validation process it throws an TypError at the line:

draw_plot(logs, metrics, figsize, max_epoch, max_cols, validation_fmt, metric2title)
26
27 if max_epoch is not None:
---> 28 plt.xlim(1, max_epoch)
29
30 plt.plot(range(1, len(logs) + 1),

TypeError: 'int' object is not callable

How can i prevent this?

How about plotting in livelossplot and having an option to seamlessly save TensorBoard data?
So you can have plots now, but if needed - explore it later.

Interesting in picking this issue? You are invited to make a Pull Request! :)

I've noticed the answer to the (The Most FA)A, but I've been trying to find a way, looking at the minimal bare API example and TF's training interactions, to make it work in a TF environment.

Has anyone been succcessful at this?

Hi,

I'm getting this error 'RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam figure.max_open_warning).
max_open_warning, RuntimeWarning)'

It seems to be opening a new graph for every epoch, anyone know a fix for this?

Hi I am using https://github.com/qfgaohao/pytorch-ssd/blob/master/train_ssd.py code in jupyter notebook but the loss is getting updated and plotted for that epoch,giving a line parallel to x-axis.
liveloss.txt

There is a problem with not-a-numbers. For example if monitored metric (e.g. F1) is nan after first epoch min and max values remain not-a-number during whole training. For example:
F1:
training (min: nan, max: nan, cur: 0.75).

I'm training a CNN model and I wanted to both a) be able to reduce the optimizer learning rate when it hits a plateau with ReduceLROnPlateau() and b) visually monitor the losses with livelossplot's PlotKerasLosses(). This combination in callbacks gives me a KeyError: 'lr'. When dropping either one of the functions the training works as it should. Hence my guess is that there must be a compatibility issue with the two.

Keras = 2.2.4
livelossplot = 0.3.4

Here's some code:

from keras.callbacks import ReduceLROnPlateau
from livelossplot import PlotLossesKeras

...

learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', 
                                            patience=3, 
                                            verbose=1, 
                                            factor=0.5, 
                                            min_lr=0.00001)

history = model.fit_generator(datagen.flow(X_train,Y_train, batch_size=batch_size),
                              epochs = epochs, 
                              validation_data = (X_val,Y_val),
                              verbose = 1, 
                              steps_per_epoch=X_train.shape[0] // batch_size,
                              callbacks=[learning_rate_reduction, PlotLossesKeras()])

And here's the error I get:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-26-bd8223167e59> in <module>
      5                               verbose = 1,
      6                               steps_per_epoch=X_train.shape[0] // batch_size,
----> 7                               callbacks=[learning_rate_reduction, PlotLossesKeras()])

~/anaconda3/envs/upwork/lib/python3.7/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~/anaconda3/envs/upwork/lib/python3.7/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

~/anaconda3/envs/upwork/lib/python3.7/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
    249                     break
    250 
--> 251             callbacks.on_epoch_end(epoch, epoch_logs)
    252             epoch += 1
    253             if callback_model.stop_training:

~/anaconda3/envs/upwork/lib/python3.7/site-packages/keras/callbacks.py in on_epoch_end(self, epoch, logs)
     77         logs = logs or {}
     78         for callback in self.callbacks:
---> 79             callback.on_epoch_end(epoch, logs)
     80 
     81     def on_batch_begin(self, batch, logs=None):

~/anaconda3/envs/upwork/lib/python3.7/site-packages/livelossplot/generic_keras.py in on_epoch_end(self, epoch, logs)
     63 
     64     def on_epoch_end(self, epoch, logs={}):
---> 65         self.liveplot.update(logs.copy())
     66         self.liveplot.draw()

~/anaconda3/envs/upwork/lib/python3.7/site-packages/livelossplot/generic_plot.py in update(self, log)
     80         self.logs.append(log)
     81         if self.plot_extrema:
---> 82             self._update_extrema(log)
     83 
     84     def draw(self):

~/anaconda3/envs/upwork/lib/python3.7/site-packages/livelossplot/generic_plot.py in _update_extrema(self, log)
     69     def _update_extrema(self, log):
     70         for metric, value in log.items():
---> 71             extrema = self.metrics_extrema[metric]
     72             if _is_unset(extrema['min']) or value < extrema['min']:
     73                 extrema['min'] = float(value)

KeyError: 'lr'

I would like to see if there is a way to support printing the graph update after N batches rather than per epoch as, some datasets are in 100k+ it would be nice to get intermediate results.

@stared
Hi. I would like to have two small features, take a look here :

Mean squared error (cost function):
training (min: 0.000, max: 0.000, cur: 0.000)
validation (min: 0.000, max: 0.000, cur: 0.000)

Accuracy:
training (min: 0.549, max: 0.549, cur: 0.549)
validation (min: 0.355, max: 0.355, cur: 0.355)

this won't be enough for many purposes on precision, why? first 10 epochs of my neural starts right off of 0.00001 epoch loss !!!! the top table is pretty much useless with that precision.

Also, a nice thing to have before the tables would be a line that shows the finishing time on last epoch (how much time the last epoch took in total) side by side with the last epoch's number (when you run 3000 epochs, reading what epoch you are in from the chart is a headache)! I would appreciate if you add these two features. for the precision, maybe you can accept an integer into the function that would let us define how many digits in decimal place we need?

Hi,

I'm just curious if you support jupyter lab. Today I went down the rabbit hole of having a dynamic plot with matplotlib, my findings were very disappointing and I ended up just clearing the output and redrawing with this: https://stackoverflow.com/a/24818304/311744.

Brief summary of what I found/tried:
I tried that: https://stackoverflow.com/a/39853938/311744, but %matplotlib notebook is not supported by jupyterlab
I tried that also: https://stackoverflow.com/a/34486703/311744, but is seems very hacky, and doesn't support retina displays... Also, it uses the same %matplotlib notebook as above so no jupyterlab support.

Hi

Here is my Fitting section of neural network

`opt = tf.keras.optimizers.Adam(lr=0.001, decay = 1e-6)
model.compile(loss='sparse_categorical_crossentropy', optimizer= opt, metrics= ['accuracy'] )

tensorboard = TensorBoard(log_dir=f"logs/{MODEL_NAME}-{int(time.time())}")
filePath = MODEL_NAME + "-{epoch:04d}-{val_loss : .6f}"

checkpoint = ModelCheckpoint("Models/{}.model".format(filePath, monitor='val_loss', verbose = 1, save_best_only = True, mode='max'))

plot_losses = PlotLossesCallback()

history = model.fit(Train_X, Train_Y,
batch_size=BATCH_SIZE,
epochs=EPOCHS,
validation_data=(Validation_X, Validation_Y),
callbacks=[tensorboard, checkpoint, plot_losses])

M5Model.save(f'./Models/{SEQ_LEN}-SEQ--{BATCH_SIZE}-BatchSize-{CHART_TO_PREDICT}-MODEL-{MODEL_TYPE}.h5')`

why am i getting this error?

I have epochs = 99 because I use early stopping. The problem is that the plot scales the x axis for the number of epochs I set, not the actual number of epochs.

I would like the plot automatically adjust to the number of current, real, epochs. Otherwise the plot looks like this and is very poorly readable - https://puu.sh/BoxPa/5a3c0ee3bb.png

Hi, I used your codes for plotting many epochs, and as there are a lot epochs, the xlabel only show 5, 10, 15, etc. Could you tell me how to exactly know which epoch my loss point belongs to? Maybe my_x_ticks = np.arange(0, 200, 1) my_y_ticks = np.arange(0, 1, 0.01) plt.xticks(my_x_ticks) plt.yticks(my_y_ticks)
something like this may help.
Thankyou for your time

Hello,

This is a great alternative tool to Tensorboard for watching deep learning models.

Having said that, I noticed a bug when using fit_generator(). PlotLossesKeras() only plots the last point for each loss/accuracy and you end up with a straight line which is not very usefull...

Could you test your tool with fit_generator and provide a solution for this method ?

For information, when I plot directly from model.history.history, I get a good result (same thing when using fit method):

Hi, thanks for the really nice package. I would like to suggest to add an option to skip plt.show() in draw_plots() so that the plots can later be modified. That is something like

• show=True: (default) run plt.show() as usual
• show=False: skip plt.show()

By skipping plt.show(), users can then, for example, change the styles, modify the labels or add annotations, and call plt.show() afterward. For instance,

liveloss.draw(show=False)
plt.xlabel('step')
plt.show()

Thanks!

For some reason Markdown does not render on https://pypi.org/project/livelossplot/.
I have no idea why - I did follow the guidelines in setup.py:

long_description=readme(),
long_description_content_type='text/markdown'

Vide:

• https://packaging.python.org/guides/making-a-pypi-friendly-readme/
• https://dustingram.com/articles/2018/03/16/markdown-descriptions-on-pypi
• pypa/packaging-problems#46
• pypi/warehouse#3664
• pypi/warehouse#3438

(I do have up-to-date versions of twine, setuptools and wheel )

EDIT: Most likely an error with twine: pypa/twine#420 (there is a fix, but not yet released).

Any help appreciated!

The majority of functions and classes miss dosctrings, i.e. helper texts.

Even if you new to livelossplot, open source or Python in general, I believe you can contribute! :)
And if you are a veteran, it will be a piece of cake.

You can do it one by one, noo need to do everything in one go. Every help is appreciated.

Please adhere to PEP 257 -- Docstring Conventions.

I'm not running from a Jupyter notebook, instead I'm running in Python directly (on OS X). How can I force inline?

It would be great if there is a trend line by smoothing the data points. This is especially useful for training GAN as the loss is usually noisy.

in setup.py

def readme():
with open('README.md', encoding='utf-8') as f:
return f.read()
...
long_description=readme()

but, in python 2.7, open function has no encoding parameter.
so, error occurs.
"TypeError: 'encoding' is an invalid keyword argument for this function"

For Keras version 2.1.3 the PlotLossesKeras crashes due to error in line 33 of keras_plot.py

Reason for this is:
self.metric2printable['loss'] = self.metric2printable.get(self.model.loss, self.model.loss) + " (cost function)"
where this breaks due to self.model.loss being a keras function. This results in

TypeError: unsupported operand type(s) for +: 'function' and 'str'

When using Tensorboard logging, it will try to create a directory with time_str = datetime.now().isoformat()[:-7].replace("T", " ").
However, the isoformat includes the ":" character which cannot be used in a file path.

The line should be updated to replace a blacklist of characters. This may vary from one OS to another, but on Windows 10, it's not possible to use Tensorboard logging without tweaking the time_str variable.

Maybe fix this warning for the future:

FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.

Hi!
I'm new to TF and keras and wanted to try out 'livelossplot'. I get the following Error:

AttributeError: module 'neptune' has no attribute 'Context'

This is my code:

`
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy', 'mean_squared_error'])

plot_losses = PlotLossesCallback()

model.fit(X_train, y_train,
epochs=2,
batch_size=1,
validation_data=(X_test, y_test),
callbacks=[plot_losses],
verbose=0)

AttributeError Traceback (most recent call last)
in
4
5
----> 6 plot_losses = PlotLossesCallback()
7
8 model.fit(X_train, y_train,

D:\Programme\Anaconda3\envs\tensorflow_env\lib\site-packages\livelossplot\keras.py in init(self, **kwargs)
7 def init(self, **kwargs):
8 keras.callbacks.Callback.init(self)
----> 9 _PlotLossesCallback.init(self, **kwargs)

D:\Programme\Anaconda3\envs\tensorflow_env\lib\site-packages\livelossplot\generic_keras.py in init(self, **kwargs)
26 class _PlotLossesCallback():
27 def init(self, **kwargs):
---> 28 self.liveplot = PlotLosses(**kwargs)
29
30 def on_train_begin(self, logs={}):

D:\Programme\Anaconda3\envs\tensorflow_env\lib\site-packages\livelossplot\generic_plot.py in init(self, figsize, cell_size, dynamic_x_axis, max_cols, max_epoch, metric2title, series_fmt, validation_fmt, plot_extrema, fig_path, target)
37 self.plot_extrema = plot_extrema
38 self.target = target
---> 39 from .neptune_integration import neptune_send_plot
40 self.fig_path = fig_path
41

D:\Programme\Anaconda3\envs\tensorflow_env\lib\site-packages\livelossplot\neptune_integration.py in
1 import neptune
2
----> 3 ctx = neptune.Context()
4
5
AttributeError: module 'neptune' has no attribute 'Context'`

Any idea?

Hi,

I'm using this lib (which seems cool btw) and I got this error message :
AttributeError: 'PlotLossesCallback' object has no attribute 'on_train_batch_begin'

I'm using TF 2.0 Beta 1 and Keras in python.

Thx !

Hi,

is it possible to use livelossplot when training manually via train_on_batch in keras?

thanks,

Jani

I have the CPU version of tensorflow and have been training a 3 layered Simple Recurrent keras model. However, the model takes takes too long to complete the an epoch (in excess of 30 minutes).

This started when I added the plot_losses = PlotLossesCallback() so that I can get live progress plots of the progress on the losses and accuracies for each epoch. I'm not sure how to remediate the issue.

Would installing cuDNN be helpful to fixing the issue or does it not work with the CPU version of tensorflow?
Or is there another when this can be fixed?

I have a model that has three outputs (named "usage", "clients", and "sessions") and one loss function (mean absolute percentage error.) As shown in the picture below, the names of my plots are a little messed up. I believe the upper right plot should be titled "usage_loss".

I think the problem is in this section of the code, but I'm not sure how to fix it.

This is a very small thing but this API does not seem to be compatible with the tf.Keras API

from livelossplot.keras import PlotLossesCallback
callbacks_list = [PlotLossesCallback()]

ModuleNotFoundError Traceback (most recent call last)
in
4 checkpoint = ModelCheckpoint("C:\home\jupyter\experiments\models",
5 monitor='acc', verbose=1, save_best_only=True, mode='max')
----> 6 callbacks_list = [checkpoint, PlotLossesCallback()]

~\AppData\Local\conda\conda\envs\tesseract\lib\site-packages\livelossplot\keras.py in init(self, **kwargs)
7 def init(self, **kwargs):
8 keras.callbacks.Callback.init(self)
----> 9 _PlotLossesCallback.init(self, **kwargs)

~\AppData\Local\conda\conda\envs\tesseract\lib\site-packages\livelossplot\generic_keras.py in init(self, **kwargs)
26 class _PlotLossesCallback():
27 def init(self, **kwargs):
---> 28 self.liveplot = PlotLosses(**kwargs)
29
30 def on_train_begin(self, logs={}):

~\AppData\Local\conda\conda\envs\tesseract\lib\site-packages\livelossplot\generic_plot.py in init(self, figsize, cell_size, dynamic_x_axis, max_cols, max_epoch, metric2title, series_fmt, validation_fmt, plot_extrema, fig_path, target)
37 self.plot_extrema = plot_extrema
38 self.target = target
---> 39 from .neptune_integration import neptune_send_plot
40 self.fig_path = fig_path
41

~\AppData\Local\conda\conda\envs\tesseract\lib\site-packages\livelossplot\neptune_integration.py in
----> 1 import neptune
2
3 ctx = neptune.Context()
4
5

ModuleNotFoundError: No module named 'neptune'

I noticed that any output besides the loss graph of livelossplot gets cleared at each training iteration. (anytime I call liveloss.draw())

Do you have a way of keeping that output ?
It seems that this mechanism is at the core of your package:

# livelossplot/core.py:15
def draw_plot(logs, metrics, figsize=None, max_epoch=None,
              max_cols=2,
              validation_fmt="val_{}",
              metric2title={}):
    clear_output(wait=True) # I see what you did here
    plt.figure(figsize=figsize)

So I suspect that finding a workaround will be pretty difficult. With that said, I still would like to know if anyone has suggestions.

how to set a dynamic x axis which was available in the previous version as dynamic_x_axis=True?

Hi,

I am trying to use this library but the following error of no "set_max_epoch" attribute is showing. Thanks.

In the example here:
https://github.com/stared/livelossplot/blob/master/examples/minimal.ipynb
the x axis loses data for epoch 1 and 2 and begins plotting at 3 when there are 3 epochs to plot.

I encountered this in my own notebook and then checked this reference example to demonstrate the same behavior

I'm running my PyTorch code in Colab. I'm wondering if it is possible to use livelossplot on Colab?

When loading livelossplot with python2.7 and keras2.2.4 when importing the module I get this return:

AttributeError                            Traceback (most recent call last)
<ipython-input-383-ab062d1d275c> in <module>()
----> 1 from livelossplot.keras import PlotLossesCallback
      2 import keras
      3 from keras.layers import Input, LSTM, Dense
      4 from keras.models import Model

/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/livelossplot/keras.pyc in <module>()
      2 from .generic_keras import _PlotLossesCallback
      3 
----> 4 class PlotLossesCallback(_PlotLossesCallback, keras.callbacks.Callback):
      5     def __init__(self, **kwargs):
      6         keras.callbacks.Callback.__init__(self)

AttributeError: 'module' object has no attribute 'callbacks'

Hi,

I'd like my script to automatically change the backend back to "inline" when it is run by Jupiter.

I tried this but it does not work (in practice) :

# If the script is not run by python but by jupyter and is using a different backend then "inline"
if matplotlib.get_backend() != 'Qt5Agg' and ( 'inline' not in matplotlib.get_backend() ) :
    print("=> BEFORE: matplotlib backend = <%s>" % matplotlib.get_backend() )
    matplotlib.use('module://ipykernel.pylab.backend_inline',warn=False, force=True) # <=> %matplotlib inline
    import matplotlib.pyplot as plt
    print("=> AFTER: matplotlib backend = <%s>" % matplotlib.get_backend() )

I get this (which theoretically is correct) :

=> BEFORE: matplotlib backend = <nbAgg>
=> AFTER: matplotlib backend = <module://ipykernel.pylab.backend_inline>

But in practice, the "inline" backend is not applied.

Can you help me ?

Hi, I'd like to use the plot in the case that I can output using nohup command, but as there is a problem of GUI, so it caused segmentation core broke, so I can't plot in the output file. adding matplotlib.use('TkAgg') in this case may solve my problem but I'm not sure in your codes where to modify it.
In any case, it works fine in Jupyter Notebook, thank you!

I would like to be able to override the label for the x-axis.
Also, sometimes I would like to explicitly give the x-value for one of the points.

For example, if you are doing minibatch training, you might give the x-axis as the total number of examples seen.

Also, you might want to have training points every minibatch, and validation points at the end of every epoch, and the xvalue is minibatch * nbatches

HI,

Please let me know how I can change the values of Y axis for log loss in x.xxx format and acc in x.xxx format

I am working in a multitask model where I am passing a dictionary of losses and metrics for each task in model_compile(). It is a mixture of classification and regression tasks so I use custom variants of MSE and binary crossentropy. There is no issue when I pass each of these custom losses in a model as a single value, but I get an error when I try to use your PlotLossesKeras as callback and a dictionary of losses.

I checked the code and I think I found what causes the issue in keras_plot.py:

 if isinstance(self.model.loss, list):
    losses = self.model.loss
elif isinstance(self.model.loss, dict):
    losses = self.model.loss.values()
else:
# by far the most common scenario
    losses = [self.model.loss]

loss_name = loss2name(losses[0])

The last line raises a type error when losses is of type dict_values since it does not support indexing.
In any case, the purpose of this line is to get a name for the general loss, which at least in my case (and I believe in most cases where a dictionary of losses will be passed) does not really make sense and a generic term like 'loss' would make more sense. (In my case this value is the equivalent of the sum of all the individual losses of each task).

My solution was just to change this line to:

loss_name = loss2name(losses[0]) if isinstance(losses, list) else 'loss'

PD: Your little package is great, I have been using for a good while and it adds a neat functionality to the networks I train in my jupyter notebooks!

Bokeh, as an alternative backend to matplotlib, would make it simple to:

• toggle between in-notebook plots and external plots (so that it would work seamlessly with both Jupyter Notebook Python scripts)
• give mouseover view of exact numerical values (no more eye squeezing!)
• zoom if needed

I don't want to have it instead of matplotlib (as this one makes it easier to save plots, so that they are visible in Jupyter Notebooks on GitHub), but as an additional backend.

My experience with Bokeh is limited, so this issue is open for YOUR contribution! :)

It would make it simpler to interact with other plots.
Now we use global methods.

Vide: http://pbpython.com/effective-matplotlib.html

It would be nice to have two subplots (or even more, if one wants to observe different metrics):

One should display the loss (as is currently possible), and the other one next to it should display a loss (selectable by user).

I hope other people would like that as well.

Hi All,

I used the livelossplot for model training. Even though, all works fine including the model training but in the plot the training and validation curves are not shown. However, when I refresh the jupyter notebook and run it from beginning the full graphs are displayed. But when I want to do model tuning the curves do not display. May be .ipynb file is large. I do not know.

Would anyone be able to help me on this regard.
The modules that were used for the visualisation are below:

from IPython.display import SVG
import livelossplot
plot_losses = livelossplot.PlotLossesKeras()

Also please find the image of the training graph without the training and validation curves below:

Your input in this issue is much appreciated.

Thanks

Michael