sparkier / luna Goto Github PK

Luna is inspired by Lucid, a framework for Feature Visualization. However, Luna is built on Tensorflow 2, and thus supports modern models and deep learning techniques.

Home Page: http://a13x.io/luna/

License: MIT License

Python 99.25% Shell 0.75%

artificial-intelligence deep-learning machine-learning tensorflow visualization

luna's Introduction

Hi there 👋

I am a Researcher at the intersection of ML and visualization. When I am not in front of my computer, you will probably find me in the mountains. My research is at the intersection of AI and HCI. In this context, I am always searching for the best method to connect humans with AI systems. I try to help developers with techniques and visualizations that foster communication and provide insights during development and work on visualizations that help them understand the decisions their AI systems make.

luna's People

Contributors

Stargazers

Watchers

Forkers

fuzhanrahmanian eddiem3 progamergov researchdaniel

luna's Issues

Type for opt_param and aug_param for visualize_filter function

I'm trying to run the example in the readme...

loss, image = featurevis.visualize_filter(image, model, "mixed6", 10, 200, 1)

but I'm getting this error...

Traceback (most recent call last): File "/home/emassey3/development/luna/test.py", line 9, in <module> loss, image = featurevis.visualize_filter(image, model, "mixed6", 10, 200, 1) File "/home/emassey3/development/luna/luna/featurevis/featurevis.py", line 64, in visualize_filter for iteration in range(opt_param.iterations): AttributeError: 'int' object has no attribute 'iterations'

What was the intended data type for opt_params in featurevis.py? I can't seem to find it anywhere else in the repo.

def visualize_filter(image, model, layer, filter_index, opt_param, aug_param):

Different Objectives

Is your feature request related to a problem? Please describe.
We currently have the filter and layer objective:

luna/luna/featurevis/featurevis.py

Lines 26 to 53 in bafa653

 def visualize( 

 image, 

 model, 

 layer, 

 optimization_parameters, 

 filter_index=None, 

 transformation=None, 

 regularization=None, 

 threshold=None, 

 minimize=False 

 ): 

 """Create a feature visualization for a filter in a layer or a whole layer of the model. 

  Args: 

  image (array): the image to be modified by the feature vis process. 

  model (object): the model to be used for the feature visualization. 

  layer (string): the name of the layer to be used in the visualization. 

  optimization_parameters (OptimizationParameters): the optimizer class to be applied. 

  filter_index (number): the index of the filter to be visualized. Whole layer if None. 

  transformations (function): a function defining the transformations to be perfromed. 

  regularization (function): customized regularizers to be applied. Defaults to None. 

  threshold (list): Intermediate steps for visualization. Defaults to None. 

  minimize (bool): whether or not to apply minimize as opposed to calling apply_gradient() 

  for adam optimizer. 

  Returns: 

  tuple: activation and result image for the process. 

  """

We should also implement a neuron objective.

Which kinds of models are/will be supported?

Hi, is Luna only meant for vision models, or is/will it be designed for other kinds of ML models as well?

Color and Sharpness Difference

Is your feature request related to a problem? Please describe.
The colors and sharpness of the Feature Visualization results are slightly worse then with Lucid. This needs to be investigated.

Describe the solution you'd like
It would be great to get the exact same results as Lucid.

Update Documentation Usage Guide

Describe the bug
The usage guide does not reflect the current setup anymore.

Pages Documentation using Sphinx

Is your feature request related to a problem? Please describe.
To provide a clean documentation, we should add automated doc generation.

Describe the solution you'd like
Use Sphinx to automatically generate documentation for us.

Referring to static weight files

Describe the bug
We have two points in the code:

luna/luna/pretrained_models/models.py

Line 76 in c71e518

return alexnet.alex_net("alexnet_weights.h5")

and

luna/luna/pretrained_models/googlenet.py

Lines 433 to 434 in c71e518

 googlenet.load_weights(os.path.join( 

 os.path.dirname(__file__), 'googlenet_weights.h5'))

which refer to static weight files. This will break if the repo is cloned without downloading these files.

To Reproduce
Clone repo, run alexnet or googlenet.

Expected behavior
Should work without having to download these weight files and put them into the luna package.

setup.py

Is your feature request related to a problem? Please describe.
Include a setup.py file to build locally.

Regularisation and Optimisation to speed up Feature Visualization

Is your feature request related to a problem? Please describe.
We currently need a high number of iterations to generate feature visualizations. More regularisation and optimisation of the process could yield faster results.

pages-publish.sh script

Describe the bug
The script does not run.

./pages-publish.sh
./pages-publish.sh: line 2: $'\r': command not found
./pages-publish.sh: line 3: syntax error in conditional expression
'/pages-publish.sh: line 3: syntax error near ']]
'/pages-publish.sh: line 3: 'if [[ $STATUS == *"nothing to commit, working tree clean"* ]]'

Also, the command sed -i "" '/build/d' ./.gitignore does not work by itself

sed: can't read /build/d: No such file or directory

I think that the double quotes ("") are unnecessary.
Also, shouldn't an .sh script start with " #!/bin/bash " ?

To Reproduce
./pages-publish.sh

Add a link to the project from the docs

Is your feature request related to a problem? Please describe.
When looking at the docs, there Is no direct link to the project. We should change that.

Compatibility with Adam

Is your feature request related to a problem? Please describe.
Currently, standard gradient ascent is used as an optimizer. Adding compatibility for Adam could lead to better results.

Describe the solution you'd like
Add other optimisers such as Adam for generating feature visualizations.

	def visualize(
	image,
	model,
	layer,
	optimization_parameters,
	filter_index=None,
	transformation=None,
	regularization=None,
	threshold=None,
	minimize=False
	):
	"""Create a feature visualization for a filter in a layer or a whole layer of the model.

	Args:
	image (array): the image to be modified by the feature vis process.
	model (object): the model to be used for the feature visualization.
	layer (string): the name of the layer to be used in the visualization.
	optimization_parameters (OptimizationParameters): the optimizer class to be applied.
	filter_index (number): the index of the filter to be visualized. Whole layer if None.
	transformations (function): a function defining the transformations to be perfromed.
	regularization (function): customized regularizers to be applied. Defaults to None.
	threshold (list): Intermediate steps for visualization. Defaults to None.
	minimize (bool): whether or not to apply minimize as opposed to calling apply_gradient()
	for adam optimizer.

	Returns:
	tuple: activation and result image for the process.
	"""

	googlenet.load_weights(os.path.join(
	os.path.dirname(__file__), 'googlenet_weights.h5'))