hips / kayak Goto Github PK

I had a set of targets of size (77,) and a set of predictions of size (77,1). When I applied an L2 loss to this, Kayak implicitly broadcast so that it applied an L2 loss to all 77^2 combinations of predictions and targets.

More generally, I think that there is an opportunity to make Kayak operations enforce explicit shapes in its arguments. This makes it slightly more inconvenient in that broadcasting has to be done explicitly, but it also avoids a lot of the silly pitfalls that come from implicit broadcasting.

Something strange is happening when the graph is dynamically altered that seems to prevent nodes (or graphs) from being garbage collected.

Adding a del method to differentiable that removes links to a node from its parents might help solve this.

Based on some advice from an avid Kayak user, I found that this was one way to do it:

invN = kayak.Constant(np.atleast_2d(1.0/y.shape[0]))
self.loss = kayak.MatMult(y,invN)

Something like this really should be a basic op.

A small issue with data.py is that it crashes out with no useful error if the user hasn't created an empty subdirectory called 'data'

We should write some documentation in the readme file indicating how to:

• Install the package
• Run the examples
• Possibly explain somewhat how to create new modules

Currently the error
TypeError: a class that defines slots without defining getstate cannot be pickled
appears when I try to pickle Kayak objects. How would we save the trained networks?

JasperSnoek should do this.

Currently there are different variable types: Input, Target, Constant, Parameter. I understand why each of these exist, but the difference between them from a computational standpoint is not obvious. The one exception perhaps is that inputs and targets can be batched?

I'm thinking that there should just be a single Variable type that has access to every basic op, including differentiation and batching. That way it would be very straightforward for someone who is new to the framework to get started.

Either that, or there should be some very explicit documentation about how all of these types differ.