Description of the problem

Example of the problem

References/Other comments

Description of the problem

With multiple variables, when the gradient is overwritten for the next variable the previous gradient pointer should be cleared from heap memory. Currently, it leads to a memory leak.

Example of the problem

References/Other comments

Description of the problem

This issue aims to discuss and result in a stable API for Tensors. Taking references from already exisiting Tensor in tensorflow and torch is required.

Example of the problem

References/Other comments

Description of the problem

• using layer->len_ops number of threads may cause an issue for a neural network where the input dimension is high.
• Threads need not be passed to _rr_scheduler in the present implementation.
  

  BNN/bnn/autodiff/forward_impl.cpp

  Lines 50 to 56 in 3d752fe

  	while(layer != NULL)
  	{
  	unsigned threads = layer->len_ops, i;
  	thread* pool[threads];
  	op_queue<data_type>* jobs[threads][2];
  	_rr_scheduler<data_type>(layer, jobs, threads);

Example of the problem

References/Other comments

Description of the problem

This issue aims at specifying the workflow and discussing the design of the framework for BNN.

Workflow

1. Modules - The modules that should be a part of the framework(in this issue).
2. APIs for each module finalised above(in separate issues).
3. Class/Function design for each of the above APIs.
4. Implementation in a PR.
5. Testing.

If you have anything to suggest about the modules and their organization then please comment below.

Example of the problem

References/Other comments

Description of the problem

APIs for models(train, test) need to be discussed in the issue. A simple use case for starting,
Image classification on MNIST dataset with 784-200-200-10.

Example of the problem

References/Other comments

Description of the problem

1. The following line is not needed,

   BNN/.travis.yml

   Line 13 in d21065c

   - ls -a

2. The following should use 1.10.0 release,

   BNN/CMakeLists.txt

   Lines 23 to 31 in d21065c

   	if(INSTALL_GOOGLETEST AND BUILD_TESTS)
   	include(${CMAKE_ROOT}/Modules/ExternalProject.cmake)
   	ExternalProject_Add(googletest
   	GIT_REPOSITORY https://github.com/google/googletest
   	GIT_TAG master
   	SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}/googletest-src"
   	BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}/googletest-build"
   	TEST_COMMAND "")
   	endif()

Example of the problem

References/Other comments

Description of the problem

This issue aims at discussing design of automatic differentiation(AD).
Basically, AD has two modes, Forward Mode and Reverse Mode. We plan to implement both of them as they have their own pros and cons.
In the context of this project, the function to be differentiated will be loss function whose variables will be the weights of the network. All of the quantities i.e., weights, inputs and output will be tensors.

Forward Mode
This mode computes the derivative alongside computing the value of the loss function. For each variable under consideration a derivative component is associated with it. Then chain rule is applied in each step of the computation.

Reverse Mode
This is similar to back propagation. Here the computation graph is formed in the forward pass and the derivative of the loss function w.r.t all the variables is calculated in reverse pass.

Example of the problem

References/Other comments