Cannot use scipy.optimize.minimize method='CG' to run gradient descent about nn HOT 3 CLOSED

Further observation: method='CG' works for all cases i in range(9) except i=8, giving error: shapes (5000,401) and (1,1,401) not aligned: 401 (dim 1) != 1 (dim 1)

from nn.

Seems to be a bug in scipy.

from nn.

Observation: in certain circumstances, scipy.optimize.minimize(... method='CG'...) will add dimension to x0 (here x0=theta0) and thus making matrix multiplication in cost function error out.

Temporary workaround: in my cost function, ravel theta0 first, make sure it is 1D; then add proper dimension to it to make it a 2D array (column vector).

More comments:

1. I dived deep into scipy's source codes hoping to find evidence that it adds dimension to x0 by mistake but couldn't. The source codes are still too hard for me to read. In the future, if possible, I'd like to find evidence, submit test report and maybe even try to fix it and submit pull request.....
2. CG refers to conjugate gradient method, for more info, take a look at wikipedia page: https://en.wikipedia.org/wiki/Conjugate_gradient_method

from nn.