Correction for Adjoint Gradient Calculations about sunode HOT 4 CLOSED

Thanks for opening such a detailed issue!
I edited the formatting in your comment and added a link to that line.

I'm not familiar with the implementation, but is this a bug due to t_intervals and grads having different lengths and the for iterator never reaches the None element in the reversed(grads)?
Because inside that loop there's this if grad is not None: which appears to have been written for this t=0 element..

from sunode.

Great! Thank you for editing my post.

There is also another issue. At sufficiently small time evaluations, the gradient computations via the adjoint equations are inaccurate when compared to those of forward sensitivities.

Following the example above, if I evaluate the adjoint equation at time = 0, 5, 10 and grads = np.ones_like(yout) then I get that

• grad_out_fwd = [29.633367875233063, -8.63361922455043, 10.2485995824757]
• grad_out_adj = [ 8.08665182 -1.22041063 8.53419337].

However as in my original post, if I evaluate the adjoint equations at np.linspace(0, 10, 21), which includes time = 0, 5, 10, and zeros-pad grads at the time not equal to 0, 5, 10, then I get

• grad_out_fwd = [29.633367875233063, -8.63361922455043, 10.2485995824757]
• grad_out_adj = [27.71999675 -7.42137746 10.507334 ].

Thank you for looking to these issues!

from sunode.

@aarcher07 Thank you for reporting this, and sorry for the very late reply...

I think the problem you are seeing is due to a small mistake in the arguments to solve_backward. If I replace it by this, I get the same results as the forward solver:

# Instead of this
#solver.solve_backward(t0=tvals_expanded[-1], tend=tvals_expanded[0], tvals=tvals_expanded[1:-1],
#                      grads=grads, grad_out=grad_out, lamda_out=lambda_out)

# It should be this
solver.solve_backward(
    t0=tvals_expanded[-1],
    tend=tvals_expanded[0],
    tvals=tvals_expanded,
    grads=grads,
    grad_out=grad_out,
    lamda_out=lambda_out
)

grad_out_adj = -sens0 @ lambda_out + grad_out
print(grad_out_adj)

# Output

# from forward
# [29.633367875233063, -8.63361922455043, 10.2485995824757]

# from adjoint
# [29.63336772 -8.63361915 10.24859955]

The problem is that by passing in tvals=tvals_expanded[1:-1] we actually don't use the first two entries of grads, and the time points for those gradients don't match the correct tvals anymore.

from sunode.

I'm closing this because I think it was a problem in the example code, but feel free to reopen or comment if you don't agree or have questions.

from sunode.