Giter Club home page Giter Club logo

cocp's Introduction

Learning convex optimization control policies

This repository accompanies the paper Learning convex optimization control policies. It contains the source code for the examples therein as IPython notebooks.

Our examples make use the Python package cvxpylayers to differentiate through convex optimization problems.

Abstract

Many control policies used in various applications determine the input or action by solving a convex optimization problem that depends on the current state and some parameters. These types of control policies are tuned by varying the parameters in the optimization problem, such as the linear quadratic regulator weights, to obtain good performance, judged by application-specific metrics. Our paper introduces a method to automate this process, by adjusting the parameters using an approximate gradient of the performance metric with respect to the parameters. Our procedure relies on recently developed methods that can efficiently evaluate the derivative of the solution of a convex optimization problem with respect to its parameters.

Citing

@article{agrawal2019cocp,
    author       = {Agrawal, Akshay and Barratt, Shane and Boyd, Stephen and Stellato, Bartolomeo},
    title        = {Learning Convex Optimization Control Policies},
    journal      = {arXiv},
    archivePrefix = {arXiv},
    eprint = {1912.09529},
    primaryClass = {math.OC},
    year    = {2019},
}

cocp's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cocp's Issues

potential error in code

hi, thanks for the work on this. I am looking at markowitz_tuning.ipynb for the simulate() function.

def simulate(ht, ut):
    ret = torch.exp(logreturn1p_dist.sample())
    ht = ret * (ht + ut)                  <----- this line
    return ht, ret

from my understanding, ht should be (ret * (ht + ut))/(ret.T @ (ht + ut)) as per formula stated for how holdings evolve in 5.3 of the paper? This formula sums to 1 (as holdings should be), while the line in code doesn't.

Is Continuous ARE supported?

Hi,

I would appreciate getting your help with the LQR example.
Following the LQR example, I notice that for a continuous problem, e.g., cart pole, with linearized dynamics at the fixed point, the LQR solution doesn't work. It returns "inf."

For example,

Consider the following dynamics matrices:
A = np.array([ [0, 0, 1, 0],
[0, -0.71707317, 0, 0],
[0, 0, 0, 1],
[0, 15.77560976, 0, 0]])

B = np.array([ [0],
[0],
[0.97560976],
[-1.46341463]])

The following code doesn't work:
So that you know, I changed the discrete ARE from your notebook to the continuous ARE.

###################################################
P = cp.Variable((n, n), PSD=True)
R0cvxpy = cp.Parameter((m, m), PSD=True)

objective = cp.trace(P@W)
constraints = [cp.bmat([
[R0cvxpy, B.T@P],
[P@B, Q0 + A.T@P + P@A]
]) >> 0, P >> 0]
R0cvxpy.value = R0
result = cp.Problem(cp.Maximize(objective), constraints).solve()
P_lqr = P.value
print(result)
##################################################

Returns: Inf

UserWarning: Solution may be inaccurate. Try another solver, adjusting the solver settings, or solve with verbose=True for more information.

I appreciate any help you can provide.

Best regards,
Solomon

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.