libbum / dice.jl Goto Github PK

Since DICE-CJL is effectively a 2007 offshoot, we may as well implement '07 as well. Won't cost that much.

Unsure why that's the case atm.

JuMP falls over itself in non-linear optimisation cases for the most part. We'd be better off refactoring to follow GalacticOptim's path once a more stable version is released.

Closing #21 and #32 have made necessary a few fudge factors that I can't seem to pin down why they are necessary. Either Ipopt isn't cut out for solving these problems as well as CONOPT, or we have a problem in how JuMP handles constraints.

This issue has been occurring for more than a year and I've finally managed to work around it, but in general I'm not happy that it's come to this point.

Please have a read of this discussion on the discourse to get a feel of the steps already taken.

The solution was to just make things function when they didn't by swapping out a fix for a quite constrained @constraint, and vice versa. v2016R2 as an example looks like this:

if typeof(scenario) <: OptimalPriceScenario
    @constraint(model, vars.Tₐₜ[1] == config.tatm₀);
    JuMP.fix(vars.K[1], config.k₀; force=true);
else
    JuMP.fix(vars.Tₐₜ[1], config.tatm₀; force=true);
    @NLconstraint(model, vars.K[1] == config.k₀);
end

If either of these values are changed, the solver gets into an infinite loop where it has reached some area of convergence, but is not happy with the result somehow. This has had to be done for all of the current implemented versions to varying degrees, generally always working with some interplay of K and Tₐₜ has seemed to be the best, although this is totally just via gruelling trial and error work.

I've given up chasing this, but if you're interested in helping out and know a little more about the inner workings of JuMP, please let me know.

I can’t tell if this is an existing issue or one I have introduced in my recent changes. I’m sure that installing the appropriate libraries would fix the issue, however it should not be crashing.

Works:

dice = solve(BasePrice, version, linear_solver = mumps);


Fails:

base = solve(BasePrice, version)

Exception of type: DYNAMIC_LIBRARY_FAILURE in file "Common/IpLibraryLoader.cpp" at line 67:
 Exception message: libhsl.so: cannot open shared object file: No such file or directory

EXIT: Library loading failure.
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "Common/IpLibraryLoader.cpp" at line 67:
 Exception message: libhsl.so: cannot open shared object file: No such file or directory

EXIT: Library loading failure.

I wasn't aware until adsun updated the AUR entry, but Ipopt 3.13 has been out for a month or so.

For the moment, the PKGBUILD is a bit messed up, but once we get the install under control, we should test to see how the brittle issues perform with the new overhauled release.

I don't think we should be adding tests for HSL since you require a license etc, but since you can get an academic one for free, it may be a good idea to write up some details on how to use some of the HSL solvers.

For the moment we get some slightly different results. But we need to finish #35 before we close this out.

DICE2016R.ipynb currently shows an offset in savings.

DICE2013RVanilla.ipynb is tracking a problem in the BasePrice Scenario.

The parameter show values are too massive to be useful. Reconsider them. They have been dropped from tests in the meantime.

I've tried multiple ways of getting this done, but every time I attempt to get this to happen, Ipopt will build from source every time.

The DICE website lists the current version from August 2017 (DICE-2016R2-083017.gms), although the source points to DICE2016R-091916ap.gms: A beta version.

If anyone has access to the most recent file, please send it my way.

Hand in hand with #21, we need to understand why there are so much issues concerning Ipopt 3.12.10 and 3.12.13.

This thread doesn't seem to be helping for our particular set of scenarios at the moment, but we will keep working on that issue.

• Remove / alter discussion in readme about ipopt versions once solution is verified.

The rr variable is updated in some scenarios through changes in ρ. I don't know if having ρ as a parameter, like it is now is enough to alter the output of the rr variable, or if it should also get upgraded to a parameter.

At the moment, most of the docs are pretty sparse. Now that most of the internals have settled down, it's probably time for a decent run and documentation.

Since Travis uses the pre-packaged Ipopt install, we get infeasibility issues as we have small version number differences. Hopefully this will smooth itself out once Ipopt stabilises whatever is happening inside their system for this to happen.

At the moment, this issue will be here to track differences. Supersedes #8.

Current Ipopt version: 3.12.13
Ipopt.jl / travis version: 3.12.10

Infeasibilities:
BasePrice, SternCalibrated: v2013R(RockyRoad)
Incorrect Utility:
OptimalPrice: v2013R(RockyRoad)
Exceeds iteration limit:
Limit2Degrees: v2013R(RockyRoad)

DICE2016R.ipynb discusses a discrepancy in forcoth. Now that we have GAMS results to compare, we can test this correctly.

Load time is going to blow up if we keep include-ing everything. We should figure out how to load modules on demand (if this is possible).

Currently there are no optimisation tests (and therefore incomplete coverage) on the CI builds since the results always come out infeasible. No idea why since Ipopt seems to build fine. Local testing on multiple machines work, so for now they are just not requested.

I asked over on the Julia discourse but haven't had much luck there as to where to begin troubleshooting.

My local builds are Arch linux using the AUR coin-or-ipopt package (current v3.12.10, but have tested 3.12.9 & 3.12.8). Travis builds from source each time using the Ipopt.jl build script. That pulls in Ipopt version 3.12.8, so it's up to date for the most part....

v0.7 landed at a bit of an awkward time for JuMP, as JuMP itself is under the process of a major overhaul and moving to MOI. A proper v0.19 release is still a few months off, but for the moment we can get things somewhat ready (hopefully) with the release-0.18 branch.

The Vanilla and RR options are basically the same, but RR has 6 or so more options. We could probably leverage ConcreteAbstractions to extend the model here rather than having separate, mostly duplicated objects.

I don't have a GAMS license, so at some stage I need to get someone to run the different versions to get me comparable outputs to validate against. For the moment I can verify OptimalPrice scenarios of 2013R Vanilla and 2016R beta.

• 2013R Vanilla
  • BasePrice
  • OptimalPrice
• 2013R Rocky Road
  • BasePrice
  • OptimalPrice
  • Limit2Degrees
  • Stern
  • SternCalibrated
  • Copenhagen
• 2016R beta
  • BasePrice
  • OptimalPrice

The current implementation cannot solve the SternCalibrated scenario under v2013R(RockyRoad).

Hello,
I'm trying to run this package. I've tried on an ARM as well as x86 processor, always get this same error during solving:

julia> using DICE;

julia> version = v2016R()
v2016R beta

julia> base = solve(BasePrice, version)

signal (6): Aborted
in expression starting at REPL[14]:1
gsignal at /usr/bin/../lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /usr/bin/../lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7fb3e4f63675)
 at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder18GetSymLinearSolverERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder22AugSystemSolverFactoryERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder18GetAugSystemSolverERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder21PDSystemSolverFactoryERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder17GetPDSystemSolverERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder30BuildSearchDirectionCalculatorERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16AlgorithmBuilder19BuildBasicAlgorithmERKNS_10JournalistERKNS_11OptionsListERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16IpoptApplication11OptimizeNLPERKNS_8SmartPtrINS_3NLPEEERNS1_INS_16AlgorithmBuilderEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16IpoptApplication11OptimizeNLPERKNS_8SmartPtrINS_3NLPEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
_ZN5Ipopt16IpoptApplication12OptimizeTNLPERKNS_8SmartPtrINS_4TNLPEEE at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
IpoptSolve at /home/jovyan/.julia/packages/Ipopt/ruIXY/deps/usr/lib/libipopt.so (unknown line)
solveProblem at /home/jovyan/.julia/packages/Ipopt/ruIXY/src/Ipopt.jl:342
runLinearSolverCheck at /home/jovyan/.julia/packages/DICE/oPs6G/src/DICE.jl:86
selectLinearSolver at /home/jovyan/.julia/packages/DICE/oPs6G/src/DICE.jl:60
solve at /home/jovyan/.julia/packages/DICE/oPs6G/src/2016R.jl:313
unknown function (ip: 0x7fb3ce4e588c)
unknown function (ip: 0x7fb3e922a75b)
unknown function (ip: 0x7fb3e922a389)
unknown function (ip: 0x7fb3e922a8f0)
unknown function (ip: 0x7fb3e922b9c8)
unknown function (ip: 0x7fb3e922c616)
unknown function (ip: 0x7fb3e9244e08)
unknown function (ip: 0x7fb3e92453c8)
jl_toplevel_eval_in at /usr/bin/../lib/x86_64-linux-gnu/libjulia.so.1 (unknown line)
eval at ./boot.jl:331
eval_user_input at /build/julia-98cBbp/julia-1.4.1+dfsg/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:86
macro expansion at /build/julia-98cBbp/julia-1.4.1+dfsg/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:118 [inlined]
#26 at ./task.jl:358
unknown function (ip: 0x7fb3e922fcbb)
unknown function (ip: (nil))
Allocations: 104312612 (Pool: 104300035; Big: 12577); GC: 131
Aborted

Here's a full log of the commands leading up to it: https://gist.github.com/jamesmurdza/dd1439a0fbe0f9d062c529ceee7371a5

Would be much appreciated if you could help me in debugging this issue!

There are 3 different time steps in the CJL files as far as I can see. There may be a need to add some Scenarios for these if they cannot be easily switched with option toggles.