odow / mathoptformat.jl Goto Github PK

View Code? Open in Web Editor NEW

33.0 12.0 9.0 333 KB

Read and write a variety of mathematical optimization file formats

License: Other

Julia 100.00%

mathoptformat.jl's People

Contributors

Stargazers

Watchers

Forkers

rschwarz chriscoey joaquimg sourabhdalvi batmanabcdefg jiazhengzhu fa-bien blegat juliatagbot

mathoptformat.jl's Issues

Issues when installing MathOptFormat

Hi,
I am trying to get the package to work in Julia 1.1.1 but I am having some type of odd interaction with the package HTTP. I tried downgrading packages, upgrading packages, with no luck. Please refer to the log below.

Any help is welcome, I need to debug a model built with JuMP and I need to print the LP.
Cheer
Sebastian

(v1.1) pkg> add https://github.com/odow/MathOptFormat.jl
Updating registry at ~/.julia/registries/General
Updating git-repo https://github.com/JuliaRegistries/General.git
┌ Warning: Cannot perform fast-forward merge
└ @ LibGit2 /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.1/LibGit2/src/merge.jl:237
┌ Warning: Some registries failed to update:
│ — /Users/spuschel/.julia/registries/General — registry failed to rebase on origin/master
└ @ Pkg.Types /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Types.jl:1269
Updating git-repo https://github.com/odow/MathOptFormat.jl
Resolving package versions...
ERROR: Unsatisfiable requirements detected for package HTTP [cd3eb016]:
HTTP [cd3eb016] log:
├─possible versions are: [0.0.1-0.0.2, 0.4.0-0.4.3, 0.5.0, 0.5.2-0.5.5, 0.5.7, 0.6.0-0.6.14, 0.7.0-0.7.1, 0.8.0] or uninstalled
└─restricted to versions 0.8.2-* by MathOptFormat [f4570300] — no versions left
└─MathOptFormat [f4570300] log:
├─possible versions are: 0.1.1 or uninstalled
└─MathOptFormat [f4570300] is fixed to version 0.1.1

Tests failing

Tests on Travis currently failing because jump-dev/MathOptInterface.jl#140 was merged.

Trouble writing a .cbf with constraints for a mixed-integer PSD program

I am using Julia 1.2 and the most recent MOI. I have the following reasonably minimal example

using MathOptFormat,SCS,JuMP


(nbuses, nlines, ngens) = (9, 9, 3)
(N, L, G) = (1:9, 1:9, 1:3)
(fromLines, toLines, fromBus, toBus) = (Array{Int64,1}[[1], [], [4], [2], [3], [5], [6], [7, 8], [9]], Array{Int64,1}[[], [7], [], [1, 9], [2], [3, 4], [5], [6], [8]], [1, 4, 5, 3, 6, 7, 8, 8, 9], [4, 5, 6, 6, 7, 8, 2, 9, 4])
(BusGeners, Y) = (Array{Int64,1}[[1], [2], [3], [], [], [], [], [], []], Dict{Any,Any}("ffR" => [0.0, 1.9421912487147266, 1.2820091384241148, 0.0, 1.155087480890097, 1.6171224732461358, 0.0, 1.1876043792911484, 1.36518771331058],"ffI" => [-17.36111111111111, -10.43168205186793, -5.409244962361526, -17.064846416382252, -9.679770426363174, -13.623478596908443, -16.0, -5.822134533308591, -11.516095563139931],"ttR" => [0.0, 1.9421912487147266, 1.2820091384241148, 0.0, 1.155087480890097, 1.6171224732461358, 0.0, 1.1876043792911484, 1.36518771331058],"ttI" => [-17.36111111111111, -10.43168205186793, -5.409244962361526, -17.064846416382252, -9.679770426363174, -13.623478596908443, -16.0, -5.822134533308591, -11.516095563139931],"ftR" => [-0.0, -1.9421912487147266, -1.2820091384241148, -0.0, -1.155087480890097, -1.6171224732461358, -0.0, -1.1876043792911484, -1.36518771331058],"tfI" => [17.36111111111111, 10.510682051867931, 5.588244962361526, 17.064846416382252, 9.784270426363173, 13.697978596908444, 16.0, 5.975134533308591, 11.60409556313993],"shR" => [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],"shI" => [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],"ftI" => [17.36111111111111, 10.510682051867931, 5.588244962361526, 17.064846416382252, 9.784270426363173, 13.697978596908444, 16.0, 5.975134533308591, 11.60409556313993],"tfR" => [0.0, -1.9421912487147266, -1.2820091384241148, 0.0, -1.155087480890097, -1.6171224732461358, 0.0, -1.1876043792911484, -1.36518771331058]))
(Pmin, Pmax, Qmin, Qmax) = ([0.1, 0.1, 0.1], [2.5, 3.0, 2.7], [-3.0, -3.0, -3.0], [3.0, 3.0, 3.0])
(Wmin, Wmax) = ([0.81, 0.81, 0.81, 0.81, 0.81, 0.81, 0.81, 0.81, 0.81], [1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21])
(PD, QD) = ([0.0, 0.0, 0.0, 0.0, 0.9, 0.0, 1.0, 0.0, 1.25], [0.0, 0.0, 0.0, 0.0, 0.3, 0.0, 0.35, 0.0, 0.5])

    mMP = Model(with_optimizer(SCS.Optimizer,verbose=1,max_iters=10000))
    @variable(mMP, -1 <= α[i=N] <= 1)
    @variable(mMP, -1 <= β[i=N] <= 1)
    @variable(mMP, γp[i=N] >= 0)
    @variable(mMP, γm[i=N] >= 0)
    @constraint(mMP, [i=N], γp[i]+γm[i] <= 1)
    @variable(mMP, ζpUB[g=G] >= 0)
    @variable(mMP, ζpLB[g=G] >= 0)
    @variable(mMP, ζqUB[g=G] >= 0)
    @variable(mMP, ζqLB[g=G] >= 0)
    @variable(mMP, x[l=L], Bin)
    @constraint(mMP, sum(x[l] for l in L) <= 3)

    @variable(mMP, λF[l=L]); @variable(mMP, λT[l=L]); @variable(mMP, μF[l=L]); @variable(mMP, μT[l=L])

  for i in N
   for g in BusGeners[i]
    @constraint(mMP,
    -α[i] + ζpUB[g] - ζpLB[g] == 0 )
    @constraint(mMP,
    -β[i] + ζqUB[g] - ζqLB[g] == 0 )
   end
  end

    @constraint(mMP, AMcf1[l in L], α[fromBus[l]] - x[l] <= λF[l]); @constraint(mMP, AMcf2[l in L], α[fromBus[l]] + x[l] >= λF[l])
    @constraint(mMP, AMcf3[l in L], -(1 - x[l]) <= λF[l]); @constraint(mMP, AMcf4[l in L], (1 - x[l]) >= λF[l])
    @constraint(mMP, AMct1[l in L], α[toBus[l]] - x[l] <= λT[l]); @constraint(mMP, AMct2[l in L], α[toBus[l]] + x[l] >= λT[l])
    @constraint(mMP, AMct3[l in L], -(1 - x[l]) <= λT[l]); @constraint(mMP, AMct4[l in L], (1 - x[l]) >= λT[l])

    @constraint(mMP, BMcf1[l in L], β[fromBus[l]] - x[l] <= μF[l]); @constraint(mMP, BMcf2[l in L], β[fromBus[l]] + x[l] >= μF[l])
    @constraint(mMP, BMcf3[l in L], -(1 - x[l]) <= μF[l]); @constraint(mMP, BMcf4[l in L], (1 - x[l]) >= μF[l])
    @constraint(mMP, BMct1[l in L], β[toBus[l]] - x[l] <= μT[l]); @constraint(mMP, BMct2[l in L], β[toBus[l]] + x[l] >= μT[l])
    @constraint(mMP, BMct3[l in L], -(1 - x[l]) <= μT[l]); @constraint(mMP, BMct4[l in L], (1 - x[l]) >= μT[l])

   @expression(mMP, C[i=1:(2*nbuses),j=i:(2*nbuses)], 0)
  for i in N
    C[i,i] += γp[i] - γm[i] + α[i]*Y["shR"][i] - β[i]*Y["shI"][i]
    C[nbuses+i,nbuses+i] += γp[i] - γm[i] + α[i]*Y["shR"][i] - β[i]*Y["shI"][i]
    for l in fromLines[i]
      C[i,i] += λF[l]*Y["ffR"][l] - μF[l]*Y["ffI"][l]
      C[nbuses+i,nbuses+i] += λF[l]*Y["ffR"][l] - μF[l]*Y["ffI"][l]
    end
    for l in toLines[i]
      C[i,i] += λT[l]*Y["ttR"][l] - μT[l]*Y["ttI"][l]
      C[nbuses+i,nbuses+i] += λT[l]*Y["ttR"][l] - μT[l]*Y["ttI"][l]
    end
  end
  for l in L
      from=fromBus[l]; to=toBus[l]
      C[from,nbuses+to] -= 0.5*(  λF[l]*Y["ftI"][l] + μF[l]*Y["ftR"][l] - λT[l]*Y["tfI"][l] - μT[l]*Y["tfR"][l]  )
      C[to,nbuses+from] += 0.5*(  λF[l]*Y["ftI"][l] + μF[l]*Y["ftR"][l] - λT[l]*Y["tfI"][l] - μT[l]*Y["tfR"][l]  )
      if from < to
        C[from,to] += 0.5*(λF[l]*Y["ftR"][l]  - μF[l]*Y["ftI"][l] + λT[l]*Y["tfR"][l] - μT[l]*Y["tfI"][l])
        C[nbuses+from,nbuses+to] += 0.5*(λF[l]*Y["ftR"][l]  - μF[l]*Y["ftI"][l] + λT[l]*Y["tfR"][l] - μT[l]*Y["tfI"][l])
      else
        C[to,from] += 0.5*(λF[l]*Y["ftR"][l]  - μF[l]*Y["ftI"][l] + λT[l]*Y["tfR"][l] - μT[l]*Y["tfI"][l])
        C[nbuses+to,nbuses+from] += 0.5*(λF[l]*Y["ftR"][l]  - μF[l]*Y["ftI"][l] + λT[l]*Y["tfR"][l] - μT[l]*Y["tfI"][l])
      end
  end

  @variable(mMP, H[i=1:(2*nbuses),j=1:(2*nbuses)], PSD)
  @constraint(mMP, SetH[i=1:(2*nbuses),j=i:(2*nbuses)], C[i,j] - H[i,j] == 0 )
  @objective(mMP, Max, sum(ζpLB[g]*Pmin[g] - ζpUB[g]*Pmax[g] + ζqLB[g]*Qmin[g] - ζqUB[g]*Qmax[g]  for g in G)
    + sum( γm[i]*Wmin[i]-γp[i]*Wmax[i] + α[i]*PD[i] + β[i]*QD[i] for i in N))

  fn_base=string("CBF/","case",9,"-",3,".cbf")
  mathoptformat_model = MathOptFormat.CBF.Model()
  MOI.copy_to(mathoptformat_model, backend(mMP))
  MOI.write_to_file(mathoptformat_model, fn_base)

The only way this code writes the cbf file without an error is if I code out all of the constraints. With the error, the message is something like

ERROR: LoadError: MathOptInterface.UnsupportedConstraint{MathOptInterface.ScalarAffineFunction{Float64},MathOptInterface.EqualTo{Float64}}: `MathOptInterface.ScalarAffineFunction{Float64}`-in-`MathOptInterface.EqualTo{Float64}` constraint is not supported by the model.
Stacktrace: ....etc.

What am I missing? Thank you.

LP writer creating invalid names for COIN CBC

In my Julia file I am creating variables. The output .lp file has brackets for the variables and is creating many errors like the following when I am running cbc from the command line:

### CoinLpIO::is_invalid_name(): Name x[1] contains illegal character '['

Is there any way to fix this? Maybe we can replace the brackets with underscores?

Round trip working

@mlubin

First draft of the implementation is completed. I didn't focus on performance. Just functionality. We use OrderedDict's to maintain a nice standardised ordering in the JSON file. This is mainly so we can test easily without the keys being reordered, but it's also useful from a readibility perspective.

For an example see https://github.com/odow/MathOptFormat.jl/blob/master/test/runtests.jl#L90-L103

I just have the conic stuff to implement.

write_to_file() writes variable bounds in MPS file without corresponding columns

If a variable has no coefficients in the objective function or constraints, it is skipped in the column section but the bounds are still reported leading to unknown column errors.

I suppose it's technically not a bug since these are trivial variables, but the implementation would be more robust if this scenario was handled.

LP writer has wrong bounds for free variables

When writing a model to an LP file, if a variable is free, the LP writer will not write any bound information into the file. However, by default if a variable has no bound in an LP file, it means the variable is nonnegative. For a minimum example, suppose we have the following minimization problem with free variables gammas:

m = Model(with_optimizer(CPLEX.Optimizer))
@variable(m, gamma[i in 1:2])
@objective(m, Min, gamma[1] + gamma[2])
lp_file = MathOptFormat.LP.Model()
MOI.copy_to(lp_file, backend(m))
MOI.write_to_file(lp_file, "trivia_bug.lp")

JuMP produces the following lp file:

minimize
obj: 1 gamma_1_ + 1 gamma_2_
subject to
Bounds
End

If we solve this lp file with CPLEX, it will return an optimal value of 0. However, the problem should be unbounded.
The correct lp file should be:

minimize
obj: 1 gamma_1_ + 1 gamma_2_
subject to
Bounds
gamma_1_ Free
gamma_2_ Free
End

[MPS] Add options and set name warning to false by default

See, e.g., #72 and #60.

ConicBenchmarkFormat conversion

Re: JuliaOpt/ConicBenchmarkUtilities.jl#19

Would it make more sense to use MOF to access the CBF library (http://cblib.zib.de/) of conic problems from JuliaOpt, rather than trying to update and maintain ConicBenchmarkUtilities? We'd like to be able to both read and write CBF files (to contribute to CBLib).

Incorrect CBF models

I am trying to export a simple example from the JuMP documentation. The exported CBF model is incorrect, but the MPS model is fine.

Here is the JuMP example:

using JuMP, SCS, LinearAlgebra
using MathOptFormat

function example_basic()
    model = Model(with_optimizer(SCS.Optimizer, verbose=1))
 
    @variable(model, 0 <= x <= 2)
    @variable(model, 0 <= y <= 30)

    @objective(model, Max, 5x + 3y)
    @constraint(model, 1x + 5y <= 3.0)
    print(model)
  
    mathoptformat_model = MathOptFormat.MPS.Model()
    MOI.copy_to(mathoptformat_model, backend(model))
    MOI.write_to_file(mathoptformat_model, "jump_example_basic.mps")

    mathoptformat_model = MathOptFormat.CBF.Model()
    MOI.copy_to(MOI.Bridges.full_bridge_optimizer(mathoptformat_model,Float64), backend(model))
    MOI.write_to_file(mathoptformat_model, "jump_example_basic.cbf")

    JuMP.optimize!(model)
end

example_basic()

and here is the generated CBF model, where bounds on the variables are missing,

VER
3

OBJSENSE
MAX

VAR
2 1
F 2

OBJACOORD
2
0 5.0
1 3.0

CON
1 1
L- 1

ACOORD
2
0 0 1.0
0 1 5.0

BCOORD
1
0 -3.0

and finally the MPS model,

NAME          
ROWS
 N  OBJ
 L  c1
COLUMNS
     x        c1       1
     x        OBJ      -5
     y        c1       5
     y        OBJ      -3
RHS
    rhs       c1        3
RANGES
BOUNDS
 LO bounds    y        0
 UP bounds    y        30
 LO bounds    x        0
 UP bounds    x        2
ENDATA

Register v0.1.0

@JuliaRegistrator register

Refactoring of MOF

Here are some proposed refactorings for MOF

There is some redundancy that is a consequence of Julia structs, rather than any necessity. We don't need to know the head field in the following objects since they can only appear in set places (e.g., the terms field in a ScalarAffineFunction can only contain ScalarAffineTerms).

Remove "head" field in ScalarAffineTerm
Remove "head" field in ScalarQuadraticTerm
Remove "head" field in VectorAffineTerm
Remove "head" field in VectorQuadraticTerm

There is some inconsistencies between SingleVariable (field variable) and ScalarAffineTerm (field variable_index). Let's normalize to variable.

Rename variable_index to variable in ScalarAffineTerm
Rename variable_index_1 to variable_1 in ScalarQuadraticTerm
Rename variable_index_2 to variable_2 in ScalarQuadraticTerm

Unable to install MathOptFormat on julia 1.0.3

I've been trying to install MathOptFormat, so I can use SDDP package, but I always end up with the same error (shown bellow). I'm using julia 1.0.3 on a windows machine and running ] add "https://github.com/odow/MathOptFormat.jl.git".

Add SDPA

Links:

What to do about NL

It's an open question whether we should move the reading/writing of NL files from AMPLNLWriter over here. Thoughts?

https://github.com/JuliaOpt/AmplNLWriter.jl

Register v0.1.1

@JuliaRegistrator register

Export runs smoothly but returns corrupted file when variable base name includes a blank

When the base name of a variable includes a blank the export runs smoothly however the exported model (in this case to .mps) is corrupted and cannot be used either in other tools, or reimported into julia

This Works just fine

using JuMP, Gurobi, MathOptFormat
model = Model(with_optimizer(Gurobi.Optimizer));
t = [1,2,3]
@variable(model, aVar[t] >= 0,  base_name="aVar");
@variable(model, OBJ[t],  base_name="Obj");
@constraint(model,[T=t], OBJ[T] == aVar[T]*3 );
@constraint(model,[T=t], aVar[T] <= T-1);
@objective(model, Max, sum(OBJ[T] for T in t))
optimize!(model)

# Export
mps_model = MathOptFormat.MPS.Model()
MOI.copy_to(mps_model, backend(model))
MOI.write_to_file(mps_model, "ModelForTest.mps")

# Import
mathoptformat_model = MathOptFormat.MPS.Model()
MOI.read_from_file(mathoptformat_model, "ModelForTest.mps")

Now just add a blank into the variable name aVar -> "a Var"

using JuMP, Gurobi, MathOptFormat
model = Model(with_optimizer(Gurobi.Optimizer));
t = [1,2,3]
@variable(model, aVar[t] >= 0,  base_name="a Var");
@variable(model, OBJ[t],  base_name="Obj");
@constraint(model,[T=t], OBJ[T] == aVar[T]*3 );
@constraint(model,[T=t], aVar[T] <= T-1);
@objective(model, Max, sum(OBJ[T] for T in t))
optimize!(model)

# Export
mps_model = MathOptFormat.MPS.Model()
MOI.copy_to(mps_model, backend(model))
MOI.write_to_file(mps_model, "ModelForTest.mps")

# Import
mathoptformat_model = MathOptFormat.MPS.Model()
MOI.read_from_file(mathoptformat_model, "ModelForTest.mps")

The code fails at MOI.read_from_file returning the error message
"ERROR: Malformed COLUMNS line: a Var[2] c8 1"

Update:
I tried it with these sepcial characters "!§%&/()={[]}-_.,;:><" and all pose no problem. It's only the blank

Not building on Mac

I get the following error when using MathOptFormat on MacOS:

ERROR: LoadError: LoadError: error compiling doc!: error compiling #handle_message#2: could not load library "libz"

As found here it is a bug of GZip.jl; it can be fixed by first having these two lines:

using Libdl
push!(Libdl.DL_LOAD_PATH,"/usr/lib")

However this is not satisfying on the long term. One suggested solution is to use CodecZlib.jl instead, would that be acceptable?

Features not implemented

Model related

modifyobjective!(m, ::ScalarConstantChange)
modifyobjective!(m, ::ScalarCoefficientChange)
~~supportsproblem~~ #12

Variable Related

~~NumberOfVariables~~ #11
~~ListOfVariableReferences~~ #12
~~VariablePrimalStart~~ #8
~~MOF.VariableName~~ #8

Constraint Related

Easy-to-use interface?

Coming from JuMP 0.18-, when I see the interface that this package proposes, I think there is a large regression in user experience: a simple writeLP(m, filename) was all it took... Why not reviving the same interface in this package (which would supplement the existing one)? If you're OK with this idea, I can propose a PR for this (LP, MPS at least, maybe others?).