atoptima / blockdecomposition.jl Goto Github PK

https://github.com/atoptima/BlockDecomposition.jl/runs/2595574890

Instead of writing

tulip() = CL.MoiOptimizer(with_optimizer(Tulip.Optimizer)())
BD.assignsolver!(master, tulip)

we should have

BD.assignsolver!(master, Tulip.Optimizer)

when we try to solve a BlockModel with Cplex.
(maybe atoptima/Coluna.jl#301 related ?)

See atoptima/Coluna.jl#325

This would allow using AxisID seamlessly in other functions in custom getindex implementations

related to atoptima/Coluna.jl#378

example : https://github.com/atoptima/ColunaDemos.jl/blob/eb642e06a4bd9927ed4ef31f099f86deac6bbfcd/src/GeneralizedAssignment/model.jl#L4-L7

syntax

model = BlockModel(coluna, direct_mode = true)

Hi again,

Nothing urgent I'd say, but it would be nice being able to allow for the possibility foreseen in JuMP of creating expressions (http://www.juliaopt.org/JuMP.jl/v0.20.0/expressions/). While this functionality can in principle also be achieved by the creation of derived variables plus their constraints, the expression mechanism allows for cleaner and more compact model formulations.

When applying Coluna to a model that contains expressions, the currently thrown error is

ERROR: LoadError: MethodError: no method matching setannotation!(::Model, ::Int64, ::BlockDecomposition.Annotation{Int64,BlockDecomposition.Master,BlockDecomposition.DantzigWolfe})                                                                                       
Closest candidates are: 
  setannotation!(::Any, ::ConstraintRef, ::Any) at .julia/dev/BlockDecomposition/src/decomposition.jl:84 
  setannotation!(::Any, ::VariableRef, ::Any) at .julia/dev/BlockDecomposition/src/decomposition.jl:85 
Stacktrace: 
 [1] _annotate_elements!(::Model, ::JuMP.Containers.DenseAxisArray{Real,2,Tuple{Array{Symbol,1},Array{Symbol,1}},Tuple{Dict{Symbol,In
t64},Dict{Symbol,Int64}}}, ::BlockDecomposition.Tree) at .julia/dev/BlockDecomposition/src/decomposition.jl:41
 [2] register_decomposition(::Model) at .julia/dev/BlockDecomposition/src/decomposition.jl:12
 [3] optimize!(::Model) at .julia/dev/BlockDecomposition/src/BlockDecomposition.jl:31
 [4] #optimize!#97(::Bool, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(optimize!), ::Mod
el, ::Nothing) at .julia/packages/JuMP/CZ8vV/src/optimizer_interface.jl:121
 [5] optimize! at .julia/packages/JuMP/CZ8vV/src/optimizer_interface.jl:107 [inlined] (repeats 2 times) 
 [6] start_optimization(::Model) at /media/files/sw/coluna/algo.jl:185
 [7] top-level scope at .julia/packages/TimerOutputs/7Id5J/src/TimerOutput.jl:214
 [8] include at ./boot.jl:328 [inlined] 
 [9] include_relative(::Module, ::String) at ./loading.jl:1094
 [10] include(::Module, ::String) at ./Base.jl:31
 [11] include(::String) at ./client.jl:431
 [12] top-level scope at REPL[2]:1
in expression starting at /media/files/sw/coluna/algo.jl:21

same of lbs & ubs

Needed in atoptima/Coluna.jl#99 or atoptima/Coluna.jl#95

Need a method to retrieve the parent annotation of an annotation

Anonymous constraint (without name) should be assigned to the master

Describe the bug
Creating conditional constraints i.e. constraints that concern only some subproblems like
@constraint(model, cap[m in Machines; m in ["c", "d"]], sum(Weight[m][j] * x[m, j] for j in Jobs) >= Min_Capacity[m])
returns

BoundsError: attempt to access 0-element Array{Tuple{Any},1} at index [1]

To Reproduce
coluna_gap_v2_conditional-constraints.txt (could not upload .jl file)

Expected behavior
The conditional constraint works with plain JuMP, thus the expected behavior is that it works as well with Coluna.

ERROR: LoadError: BoundsError: attempt to access 0-element Array{Tuple{Any},1} at index [1]
Stacktrace:
[1] getindex at ./array.jl:728 [inlined]
[2] look_for_dec_axis(::BlockDecomposition.Tree, ::JuMP.Containers.SparseAxisArray{ConstraintRef{Model,C,Shape} where Shape<:AbstractShape where C,1,Tuple{Any}}) at /home/nsc/.julia/packages/BlockDecomposition/2uvhk/src/decomposition.jl:56
[3] register_decomposition(::Model) at /home/nsc/.julia/packages/BlockDecomposition/2uvhk/src/decomposition.jl:16
[4] optimize!(::Model) at /home/nsc/.julia/packages/BlockDecomposition/2uvhk/src/BlockDecomposition.jl:29
[5] #optimize!#78(::Bool, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(optimize!), ::Model, ::Nothing) at /home/nsc/.julia/packages/JuMP/iGamg/src/optimizer_interface.jl:131
[6] optimize! at /home/nsc/.julia/packages/JuMP/iGamg/src/optimizer_interface.jl:111 [inlined] (repeats 2 times)
[7] top-level scope at /media/files/sw/coluna/coluna_gap_v2_conditional-constraints.jl:101
[8] include at ./boot.jl:328 [inlined]
[9] include_relative(::Module, ::String) at ./loading.jl:1094
[10] include(::Module, ::String) at ./Base.jl:31
[11] include(::String) at ./client.jl:431
[12] top-level scope at REPL[1]:1
in expression starting at /media/files/sw/coluna/coluna_gap_v2_conditional-constraints.jl:101

Environment

• Linux
• Julia 1.2.0
• JuMP v0.20.0
• BlockDecomposition v1.0.0 #master
• Coluna v0.1.0 #master

Syntax :

for k in K # K is an axis
    w_val = JuMP.getvalue(w[k]) # w is the setup variable
    knapsacks = BlockDecomposition.getnonzerocolumns(model, subproblem[k])
    for knp in knapsacks
        lambda_val = BlockDecomposition.getvalue(knp) # value of the master column var
        for t in T
            z_val = BlockDecomposition.getvalue(knp, z[k,t]) # z subproblem var
        end
    end
end

atoptima/Coluna.jl#118

Current error message :

 ERROR: LoadError: MethodError: no method matching decompose_leaf(::Model, ::Type{BlockDecomposition.DantzigWolfe}, ::UnitRange{Int64})
Closest candidates are:
  decompose_leaf(::Model, ::Type{<:BlockDecomposition.Decomposition}, ::BlockDecomposition.Axis) at ~/.julia/packages/BlockDecomposition/BcFsv/src/tree.jl:133
  decompose_leaf(::BlockDecomposition.AbstractNode, ::Type{<:BlockDecomposition.Decomposition}, ::BlockDecomposition.Axis) at ~/.julia/packages/BlockDecomposition/BcFsv/src/tree.jl:138
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/BlockDecomposition/BcFsv/src/tree.jl:171

Undefined behavior, should throw an error.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

using BlockDecomposition
A = 1:3
B = ['a', 'b', 'c’]

Now

p = Iterators.product(A,B)
@axis(C, collect(p))

Future

@axis(C, A × B)

Create a method vcat(axis, array) to avoid vcat(axis.container, array).

We can change the specify! method in BlockDecomposition to allow the user to pass a vector of optimizers :

BD.specify!.(subproblems, lower_multiplicity = 0, solver = [MOI.Gurobi, my_callback_stage2, my_callback_stage3])

and we change the dispatch argument into solver :

stages = [ 
    ClA.ColumnGeneration(pricing_prob_solve_alg = ClA.SolveIpForm(solver = 1)),
    ClA.ColumnGeneration(pricing_prob_solve_alg = ClA.SolveIpForm(solver = 2)),
    ClA.ColumnGeneration(pricing_prob_solve_alg = ClA.SolveIpForm(solver = 3)),
]

So in that case, we do the exact phase with MOI.Gurobi and we use a callback for stages 2 & 3. The user will have to write a callback for each phase.

I don't see any better solution at the moment to handle atoptima/Coluna.jl#525 (comment).

Moreover, if we don't fix this, we'll have a design issue with customized optimizers. I would like to have these optimizers connected to Coluna through MOI. So to avoid buffering changes several times, I suggest to identify the solvers that have the same MOI optimizers and set the parameters just before optimizing the formulation. For instance, if you define solvers of your subproblem like this,

solvers = [
     with_attributes(MOI.RCSP, mode = :exact), 
     with_attributes(MOI.RCSP, mode = :heuristic),
     with_attributes(MOI.RCSP, mode = :speed_heuristic)
]

Coluna will instanciate MOI.RCSP only once for the subproblem, and will set the parameters just before optimizing the subproblem. We'll have to find a way to set parameters only when needed (because changing parameters may need some calculations ?).

What do you think ? (cc @vitornesello )

Originally posted by @guimarqu in atoptima/Coluna.jl#525 (comment)

Can be reproduced with cutting stock demo.

  MethodError: no method matching look_for_dec_axis(::Array{ConstraintRef{Model,C,Shape} where Shape<:JuMP.AbstractShape where C,1})
  Closest candidates are:
    look_for_dec_axis(!Matched::VariableRef) at /Users/gmarqu100p/.julia11/packages/BlockDecomposition/zvazJ/src/decomposition.jl:81
    look_for_dec_axis(!Matched::JuMP.Containers.SparseAxisArray) at /Users/gmarqu100p/.julia11/packages/BlockDecomposition/zvazJ/src/decomposition.jl:67
    look_for_dec_axis(!Matched::JuMP.Containers.DenseAxisArray) at /Users/gmarqu100p/.julia11/packages/BlockDecomposition/zvazJ/src/decomposition.jl:71

It would be nice being able to generate subproblems only for a certain subset, as for example in the case of the GAP, only for some of the machines. This would correspond to e.g. having centralized and decentralized machines.

DecMachines = ["a","b"]	# DECENTRALIZED MACHINES FOR WHICH SUBPROBLEMS SHALL BE GENERATED
CenMachines = ["c","d"]	# CENTRALIZED MACHINES WHICH SHALL BE PART OF THE MASTERPROBLEM
@axis(DecMachines)

@variable(model, x[m in vcat(DecMachines.container, CenMachines), j in Jobs], Bin)

This would allow being able to use the same variable and constraint definitions independent if subproblems shall be created only for parts of a certain set.

However, this results in the following error:

ERROR: LoadError: MethodError: no method matching store!(::Coluna.MathProg.Annotations, ::Nothing, ::Coluna.MathProg.Constraint)
Closest candidates are:
store!(::Coluna.MathProg.Annotations, !Matched::BlockDecomposition.Annotation, ::Coluna.MathProg.Constraint) at /home/nsc/.julia/dev/Coluna/src/MathProg/problem.jl:31
store!(::Coluna.MathProg.Annotations, !Matched::BlockDecomposition.Annotation, !Matched::Coluna.MathProg.Variable) at /home/nsc/.julia/dev/Coluna/src/MathProg/problem.jl:21
store!(::Coluna.MathProg.Annotations, !Matched::Coluna.MathProg.AbstractFormulation, !Matched::BlockDecomposition.Annotation) at /home/nsc/.julia/dev/Coluna/src/MathProg/problem.jl:41

To Reproduce
coluna_gap_v3_decompose-subsets.txt

Stacktrace

[1] create_origconstr!(::Coluna.MathProg.Formulation{Coluna.MathProg.Original}, ::Coluna.Optimizer, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::String, ::MathOptInterface.ScalarAffineFunction{Float64}, ::MathOptInterface.GreaterThan{Float64}, ::MathOptInterface.ConstraintIndex{MathOptInterface.ScalarAffineFunction{Float64},MathOptInterface.GreaterThan{Float64}}, ::Dict{Int64,Coluna.MathProg.Id{Coluna.MathProg.Variable}}) at /home/nsc/.julia/dev/Coluna/src/MOIwrapper.jl:154
[2] create_origconstrs!(::Coluna.MathProg.Formulation{Coluna.MathProg.Original}, ::Coluna.Optimizer, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::Bool, ::Dict{Int64,Coluna.MathProg.Id{Coluna.MathProg.Variable}}) at /home/nsc/.julia/dev/Coluna/src/MOIwrapper.jl:172
[3] register_original_formulation!(::Coluna.Optimizer, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::Bool) at /home/nsc/.julia/dev/Coluna/src/MOIwrapper.jl:195
[4] #copy_to#7(::Bool, ::typeof(MathOptInterface.copy_to), ::Coluna.Optimizer, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}) at /home/nsc/.julia/dev/Coluna/src/MOIwrapper.jl:206
[5] (::getfield(MathOptInterface, Symbol("#kw##copy_to")))(::NamedTuple{(:copy_names,),Tuple{Bool}}, ::typeof(MathOptInterface.copy_to), ::Coluna.Optimizer, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}) at ./none:0
[6] attach_optimizer(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}) at /home/nsc/.julia/packages/MathOptInterface/C1XBe/src/Utilities/cachingoptimizer.jl:149
[7] optimize!(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.AbstractOptimizer,MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}) at /home/nsc/.julia/packages/MathOptInterface/C1XBe/src/Utilities/cachingoptimizer.jl:185
[8] #optimize!#78(::Bool, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(optimize!), ::Model, ::Nothing) at /home/nsc/.julia/packages/JuMP/iGamg/src/optimizer_interface.jl:141
[9] #optimize! at ./none:0 [inlined] (repeats 2 times)
[10] optimize!(::Model) at /home/nsc/.julia/dev/BlockDecomposition/src/BlockDecomposition.jl:30
[11] #optimize!#78(::Bool, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(optimize!), ::Model, ::Nothing) at /home/nsc/.julia/packages/JuMP/iGamg/src/optimizer_interface.jl:131
[12] optimize! at /home/nsc/.julia/packages/JuMP/iGamg/src/optimizer_interface.jl:111 [inlined] (repeats 2 times)
[13] top-level scope at /media/files/sw/coluna/coluna_gap_v3_decompose-subsets.jl:103
[14] include at ./boot.jl:328 [inlined]
[15] include_relative(::Module, ::String) at ./loading.jl:1094
[16] include(::Module, ::String) at ./Base.jl:31
[17] exec_options(::Base.JLOptions) at ./client.jl:295
[18] _start() at ./client.jl:464
in expression starting at /media/files/sw/coluna/coluna_gap_v3_decompose-subsets.jl:103

Environment

• Julia 1.2.0
• JuMP v0.20.0
• BlockDecomposition v1.1.0 [~/.julia/dev/BlockDecomposition] (current master)
• Coluna v0.2.0 [~/.julia/dev/Coluna] (current master)

Method is undefined because first argument is the optimizer.