an Instance type about mathoptinterface.jl HOT 25 CLOSED

If we have an API for InstanceData then we might as well make it abstract. The default one would be e.g.

struct InstanceData{T} <: AbstractInstanceData{T}
    vars::Vector{VariableReference}
    obj::Objective{T}
    linconstrs::Vector{Tuple{AbstractSet, LinConstr{T}}}
    quadconstrs::Vector{Tuple{AbstractSet, QuadConstr{T}}}
    veclinconstrs::Vector{Tuple{AbstractSet, VecLinConstr{T}}}
    vecquadconstrs::Vector{Tuple{AbstractSet, VecQuadConstr{T}}}
    varconstrs::Vector{Tuple{AbstractSet, VariableReference}}
    vecvarconstrs::Vector{Tuple{AbstractSet, Vector{VariableReference}}}
end

where LinConstr, ... are defined as one would expect (Note that we could also define addconstraint!(..., ::LinConstr, ...), ... accordingly).
If one finds it inefficient to have AbstractSet instead of a concrete type, he can define a new AbstractInstanceData subtype with one field per set that he needs in his package.

from mathoptinterface.jl.

I spoke with Miles since he posted and he is pretty sure it should not be in MOI.

I agree @joaquimg - MOI should be simple and easy to understand and lightweight.

from mathoptinterface.jl.

@ccoffrin @odow @blegat @joaquimg @chriscoey @joehuchette @jrevels

from mathoptinterface.jl.

Type stability might be meaningful here. On the other hand, a dictionary would be nice, since it would be more natural to add new constraint types.

from mathoptinterface.jl.

You are thinking about keeping all the instance data in the struct, including matrix coefficients?
I so I would be a little worried about keeping the same info in 3 places (JuMP, MOI and Solver).
But if all the data you mean all the references I think its a nice idea.
I prefer not to have a dictionary, arrays are so much faster... In my opinion, MOI should be lightweight and JuMP should make everything pretty.
If type stability is important here we should care about it so that MOI has minimal overhead.

from mathoptinterface.jl.

You are thinking about keeping all the instance data in the struct, including matrix coefficients?

Yes, everything.

JuMP will need an internal data structure to represent an MOI instance. I'm proposing to make this into a standard, documented data structure at the MOI level instead of at the JuMP level, because it will be useful outside of just JuMP. This data structure will replace linconstr, quadconstr, sosconstr, etc., so there will still only be two copies of the model.

from mathoptinterface.jl.

so there will still only be two copies of the model

This is good.

throwing everything to a SolverInstance at once

I like it.

Do you have a sketch?

Do you have more pros and cons on the top of the head?

from mathoptinterface.jl.

I think a reasonable interface for this data structure would be the existing getters and setters like addconstraint!, getconstraintaffine, getconstraintquadratic, etc. so that we have the ability to easily modify the internal representation later on.

Conceptually we'll have the following high-level objects:

• Solver: the existing solver objects like GurobiSolver()
• SolverInstance: an instance whose data belongs to the solver API, supports optimize!, getting solutions back, etc.
• InstanceData: the concrete data for an instance, independent of any solver. JuMP will use this data structure internally to store essentially everything that it will pass to a solver. The interface mimics SolverInstance but you can't call optimize! or ask for SolverInstance attributes. A complete implementation would support adding and removing constraints and variables. On second thought, this might belong in a separate package or in JuMP itself.

from mathoptinterface.jl.

On second thought, this might belong in a separate package...

I like this idea, and keep MOI very lightweight

from mathoptinterface.jl.

I don't mind where it lives, but having a struct specifically for instance data seems like great idea to me.

@joaquimg, can you elaborate on your performance concern of Dict vs Array? In practice, if the hash function is good and easy to compute, Dicts can be extremely fast for add+lookup. And you get fast remove, which you don't get for Arrays.

from mathoptinterface.jl.

Yes, definately single access with good hash functions is great. And being able to remove is very useful for many applications. Arrays are good for some other aplications that are "loop intensive" with large data, one get huge speedups by good use of cache. Maybe dicts turn to be good for this applications, I just want to keep alive the discussion on efficiency and speed.

from mathoptinterface.jl.

Sure thing. I am always in support of doing a speed comparison and seeing if the benefits justify any performance hit.

from mathoptinterface.jl.

@blegat, that seems too inefficient to use in JuMP, for example. It would be nice to have an easy way to create a fast container with concrete types that's also general enough to work with arbitrary sets.

Anyway I'm convinced at the moment that this belongs in a separate package.

from mathoptinterface.jl.

Since the InstanceData is solver agnostic, shouldn't it be called rather ProblemData or ModelData ? It looks a bit like the replacement of the LPQP and Conic models in MPB. For example, one could do:

struct LPQPInstanceData <: AbstractInstanceData # Here Instance sounds weird
    ... # one field for each set/function but without SOC, SDP, ...
end

from mathoptinterface.jl.

From my experince these are the meaning of various terms. Problem and Model are interchangeable, and most often refer to a mathematical specification that defines both constants and variables. Instance seems to be reserved for the case where the problem/model has been specified and you are providing specific values for the constants defined in the problem/model.

from mathoptinterface.jl.

ProblemData and ModelData sound fine to me for this.

I like SolverInstance to replace MPB's Model to make it clear that we're imagining it to be an instance loaded into the solver's API, while Model seems to misleadingly imply that there's a separation between the solver and the problem data.

from mathoptinterface.jl.

Some people may be coming from the AMPL world and think "data" means the data for a particular "model" (or algebraic, data-independent representation of a class of optimization problems). We should avoid that confusion.

from mathoptinterface.jl.

How do we avoid that confusion without avoiding the terms model or data? AMPL doesn't have ownership over these terms.

from mathoptinterface.jl.

Carleton said

Problem and Model are interchangeable, and most often refer to a mathematical specification that defines both constants and variables.

But in AMPL a model (.mod file) does not define "constants", rather those go in the data (.dat file). So I tend to think of a model as a data-independent algebraic representation.

from mathoptinterface.jl.

Generally I would agree that "instance" refers to a concrete optimization problem with data.

from mathoptinterface.jl.

It seems that we agree that what we have here is a "Model Instance Data" (or "Problem Instance Data").
The name InstanceData might be confused with "Solver Instance Data" which is not what we want.
Which name is better then ? ModelInstanceData, ModelData, ModelInstance ProblemInstanceData, ProblemData, ProblemInstance ?

We might do something similar to solvers but with models.
We can define different types of models (e.g. a model supporting only linear (i.e. LPQP), one supporting only SDP and linear (i.e. SDPA format/github.com/JuliaOpt/SemidefiniteModels.jl).
Each of them would have its type Model and we can create instances which would have type ModelInstance. It would be exactly like Solver and SolverInstance so it would be easy to understand and remember. What do you think ?

from mathoptinterface.jl.

@blegat, I think we should try to be consistent with the name Instance. We have two things that we need to name: 1) an instance loaded into the API of a solver (currently SolverInstance), and 2) an instance loaded into an in-memory data structure not attached to a solver. Maybe SolverInstance and SolverfreeInstance? The main point of distinction is if the instance belongs to a solver or not.

Having different SolverfreeInstance types for specialized problem classes is okay but doesn't solve the most pressing issue which is that JuMP needs a data structure which is completely general and also reasonably fast.

from mathoptinterface.jl.

Maybe just Instance and SolverInstance?

from mathoptinterface.jl.

I am fine with Instance

from mathoptinterface.jl.

Ok, so now we should define which attributes and methods apply to both Instance and SolverInstance.
Easy cases:

• TerminationStatus, optimize!, SolveTime, ... are obviously SolverInstance only
• addconstraint!, setobjective!, ObjectiveFunction, NumberOfConstraints clearly apply to both

What about: VariablePrimalStart?

Do we really need two separate concepts for Instance and SolverInstance, or could Instance just be a case where non-relevant attributes are not accessible and optimize! throws an error?

from mathoptinterface.jl.