Formalize the Master-Part relationship. about datajoint-matlab HOT 18 CLOSED

I agree to formalizing the master/part relationship. However, I suggest simplifying the complex syntax part of schema.Class. Instead, let's use only part and define the first reference (i.e. -> schema.Class) as the master.

from datajoint-matlab.

I have considered that option but it seems like another implicit rule that must be explained in the documentation. I am not sure if there are cases when users might not want to have the first dependency to be on the master table. I have not yet thought of such an example but I am thinking of its potential uses.

from datajoint-matlab.

I agree with Dimitri on that. Such kind of implicit rules can potentially lead to obscure behaviour. I don't think that part of ... is really a complex syntax and it makes it very clear what is happening.

from datajoint-matlab.

We can throw a warning if the first dependency is not on the master table.

from datajoint-matlab.

It just feels unnecessarily redundant. In principle the entire first line of the definition is unnecessary; we could just subclass from dj.Imported, dj.Part etc. as in Python. So when we're changing the syntax anyway, I'd rather reduce redundancy that increase it. Ideally we use every name exactly once.

What would be a use case where the first reference is not the master? The order of references does not matter anyway, does it?

What kind of obscure behavior do you expect? From what I can tell the master/part relationship does not produce any "behavior". The only potential behavior I can see is preventing it from implementing makeTuples(), but the only way I can see we can accomplish that in Matlab is by defining the table their by subclassing dj.Part as in Python and not deriving dj.Part from dj.AutoPopulate.

from datajoint-matlab.

Yes, we could subclass from new classes dj.Manual, and dj.Lookup, dj.Imported, dj.Computed, etc. However, this would introduce backward incompatibility.

Master/part relationship has consequences for deletions. Our current plan is to delete the master if any of the parts is deleted. This prevents deleting any parts independently.

Another implication is for naming conventions. On the Python side, the convention is schema.master_name__part_name with double underscore separating the two names.

from datajoint-matlab.

The only problem that remains unresolved is issue #40. This is not a problem in python, because tables are created at the time of importing the module. In MATLAB, tables are created at the time of first use. This causes an implicit commit during transactions, which is a particular problem for part relations because they might not be first used until they are called from inside their master's makeTuples call.

from datajoint-matlab.

One solution for #40 is to make the master aware of all its parts and make it instantiate all the parts before entering the makeTuples transaction. What's the best syntax or mechanism for that?

from datajoint-matlab.

We might have to crawl the search path to discover all the classes that inherits from dj

from datajoint-matlab.

Crawling the search path could take seconds every time.

One approach is to declare parts as properties of master.

classdef Master < dj.Relvar
    properties(Constant)
        parts = [schema.Part1, schema.Part2]
    end
end

One problem is that matlab has a hard time clearing properties of classes. Another problem is that now master/part relationship is established in two places.

from datajoint-matlab.

I would be strongly opposed to manipulating (constant) properties in Matlab - it' gets messy very quickly. Maybe we can maintain the list of parts in a separate file that gets read in? This sounds terrible because it'll be another separate file, but it will be cleaner update.

from datajoint-matlab.

Parts could also be listed in the table declaration.

from datajoint-matlab.

But if you modify it later, wouldn't that be a mess?

from datajoint-matlab.

We could list them in a string:

classdef Master < dj.Relvar
    properties(Constant)
        parts = 'Part1, Part2'
    end
end

from datajoint-matlab.

Actually I take that back - if we put that as part of the comment at the top, it's no different from my separate file with list solution.

from datajoint-matlab.

@eywalker and I just spoke and came up with the following:

1. If a table declaration is attempted within a transaction, an error will be raised.
2. Master and part must be in the same schema (that's also true in Python)
3. Parts will be declared with part of schema.Master in the first line of the declaration.
4. When populate gets called, it will examine all the .m files in the package, parse the declarations, and find all classes that declare themselves as part of the calling class. It will then instantiate all the part classes and trigger their declarations.

from datajoint-matlab.

Fixed in #72 by introducing a the dj.Part class.

from datajoint-matlab.

Master and pard table concepts are now formally introduced into DataJoint MATLAB beginning in v3.0.0 (PR #83). @dimitri-yatsenko Should we just close this issue now?

from datajoint-matlab.