No documentation for associations/foreign key joins about rom HOT 30 CLOSED

Hey sorry for a late reply - ROM doesn't support relationship DSL but it does support joining relations. I will come up with some examples that will (hopefully) work so you should be able to judge yourself if it's good enough for you.

from rom.

No big rush, I tried using ROM with the axiom-memory-adapter and emulating ActiveRecord::Base.first_or_create! by using ROM::Relation#restrict in a spec and 6 examples took 71 seconds, so I'm not going with ROM for now and just manually optimizing searches with hashes, etc.

It appeared that doing the check for collisions with restrict was adding 60 seconds on top of just creating the rows. Here's the code I was using. If I'm doing something incredibly stupid please let me know as I'd much prefer to be able to use ROM and an AREL-like syntax than have to manually do the in-memory search when the user doesn't have a database and I can't use AREL.

BITS_PER_BYTE = 8
RANDOM_ID_BYTES = 16
RANDOM_ID_BITS = RANDOM_ID_BYTES * BITS_PER_BYTE
RANDOM_ID_N = 2 ** RANDOM_ID_BITS

#
# Methods
#

# An environment for in-memory models.
#
# @return [ROM::Environment] environment with schema and mapping already
#   declared
def self.environment
    environment = ROM::Environment.setup(
            memory: 'memory://metasploit-framework'
    )

    environment.schema do
        base_relation :architectures do
            repository :memory

            attribute :id, Integer
            attribute :abbreviation, String
            attribute :bits, Integer
            attribute :endianness, String
            attribute :family, String
            attribute :summary, String

            key :id
        end
    end

    environment.mapping do
        architectures do
            map :id,
                    :abbreviation,
                    :bits,
                    :endianness,
                    :family,
                    :summary

            model Metasploit::Framework::Architecture
        end
    end

    seed(environment)

    environment
end


# Generates a random ID with the same number of bits as
# `SecureRandom.uuid`
#
# @return [Integer]
# @see SecureRandom.random_number
def self.random_id
    SecureRandom.random_number(RANDOM_ID_N)
end

def self.seed(environment)
    ROM::Session.start(environment) do |session|
        Metasploit::Model::Architecture::SEED_ATTRIBUTES.each_with_index do |attributes, i|
            relation = session[:architectures].restrict(attributes)

            if relation.count == 0
                seed = Metasploit::Framework::Architecture.new(attributes)
                seed.id = random_id
                seed.valid!

                # XXX not sure why I need to manually call track since the
                # rom-rb.org shows me just using save.
                session[:architectures].track(seed)
                session[:architectures].save(seed)
            end
        end

        session.flush
    end
end

from rom.

Oh wow that's great feedback thank you for this. I will take a closer look and get back to you asap.

from rom.

while somewhat offtopic your can create an inmemory sqlite database: https://gist.github.com/schmurfy/6378497

from rom.

If you are on *nix you can create the sqlite db on the /tmp folder, since
it is RAM based.
Em 29/08/2013 11:05, "Julien Ammous" [email protected] escreveu:

while somewhat offtopic your can create an inmemory sqlite database:
https://gist.github.com/schmurfy/6378497

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-23491614
.

from rom.

metasploit-framework used sqlite in the past, but metasploit-framework is threaded and sqlite kept having issues, so I was advised that it wasn't worth using sqlite @schmurfy. Additionally, the full schema from metasploit_data_models using postgres only features like INET columns.

metasploit-framework needs to run on Windows and *nix-like systems (Linux and OSX mostly), so I can't depend on that behavior @mereghost.

from rom.

@solnic I wanted to make sure you noticed, but I had to make two non-obvious changes from the examples on rom-rb.org:

1. I had to call session[:archtectures].track(seed) before I could call session[:architectures].save(seed).
2. I had to use session.flush instead of session.commit.

from rom.

Yeah that's my bad. I will fix the examples. Thanks for pointing this out.

On Thu, Aug 29, 2013 at 4:24 PM, Luke Imhoff [email protected]
wrote:

@solnic I wanted to make sure you noticed, but I had to make two non-obvious changes from the examples on rom-rb.org:

1. I had to call session[:archtectures].track(seed) before I could call session[:architectures].save(seed).

2) I had to use session.flush instead of session.commit.

Reply to this email directly or view it on GitHub:
#3 (comment)

from rom.

@limhoff-r7 I updated the example.

You can use a short-cut syntax to create new object and track it automatically via session[:architectures].new(seed).

How many objects do you have in that seed btw?

from rom.

@limhoff-r7 can you see if it gets a bit faster when doing environment[:architectures].restrict(seed).count instead of using session?

The difference is that session sets up IdentityMap and state tracking and "plain" environment does not. You might get a perf boost. We also don't have an optimised #count yet so at the moment it loads all the objects matching criteria.

Is this code publicly available? I could try it out locally and see how I could improve stuff :)

from rom.

@limhoff-r7 oh and btw how many objects do you have there?

from rom.

16 objects when I did the test.

from rom.

I wouldn't have used count if there was an exists? or exist? method.

from rom.

@limhoff-r7 ugh 16 objects? that's very weird. I need to try to reproduce this, something is wrong. I was expecting you'd say 16k objects or something heh

from rom.

I pushed up a branch with the specs running slow here: https://github.com/limhoff-r7/metasploit-framework/tree/feature/rom (so the feature/rom branch of my fork of metasploit-framework).

git clone [email protected]:limhoff-r7/metasploit-framework.git
cd metasploit-framework
git checkout feature/rom
rspec spec/app/models/metasploit/framework/memory_spec.rb

The specs as a whole will fail since this branch was cut off of feature/module-caching, which is under going major rework, so only run this one spec.

from rom.

If you're not using an IDE like Rubymine that allows you to click-through to definition, it may not be obvious what the seed attributes actually are, so here's a direct link to them in metasploit-model: https://github.com/rapid7/metasploit-model/blob/feature/module-caching/lib/metasploit/model/architecture.rb#L51

from rom.

@limhoff-r7 thank you! I'll investigate during the weekend

from rom.

@snusnu do you think it makes sense to show how you can set up joined relations? it would be pretty verbose and people might be confused by it (maybe?). wdyt?

from rom.

@solnic yes i think that makes sense, and i don't think that it's pretty verbose by the way. Imo we shouldn't only think of this as a relationship replacement, but as demonstrating a way to map SQL views (with the added benefit of being writable under a lot of circumstances). Furthermore, I have the feeling that apps developed with ROM will typically involve some mappings of this kind. I think that quite a few places will emerge, were modeling a specific set of data is more naturally expressed by means of (mapping) a view , than by (building new objects around) relationships.

from rom.

@snusnu I wholeheartedly agree there’s only one caveat right now - mappers don’t support loading data from a joined relation

from rom.

@solnic oh heh, i forgot. Do EVs work already, or is it just ECs that don't work atm?

from rom.

@snusnu you'd have to build your own mapper, there's no EV/EC concept in rom-mapper yet but we could probably easily add it

from rom.

@solnic then i guess we should show an EV example with a simple custom mapper. For EC, i guess we'd either postpone it until group/ungroup lands in axiom at which point it will be trivial for rom-mapper. Or we try to come up with some intermediate solution that knows how to hand a set of flat tuples to a mapper instance for consumption. I'd hate to do that fwiw, and that's also the reason why I made a case for having group/ungroup support in axiom sooner rather than later. It's imo even more important than SQL write support, as i've said before.

from rom.

@solnic our recent design change, resulting in mapper only ever seeing one tuple, makes group/ungroup even more important.

from rom.

@solnic fwiw, that doesn't mean i'm against providing something that can be used in ROM::Relation#each and "collapses" the n flat tuples into one grouped tuple, sending that to mapper.load. But I'm afraid that we will need quite some infrastructure for that to work out reasonably well (and I haven't yet thought about write support). We'd need FK information in order to perform group properly ourselves. Unless I'm missing something, of course.

from rom.

@solnic I'm afraid that hand rolling group/ungroup in rom-relation, will add tons of completely superfluous complexity, given that it will be replaced by something really trivial eventually. I'm totally not sure if that's worth it.

from rom.

Who’s talking about group/group? :) I’m talking about mapping a bunch of attributes into an embedded object.

from rom.

@solnic yeah, no problem for EV. but how to do EC without group/ungroup?

from rom.

@solnic let's continue this in IRC (needn't be now)

from rom.

FYI, Axiom::Relation now supports #group,#ungroup, #wrap and#unwrap`. This should open up EC and EV support in ROM once the following is added:

• ROM::Relation#group
• ROM::Relation#ungroup
• ROM::Relation#wrap
• ROM::Relation#unwrap

from rom.