Data seeding shell about migrations HOT 26 CLOSED

Couldn't you make the data seeding a migration?

from migrations.

Yes.. I know about this feature. But i have found this process too repetitive.
Таке a look at this migration https://github.com/iteam-pro/passengers/blob/master/config/Migrations/20141015144419_initial_migration.php
Data seeding pieces of code for Roles table and Users table very similar. except password of course.
But i sure this logic

        $things = TableRegistry::get('PluginName.Things');
        foreach ($this->things as $thing){
            $thing = $things->newEntity($thing);
            $things->save($thing);
        }

acceptable for 99% data seeding processes.
Simple piece of code with some modifications can pickup data from file and insert it into database.
Developer only need to write data files or dump it by shell from own db(for example)
Also we can put callback function in data files to possibility transform it on the fly. Change password for example.

from migrations.

How could it be made less repetitive?

from migrations.

@lorenzo for example when my migration contain more than 5 tables.

from migrations.

I understand that it can be repetitive, but I'm just asking what would be a good way for you to lessen the repetitive stuff

from migrations.

I can write shell and sample plugin to demonstrate what i mean. And after this you with your teammates will take decision accept this idea as pull request or not. ok?

from migrations.

Croogo already has something like this: https://github.com/croogo/croogo/blob/master/Extensions/Lib/Utility/DataMigration.php

Bulk load datafiles from a directory: https://github.com/croogo/croogo/blob/master/Install/Model/Install.php#L69-L82
Load selective records from a file: https://github.com/croogo/croogo/blob/master/Settings/Config/Migration/1392779470_setting_updates.php#L40-L50

from migrations.

@rchavik So why shouldn't to make this logic as default for migrtaions plugin?

from migrations.

@TamiasSibiricus Sure, it's up to the migrations plugin maintainer. I'm just pointing those files out should you want to reuse the code as a base for the PR.

from migrations.

@TamiasSibiricus sure, I'd like to see your idea

from migrations.

Why not? I'd like to see a PR, too.

from migrations.

Closing as there is a PR open.

from migrations.

Ok. So my vision for seeding shell:

• data stores as [ yaml | json | toml | csv ] files. for first time we could realize csv or json. JSON can store data for multiple tables in one file. With csv this is little bit a problem.
• if possible data could be translatable. I know this is not first priority, but...
• make choice between seeding strategies: 1."data as part of migration process" vs 2."data as postmigration process"

from migrations.

Honestly I think this should be a plugin on it's own that could extend or make use of migrations. Either migrations gets events fired on before / after / up / down or extend the migrations plugin in another plugin to get this done.

from migrations.

@burzum I think about seeding as separate plugin. But here is ine problem. Callbacks runs only after whole migration process. So there is impossile relate seeding process to selected migration. Phing currently does not return any info about applied migrations. Also if we realize seeding process as callback it would run every migration and try overwrite existing data after each migration process(i think). Currently there is no mechanism to prevent run seeding twice or more.

from migrations.

@TamiasSibiricus Like I said before, this is easy to resolve with events:

new Event('Migration.beforeMigration', $this, ['direction' => 'up']);
new Event('Migration.beforeMigration', $this, ['direction' => 'down']);
new Event('Migration.afterMigration', $this, ['direction' => 'up']);
new Event('Migration.afterMigration', $this, ['direction' => 'down']);

Listen to the event(s) and put your logic in the event listener that deals with it. Identify the migration by class name and you get the direction from the event data and act accordingly. I think it should be possible to know what was done (or is going to be done) as well and to pass it to the event.

@lorenzo can I add them? 😄

from migrations.

@burzum someone did that already

from migrations.

@burzum Events for migration actions migrate and rollback already exists but without subject and options arguments. Just take a look at code for this actions. I think in current version options does not needed.
Also i was try to debug $input and $output vars. They does not contain any helpful info about applied migrations. So i take decision do not transfer them to event callback as options.

from migrations.

Ref: http://docs.phinx.org/en/latest/seeding.html

from migrations.

Everyone's situation is going to be different obviously, but my team and I ended up writing a Cake plugin to do data seeding. It still isn't tied to specific migrations, but there's no reason it couldn't be (just have your migration's after() (or equivalent event) invoke the appropriate Shell command and seed file name).

Anyway, here's another alternative in case it's useful to anyone that finds this thread:

https://github.com/loadsys/CakePHP-Basic-Seed

from migrations.

Would love to see this implemented, would be a whole lot easier!

Ref: @sdustinh http://docs.phinx.org/en/latest/seeding.html

from migrations.

@beporter I'd love to somehow have the following feature implemented for the CakePHP ORM: data-seeding from a production database. Maybe something you guys at Loadsys can investigate ;)

from migrations.

You can still use the built in Phinx data seeding classes in your change() or up() methods. It would be nice though if we could bake separate Seeding files.

$data = [
    [
        'name' => 'Return of the Jedi',
        'series' => 'VI'
    ],
    [
        'name' => 'The Force Awakens',
        'series' => 'VII'
    ]
];

$this->table('movies')
       ->insert($data)
       ->save();

from migrations.

What would be the initial seed data? Just empty?

from migrations.

I've been using the TableRegistry in the up() method with a migration file prefixed like 'SeedRoles'. Honestly the only complaint I have with this method is that it marks the migration as migrated. If I could toggle that behavior I'd be set.

from migrations.

With the support of seeders added to phinx and the migrations plugin enhancements following this, I think this can be closed.

from migrations.