percolatestudio / meteor-migrations Goto Github PK

Currently, the migrations collection only keeps track of the current migration and whether or not control is locked. While this is helpful information, it'd be great if there was an option for the package to insert a document in the collection that stored what migration ran, when it ran, if it was successful, and which direction it was run. It may be something that is disabled by default and enabled with a configuration option.

I have migrations versioned rails-style by timestamps. There's 8 migrations so far. My 8th migration version is 1454599427. If I do Migrations.migrateTo('1454599427,rerun') (passing parameters as a string is extremely lame btw), I get 'no method up() for undefined'. Reason is migrations is taken from list by index 1454599427, and real index is 8. If I do Migrations.migrateTo('8,rerun'), it works.

I propose to change getting by index implementation to at least finding by {version: ...} implementation which won't harm performance at all: suppose you have enterprise-grade Meteor project (is it real?) with 1000 migrations, your fetch will look like O(1000) loop. Not an issue as it still very fast and is run only by developer anyways.

If you're ok with this, let me make a pull request.

As an alternative there can be additional object that keeps indexed migrations (as you need this array anyways I suppose for its ordering)

Removing a column from SimpleSchema is problematic. The obvious code is

SomeCollection.update({}, {$unset: { someAttribute: 1 }}, {multi: true});

But SimpleSchema fails with Error: someAttribute is not allowed by the schema. This works...

SomeCollection.update({}, {$unset: { someAttribute: 1 }}, {multi: true, validate: false});

...but it feels hacky and something no one will think of until they've been burned. Some ideas:

• The framework could provide a function removeField(collection, field)
• You could temporarily override Collection.update() and always add validate: false.
• If nothing else, it should be mentioned in the README.
• Something better that we didn't think of.

I'm getting this error when running the package, perhaps because there is no "meteor logging package" anymore?

After setting "console" as the logger it does work for me.

  Migrations.config
    log        : on
    logger     : console
    logIfLatest: on

I think the FAQ about migration on the doc a bit over complicated. In the end I added this
if statement in my code. $UNLOCK_MIGRATE is set to false and I can simply change
its value without messing up with "meteor mongo"

  if process.env.UNLOCK_MIGRATE
    Migrations._collection.update({_id: "control"}, {$set: {locked: false}})

hope this helps

I don't see anywhere in the docs a mention of callbacks. I see Migrations is synchronous, but a migration may have async operations. In that case one needs to use futures or such to have a return wait.

I would expect there to be an API like:

Migrations.migrateTo(1, function(err, res){ 
    console.log('hello')
    //run maintenance method
    //trigger external api call
    // etc. etc.
 });

Am I not seeing something in the docs or is this intentionally not implemented?

It would also be worth mentioning an example of async updates in the docs. For example, bulk updates will finish after the migration returns completed, unless using Futures or other.

First thanks for your package, I have been using it for a while and works perfectly !
I wanted to configure a config with a custom collection name but it doesn't seem to work. I have two apps on the same database, and would like to handle 2 separate flows of migrations.
Code snippet imports/startup/server/index.js:

Meteor.startup(function () { 
    Migrations.config({
        collectionName: "migrationssecondapp"
    });
    Migrations.unlock();
    Migrations.migrateTo(1);
}

error : Error: [Can't find migration version 7], at Object.Migrations._findIndexByVersion
Version 7 is the version I migrated in the other app.

Trying to log Migrations._collection still gives the collection migrations.

[email protected]
percolate:[email protected]

Any hint on how to fix ? I could do a PR if needed
Thanks a lot

Is there a "best practice" way of detecting a failed migration from within Meteor? My use case is that my app is hosted on Modulus.io, so if a migration fails I have no way of knowing unless I check the logs there.

Basically, if a migration fails I want a way to flag this to our admin team from the UI so that they can do something about it.

thanks for the package guys. works like magic on the command line. However when i place the

Migrations.migrateTo('latest');

W20140128-15:30:07.907(8)? (STDERR) Error: [Can't find migration version latest]
W20140128-15:30:07.908(8)? (STDERR)     at Object.Migrations._findIndexByVersion (packages/percolatestudio-migrations/migrations_server.js:179)
W20140128-15:30:07.908(8)? (STDERR)     at Object.Migrations.migrateTo (packages/percolatestudio-migrations/migrations_server.js:121)
W20140128-15:30:07.908(8)? (STDERR)     at app/server/main.js:1:47
W20140128-15:30:07.908(8)? (STDERR)     at app/server/main.js:3:3
W20140128-15:30:07.908(8)? (STDERR)     at /Users/shawnlim/Documents/Code/meteorjs/microscope/.meteor/local/build/programs/server/boot.js:155:10
W20140128-15:30:07.908(8)? (STDERR)     at Array.forEach (native)
W20140128-15:30:07.909(8)? (STDERR)     at Function._.each._.forEach (/Users/shawnlim/.meteor/tools/09b63f1ed5/lib/node_modules/underscore/underscore.js:79:11)
W20140128-15:30:07.909(8)? (STDERR)     at /Users/shawnlim/Documents/Code/meteorjs/microscope/.meteor/local/build/programs/server/boot.js:82:5

this is my app code where i placed the migration code, located in /server/migrations.js:

Migrations.add({
    name: 'Add intercom hash to users.',
    version: 1,
    up: function(){
        var users = Meteor.users.find({intercomHash: {$exists: false}});
        users.forEach(function (user) {
            Meteor.users.update(user._id, {$set: {
                intercomHash: IntercomHash(user, 'abc           
                        }})
        });
    },
    down: function(){
        Meteor.users.update({}, {$unset: {intercomHash: true}}, {multi: true});
    }
})

Migrations.add({
    name: 'Add a dummy field dummy',
    version: 2,
    up: function(){
        var users = Meteor.users.find({dummy: {$exists: false}});
        users.forEach(function (user) {
            Meteor.users.update(user._id, {$set: {
                dummy: true         
            }})
        });
    },
    down: function(){
        Meteor.users.update({}, {$unset: {dummy: true}}, {multi: true});
    }
})

Migrations.migrateTo('latest');

I want to be able to add multiple migrations in one version, below is a sample

Migrations.add({
version: 1,
name: 'Adds pants to some people in the db.',
up: function() {//code to migrate collection 1 up to version 1}
down: function() {//code to migrate down to version 0}
});

Migrations.add({
version: 1,
name: 'Adds pants to some people in the db.',
up: function() {//code to migrate collection 2 up to version 1}
down: function() {//code to migrate down to version 0}
});

How can I do this? A scenario for this would, we can have multiple people writing migrations, so would want to separate their individual migrations and run them in some order.

Is there a planned update for Meteor 0.9?

Reason is you set locked state in db and unset it at the end. If migration didn't end due to network fail, package will continue to be in locked state infinitely unless you unlock with Migrations._collection.update({_id: 'control'}, {$set: {locked: false}}) manually but even then your migration is still 'in the middle', meaning part of data is migrated and another part is not. It is ok if migration script is idempotent though.

I think the option --run has been changed to just run .

could you confirm this as a bug, if so I'll make a PR.

Is there a way to use the Migration package to support multiple package-specific data versions?

For example, package-A introduces its own collections, which is versioned. package-B has its own collections. The app as a whole has its collections. Therefore there are three separate independent version series.

Perhaps this is a feature request.
Thanks
Ted

Really enjoying this plugin, it works really well for us.

I would like an option to tell this plugin to be silent unless it actually does a migration.

My goal is to hide these lines during development:

=> Meteor server restarted
I20140227-16:03:19.976(-8)? Not migrating, already at version 5
=> Meteor server restarted
I20140227-16:03:46.872(-8)? Not migrating, already at version 5
=> Meteor server restarted
I20140227-16:05:17.058(-8)? Not migrating, already at version 5
=> Meteor server restarted
I20140227-16:05:33.446(-8)? Not migrating, already at version 5

After running meteor update to upgrade from meteor version 1.1.0.3 to 1.2.0.2, I'm getting the following error:

W20151013-13:48:35.154(9)? (STDERR) /home/jeremy/.meteor/packages/meteor-tool/.1.1.9.1akpeh2++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:245
W20151013-13:48:35.154(9)? (STDERR)                         throw(ex);
W20151013-13:48:35.154(9)? (STDERR)                               ^
W20151013-13:48:35.164(9)? (STDERR) TypeError: Cannot call method 'info' of undefined
W20151013-13:48:35.165(9)? (STDERR)     at Object.Migrations._migrateTo (meteor://💻app/packages/percolate_migrations/migrations_server.js:161:1)
W20151013-13:48:35.165(9)? (STDERR)     at Object.Migrations.migrateTo (meteor://💻app/packages/percolate_migrations/migrations_server.js:139:1)
W20151013-13:48:35.165(9)? (STDERR)     at meteor://💻app/server/migrations.js:74:1
W20151013-13:48:35.166(9)? (STDERR)     at meteor://💻app/server/migrations.js:74:1
W20151013-13:48:35.166(9)? (STDERR)     at /home/jeremy/Code/iohk/enrollment-app/attainenroll/.meteor/local/build/programs/server/boot.js:242:10
W20151013-13:48:35.166(9)? (STDERR)     at Array.forEach (native)
W20151013-13:48:35.166(9)? (STDERR)     at Function._.each._.forEach (/home/jeremy/.meteor/packages/meteor-tool/.1.1.9.1akpeh2++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/underscore/underscore.js:79:11)
W20151013-13:48:35.167(9)? (STDERR)     at /home/jeremy/Code/iohk/enrollment-app/attainenroll/.meteor/local/build/programs/server/boot.js:137:5

Here are my package versions: https://gist.github.com/jjman505/0863738de256184616c5

Would it be a good idea to add a new document for each migration that includes a list of the IDs of all the documents modified by the migration? This way each migration would be more easily reversable?

I have written the following migration that works fine if run from the server/main.js file ...

import '../imports/startup/server';

import { Meteor } from 'meteor/meteor';
import { Migrations } from 'meteor/percolate:migrations';
import { Tags } from '../imports/api/collections/tags/tags.js';


Migrations.add({
  'version': 1,
  'name': "Add multi practice capability to user.",
  'up': function(){
    // the users are assumed to be the owner.
    
    const ownerTagId = Tags.findOne({name: "Practice Owner"})._id;
    if(!ownerTagId){
      return;
    }
    const search = { $and: [
      {practiceId: { $exists: true }},
      {practices: { $exists: false }}
    ]};

    let users = Meteor.users.find(search);
    users.forEach(user =>{
      const practice = {
        id: user.practiceId,
        tags: [ ownerTagId ],
        active: true
      };
      const practices = [practice];
      Meteor.users.update(user._id, {$set: {practices: practices}});
    });
  },
  'down': function(){
    const users =  Meteor.users.find();
    users.forEach((user)=>{
      Meteor.users.update(user._id, {
        $unset: { practices: "" } 
      });
    });
  }
});

Meteor.startup(()=>{
  Migrations.migrateTo(1);  
});

However I really wanted to have a separate migrations.js file or maybe even one per version to sit in the /imports/startup/server folder.

However when trying to run that way - despite having all imports working, I get an error like this:

W20161111-23:36:20.538(0)? (STDERR) TypeError: Cannot read property 'findOne' of undefined
W20161111-23:36:20.539(0)? (STDERR) at Object.Migrations.getControl (packages/percolatemigrations.js:269:33)
W20161111-23:36:20.539(0)? (STDERR) at Object.Migrations.migrateTo (packages/percolatemigrations.js:185:22)
W20161111-23:36:20.540(0)? (STDERR) at Object.Migrations.migrateTo (packages/percolate_migrations.js:167:10)
W20161111-23:36:20.540(0)? (STDERR) at meteorInstall.imports.startup.server.initializeApplication.js (imports/startup/server/initializeApplication.js:60:12)
W20161111-23:36:20.541(0)? (STDERR) at fileEvaluate (packages/modules-runtime.js:181:9)
W20161111-23:36:20.541(0)? (STDERR) at Module.require (packages/modules-runtime.js:106:16)
W20161111-23:36:20.541(0)? (STDERR) at Module.Mp.import (/home/hans/.meteor/packages/modules/.0.7.7.4uxp90++os+web.browser+web.cordova/npm/node_modules/reify/lib/runtime.js:70:16)
W20161111-23:36:20.542(0)? (STDERR) at meteorInstall.imports.startup.server.index.js (imports/startup/server/index.js:1:1)
W20161111-23:36:20.542(0)? (STDERR) at fileEvaluate (packages/modules-runtime.js:181:9)
W20161111-23:36:20.543(0)? (STDERR) at Module.require (packages/modules-runtime.js:106:16)
=> Exited with code: 1

It appears that Migrations._collecction is not defined in the getControl method.

Error: Created by is required

the migration:

Migrations.add({
    version: 7,
    up: function() {
      Tasks.find({taskValue: 0}).forEach(function (tsk) {
          var tv = taskValue(tsk.importance,  new Date(), tsk.deadline, "");
        ###  Tasks.update(tsk._id,  {$set: {taskValue: tv}}    );
      });
    },
});

a piece of my schema:

    taskValue: {
        type: Number,
        decimal: true,
    },

    createdBy: {
        type: String,
        autoValue: function(){
             return this.userId
        },
    },

Thank you for your work ! :-)

In Dr. Mongo we are prefixing all our collection with drmongo. string https://github.com/DrMongo/DrMongo/blob/master/lib/constants.js#L3

is it passible to customizing name of migration collection?

Note: I'm relatively new to Mongo coming from a MySQL background, so please correct me if I'm mistaken anywhere.

The Mongo FAQ on ACID states

MongoDB does not support multi-document transactions

Which leads me to the conclusion that while a migration is running other documents may be created or existing document may be updated. This can be a problem for long running migrations.

So in extending this package for my own application I'm writing a method, for lack of a better word, called "migrationate". It basically takes search parameters and an updater function.

• The search parameters locate non-migrated documents
• The updater function should accept a document from the search, and migrates it in the collection (Note: It must make changes that are detectable from the search parameters)

The search is re-run and documents are sent to the updater function until no documents are found.

There are other details and complexities I'm working through but the above is the gist of it. I think it'll lead to good performance / not lock up mongo, and help ensure every document is migrated.

Questions

1. Am I thinking correctly / in a mongo way?
2. Does this sound like something helpful to other people?
3. Presuming it's helpful, would it be better for me to integrate into this package and make a pull request, or make a separate package?

Could support be added for version strings with various formats? The most common is major.minor.increment. We could provide an option for which to use, or a comparator function to determine the correct ordering.

I followed the pattern that you used in the examples folder.

I have a migrations.js file in /server.

It's structured like this:

if(Meteor.isServer) {
  Migrations.add({
    version: 1,
    name: 'Replace username column on tasks with userId',
    up: function() {
      Tasks.forEach(function(task){
        Tasks.update({_id: task._id}, {$set: {
          ...
        }});
      });
    },
    down: function() {
      Tasks.forEach(function(task){
        Tasks.update({_id: task._id}, {$set: {
          ...
        }});
        Tasks.update({_id: task._id}, {$unset: {
          ...
        }});
      });
    }
  });

  // Add other migrations above here.

  Meteor.startup(function(){
    Migrations.migrateTo('latest');
  });

}

The problem is that every time I restart my server I get the following error:

[abc] /opt/foo/app/programs/server/node_modules/fibers/future.js:173
                        throw(ex);
                              ^
TypeError: Object [object Object] has no method 'forEach'
    at Migrations.add.up (app/server/migrations.js:6:13)
    at migrate (packages/percolate:migrations/migrations_server.js:142:1)
[104.236.142.118]     at Object.Migrations._migrateTo (packages/percolate:migrations/migrations_server.js:153:1)
    at Object.Migrations.migrateTo (packages/percolate:migrations/migrations_server.js:80:1)
    at app/server/migrations.js:30:16
    at /opt/foo/app/programs/server/boot.js:212:5
error: Forever detected script exited with code: 8
error: Script restart attempt #1
[104.236.142.118] Not migrating, control is locked.[104.236.142.118]

It's blowing up on calling Tasks.forEach.

I tried naming the file main.migrations.js in order to get it to load after everything else. But it didn't make a difference.

Did I define something wrong in my migration? Or am I calling it wrong in startup? Something else maybe?

i need to insert one collection data to another collection. how can i write migration script for this purpose. card payments and corporate customer payment are two collections.

corporate customer payment collection has below attributes customerID,outsTanding,date,amount,headOfficeId,insertDate

cards collection has below attributes
customerID,amount,card,packageId,cardId,classType,expireDate,headOfficeId,insertDate

i'm inserting cards collection following datas to corporate customer collection.
CustomerID -> card.customerID
outsTanding -> card.cartId
date -> card.insertDate
amount -> card.amount
headOfficeId -> card.headOfficeId

so how can i apply this change for old datas. how need i write meteor migration for up and down.
anyone who had this issue assist me
thanks

Since I'm no longer actively developing an app that uses migrations, I've lost touch with the codebase and the project. I'm looking for folks to take over development and maintenance, please raise your hand if you're interested.

I'm trying to revive an older Meteor project that I have. It's currently crashing on startup with this error:

W20160312-18:28:45.691(-8)? (STDERR) TypeError: Cannot call method 'findOne' of undefined
W20160312-18:28:45.691(-8)? (STDERR)     at Object.Migrations._getControl (packages/percolate_migrations/migrations_server.js:245:1)
W20160312-18:28:45.691(-8)? (STDERR)     at Object.Migrations._migrateTo (packages/percolate_migrations/migrations_server.js:161:1)
W20160312-18:28:45.691(-8)? (STDERR)     at Object.Migrations.migrateTo (packages/percolate_migrations/migrations_server.js:143:1)
...(redacted)

My database contains the control record:

meteor:PRIMARY> db.migrations.find()
{ "_id" : "control", "version" : 4, "locked" : false }

Not sure what's going on here. Apparently Migrations._collection is undefined at packages/percolate_migrations/migrations_server.js:245:1, but I can't see why.

I'm running Meteor 1.2.1

Everything seems to work fine for migrating to and from versions above 0. However when trying to migrate back down to 0 nothing seems to happen.
Using command:

Migrations.migrateTo(0);

It just stays on the current version it was already at and I do not get any errors.

Thanks for your help

#22 is necessary for this package to work within another package. Otherwise I just get ReferenceError: check is not defined when it tries to start up. Can we get a bump to 0.7.4 so this can be easily included in a package?

As https://github.com/percolatestudio/meteor-momentum/ does

Hi,

I just added this package to my app and it works fine locally. However, when I push it to my meteor.com staging site or my mup production site, the default iron router splash screen shows and I can't access any other routes. When I turn the package off and comment out my migrations the app works fine in production. I've checked both the logs for the staging and production sites and neither show any signs of what is going awry. They simple say:

[Mon May 04 2015 14:21:20 GMT+0000 (UTC)] NOTICE Starting application on port 17903
[Mon May 04 2015 14:21:20 GMT+0000 (UTC)] INFO STATUS null -> starting
[Mon May 04 2015 14:21:20 GMT+0000 (UTC)] INFO STATUS starting -> running
[Mon May 04 2015 14:21:22 GMT+0000 (UTC)] INFO Migrating from version 0 -> 1
[Mon May 04 2015 14:21:22 GMT+0000 (UTC)] INFO Running up() on version 1 (Adds user_email property
to entries)
[Mon May 04 2015 14:21:25 GMT+0000 (UTC)] INFO Finished migrating.
[Mon May 04 2015 14:21:25 GMT+0000 (UTC)] INFO Kadira: completed instrumenting the app
[Mon May 04 2015 14:21:26 GMT+0000 (UTC)] INFO Kadira: successfully authenticated

Any ideas as to what is happening?

What are the best ways of running the Migrations.migrateTo('latest'); in PROD. I know we cannot use meteor shell in PROD.

I want to be able to run Migrations.migrateTo('latest'); outside of the code preferably in some sort of deploy script, which would give me greater control when I an upgrading my meteor app.

I like how this package at least lets me not have migrations run until I've written the code for them. But is there a decent way to script migrations and have them get run by a deploy script? If you've done anything like this with the package, it'd be neat to have some examples in the README.

Baffled by this code not working. I've tested it from the mongo console which works, I've console logged to ensure I am in the up/down functions, but the update has no effect.

  version: 2,
  name: 'Add blah to Collection',
  up: function () {
    Collection.update({}, {$set:{blah:""}}, {multi:true});
  },
  down: function () {
    Collection.update({}, {$unset: {blah:1}}, {multi:true});
  }
});```

I think meteor-migrations could be significantly improved by removing the dependency on having a version number in each migration. Having a fixed version number makes migrations inflexible when working with a team.

For example, the case where you have a team of 3 people working on a product and the current migration is version 1 (or base). They all are working on new features that require database schema changes so they all make a new migration in their own branch and increment the version to 2. This breaks down when all the work is merged to master because the versions have to be re-ordered.

Ideally, this migration plugin would keep track of what migrations have already run (eg: by file name or another unique version name) and only run the migrations that have not executed yet (ie. keep a list of all successfully authenticated migrations). This solves the problem above and allows long lived feature branch to be merged in without change even if new migrations were created and added to master after the feature branch was created.

import { Songs } from '../imports/api/songs.js';

// omit the version 1

Migrations.add({
  version: 2,
  name: 'update the songId',
  up() {
    // This is how to get access to the raw MongoDB node collection that the Meteor server collection wraps
    const batch = Songs.rawCollection().initializeUnorderedBulkOp();
    // Mongo throws an error if we execute a batch operation without actual operations, e.g. when Lists was empty.
    let hasUpdates = false;

    let count = 0;
    Songs.find({}).forEach(function(song) {
      console.log(count);
      batch.find({ _id: song._id }).updateOne({ $set: { songId: count } });
      hasUpdates = true;
      count++;
    }, function(err) {
      console.info(err);
      console.log('seetting songId finished!');
    });

    if (hasUpdates) {
      // We need to wrap the async function to get a synchronous API that migrations expects
      const execute = Meteor.wrapAsync(batch.execute, batch);
      return execute();
    }
    return true;
  },
  down() {
    // This is how to get access to the raw MongoDB node collection that the Meteor server collection wraps
    const batch = Songs.rawCollection().initializeUnorderedBulkOp();
    // Mongo throws an error if we execute a batch operation without actual operations, e.g. when Lists was empty.
    let hasUpdates = false;

    Songs.find({}).forEach(function(song) {
      batch.find({ _id: song._id }).updateOne({ $unset: { songId: '' } });
      hasUpdates = true;
    });

    if (hasUpdates) {
      // We need to wrap the async function to get a synchronous API that migrations expects
      const execute = Meteor.wrapAsync(batch.execute, batch);
      return execute();
    }
    return true;
  },
});

I'm tried to think about this problem.
I could not succed add the songId to my songs collections's document after running it.Have i did somthing wrong? Guys,Help me.

I made a reactive admin panel for this migrations package. I was wondering if others have interest and hence if I should share it.

How it works
It's a simple reactive bootstrap interface that migrates between migrations and shows basic info all in a table.

Screenshot
https://i.imgur.com/Tlw9OAC.png

Screencast
http://youtu.be/X76OronNzRk

Update
Now with locked status and ability to unlock as shown @ https://i.imgur.com/51Cti2A.png

Packaging
I figure this should probably be it's own package with the meteor-migrations package as a dependency instead of fully integrating into the meteor-migrations package. Though if you want to integrate that's fine with me of course too.

I haven't published a meteor package before. Though since I'm using packages as my application organization structure it probably won't be too hard for me to isolate this from my application code. And a good learning for me experience too.

First of all, I love having this package and we couldn't live without it, thank you! :)

So after upgrading to Meteor 1.2, I'm seeing the following fatal error when trying to run Migrations.migrateTo('latest');, regardless of the migration actually defined:

W20151005-01:08:35.797(2)? (STDERR) 
W20151005-01:08:35.797(2)? (STDERR) /home/farlion/.meteor/packages/meteor-tool/.1.1.9.7in8iw++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:245
W20151005-01:08:35.797(2)? (STDERR)                         throw(ex);
W20151005-01:08:35.797(2)? (STDERR)                               ^
W20151005-01:08:35.803(2)? (STDERR) TypeError: Cannot call method 'info' of undefined
W20151005-01:08:35.803(2)? (STDERR)     at Object.Migrations._migrateTo (meteor://💻app/packages/percolate_migrations/migrations_server.js:161:1)
W20151005-01:08:35.803(2)? (STDERR)     at Object.Migrations.migrateTo (meteor://💻app/packages/percolate_migrations/migrations_server.js:139:1)
W20151005-01:08:35.803(2)? (STDERR)     at meteor://💻app/server/migrations/migrate.js:3:1
W20151005-01:08:35.803(2)? (STDERR)     at meteor://💻app/server/migrations/migrate.js:4:1
W20151005-01:08:35.803(2)? (STDERR)     at /home/farlion/code/dropz/platform/dropz/web/.meteor/local/build/programs/server/boot.js:242:10
W20151005-01:08:35.803(2)? (STDERR)     at Array.forEach (native)
W20151005-01:08:35.804(2)? (STDERR)     at Function._.each._.forEach (/home/farlion/.meteor/packages/meteor-tool/.1.1.9.7in8iw++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/underscore/underscore.js:79:11)
W20151005-01:08:35.804(2)? (STDERR)     at /home/farlion/code/dropz/platform/dropz/web/.meteor/local/build/programs/server/boot.js:137:5
=> Exited with code: 8
=> Your application is crashing. Waiting for file change.

The following line in migrations_server.js is throwing the error, it seems that the log variable is undefined for some reasons:

161 log.info('Not migrating, control is locked.');

Here's the output of meteor list | grep "log\|migrat" for that project

logging                               1.0.8  Logging facility.
percolate:migrations                  0.9.6  Define and run db migrations.

Any ideas what might be going on?

Well, they do work if you do the following:

package.js

change this

Package.on_test(function (api) {
api.use('percolatestudio-migrations'); <<------
api.add_files('migrations_tests.js', ['server']);
});

to this

Package.on_test(function (api) {
api.use(['percolatestudio-migrations', 'tinytest'], 'server'); <<------
api.add_files('migrations_tests.js', ['server']);
});

and in migrations_server.js

change

//reset (mainly intended for tests)
Migrations._reset = function() {
this._list = [{version: 0, up: function(){}}];
this._collection.remove({}, false); <<-------
}

to this

//reset (mainly intended for tests)
Migrations._reset = function() {
this._list = [{version: 0, up: function(){}}];
this._collection.remove({}); <<-------
}

I'm getting a ReferenceError: Collection is not defined no idea what's casing the error

Migrations.add({
    version: 1,
    name: 'Adds email slug to topics in the db.',
    //code to migrate up to version 0.1
    up: function() {
        Topics.update({}, {$setOnInsert:{emailSlug:""}}, {multi:true});
    },
    //code to migrate down to version 0
    down: function() {
    }
});

Is there some other configuration I need to change?

I'm trying to upgrade my app to 1.3.2.4 and move all my server code into my /imports folder, but I'm running into trouble with this package. When I include Migrations.migrateTo('latest'); inside of /imports/server/startup, I get the error "Cannot call method 'findOne' of undefined".

I've tried with and without import { Migrations } from 'meteor/percolate:migrations'; at the top of the file. It doesn't look like Migrations is an exported variable in the package, so I would assume this would remain available as a global. Perhaps this folder loads before the package Global is available?

Confused 😕

I would like to see a context object (probably the whole migration object, honestly) provided as an argument to the invocation of up and down.

This can be beneficial for several reasons:

• To avoid copy/paste errors

  • The first Migration I wrote had a copy/paste error in the down function that made it asymmetrical with the up function: a query field value that was not updated.

  • This is easily solved by centralizing important shared values into a new data (or some other name) property to be added onto the Migration object [alongside the existing name/version/up/down properties], which would then be included as an argument (or sub-argument) to the actual invocations of the up and down functions.

  • Example:

    Migrations.add({
      name: "My Migration",
      version: 1,
      data: {
        mainDocId: "control"
      },
      up: function(migration) {
        SomeCollection.insert({ _id: migration.data.mainDocId, someValue: "important" });
      },
      down: function(migration) {
        SomeCollection.remove({ _id: migration.data.mainDocId });
      }
    });

• For logging purposes. Example:

  Migrations.add({
    name: "My Migration",
    version: 1,
    up: function(migration) {
      ExternalLoggingService.log("Running migration `up` to version: " + migration.version);
      // ...
    },
    down: function(migration) {
      ExternalLoggingService.log("Running migration `down` from version: " + migration.version);
      // ...
    }
  });

• For auditing purposes. Example:

  Migrations.add({
    name: "My Migration",
    version: 1,
    up: function(migration) {
      SomeCollection.insert({
        someValue: "important",
        created: {
          by: "Migrations[" + migration.version + "]",
          at: Date.now()
        }
      });
    },
    down: function(migration) {
      ExternalLoggingService.log("Running migration `down` from version: " + migration.version);
    }
  });

• For manual error handling purposes IF AND ONLY IF you know your up AND downare 100% symmetrically recoverable (although this is also a per-Migration option I'd like to see implemented separately in this package... but I'll open a separate issue for that). Example:

  Migrations.add({
    name: "My Migration",
    version: 1,
    up: function(migration, isRecovery) {
      try {
        SomeCollection.insert({ _id: "featureA", someValue: "important" });
      }
      catch (err) {
        if (!isRecovery) {  // avoid infinite loop
          try {
            migration.down(migration, true);
            throw new Error("Migrations[" + migration.version + "] up failed but was " +
              "successfully recovered.\n" + err);
          }
          catch (recoveryErr) {
            throw new Error("Migrations[" + migration.version + "] up failed AND failed to " +
              "recover!\n" + err + "\n" + recoveryErr);
          }
        }
      }
    },
    down: function(migration, isRecovery) {
      try {
        SomeCollection.remove({ _id: "featureA" });
      }
      catch (err) {
        if (!isRecovery) {  // avoid infinite loop
          try {
            migration.up(migration, true);
            throw new Error("Migrations[" + migration.version + "] down failed but was " +
              "successfully recovered.\n" + err);
          }
          catch (recoveryErr) {
            throw new Error("Migrations[" + migration.version + "] down failed AND failed to " +
              "recover!\n" + err + "\n" + recoveryErr);
          }
        }
      }
    }
  });

I'd like logging to be configurable, similar to what you did with synced-cron. Thoughts?

setLocked should make sure a document was actually updated. If not, another app running the same code connecting to the same database may have already run the migration.

The README regarding logging indicates...

Migrations.add({ name: 'Test Job', ... });
Migrations.start();

yet the method start() does not exist and add must contain an "up" function. With some logical deduction, I assume those two lines are intended to create errors that will log to the console... nope, just crashes my app instead. Hmm.

Consider editing the README to clarify this.

First Thanks for the Great work @zol and all Maintainers 👍
I have a very specific use case. I elaborate it with Example
let's say I added migration 8 in which I adding a new key to documents of a collection if not exists. suddenly my manager said we don't need it :( . so ok right I downgraded to previous version 7 and all is well. Now the issue is am going to add v9. but whenever I migration to v9 or latest it first run the v8 and then v9 so how can I skip the v8 and directly migrate to v9
Note :- I know that I just overwrite the v9 functionality within v8 but I have multiple environments so I don't want to interfere the old one I just want to add new migration version
Any help will be greatly appreciated

Release tags seem to be missing for the last few version of this package?

This is fairly simple, but it would be nice to see which version you are on. Could be useful if you are tinkering around. Something like -> Migrations.currentVersion()

Could I user version of migration with stirng like '0.1.5'?