Giter Club home page Giter Club logo

odoo-module-migrator's Introduction

License: AGPL-3

Python support: 3.6

https://app.travis-ci.com/OCA/odoo-module-migrator.svg?branch=master

odoo-module-migrator

odoo-module-migrator is a python3 library that allows you to automatically migrate module code to make it compatible with newer Odoo version. for exemple:

  • renaming __openerp__.py file into __manifest__.py
  • removing # -*- encoding: utf-8 -*- since V11.0
  • replacing openerp import by odoo import
  • removing migrations folders
  • changing <act_window> to <record model="ir.actions.window">
  • ...

This library will so:

  • (optionnaly) get commits from the old branch (if format-patch is enabled)
  • apply automatically changes. (renaming, replacing, etc.)
  • commit your changes.
  • Display warnings or errors in log if your code belong obsolete code patterns.

This project is about migrating code. If you're looking for database data migration between Odoo versions, take a look at the https://github.com/OCA/OpenUpgrade project.

INFO log

It mentions that the lib automatically changed something. A priori you have nothing to do. For example, for a migration from 8.0 to a more recent version:

12:38:54 INFO Renaming file: '/my_module/__openerp__.py' by '/my_module/__manifest__.py'

WARNING log

It mentions that you should check something. There is maybe something to do to make the module working. For exemple:

19:37:55 WARNING Replaced dependency of 'account_analytic_analysis' by 'contract' (Moved to OCA/contract)

ERROR log

It mentions that you should change something in your code. It not, the module will not work for sure. (not installable, or generating error during the execution)

For example, if you have a 8.0 module that depends on 'account_anglo_saxon', that disappeared in more recent version, the following log will be displayed

12:38:54 ERROR Depends on removed module 'account_anglo_saxon'

Development and improvment

If you want to improve or complete this library, please read the DEVELOP.rst file and the 'Roadmap / Know issues' sections.

Installation

pip3 install odoo-module-migrator

Usage

Using Format Patch command

(Recommanded by the OCA)

If you want to migrate an Odoo module from a version 8.0 to 12.0, for exemple the module pos_order_pricelist_change in the OCA "pos" repository.

git clone https://github.com/OCA/pos -b 12.0
cd pos
odoo-module-migrate
    --directory             ./
    --modules               pos_order_pricelist_change
    --init-version-name     8.0
    --target-version-name  12.0
    --format-patch

Without format Patch command

(Mainly for your custom modules)

if you have created a new branch (for exemple 12.0) based on your 10.0 branch you can run the following command

odoo-module-migrate
    --directory             /path/to/repository
    --modules               module_1,module_2,module_3
    --init-version-name     10.0
    --target-version-name   12.0

This tools will operate the changes for each module.

Available arguments

name shortcut Options description
--directory -d Default: ./ Local folder that belongs the module(s) to migrate.
--init-version-name -i Required Initial version of your module(s) you want to migrate.
--target-version-name -t Default: the laster odoo version Final Version you want to migrate.
--modules -m Default: All modules present in the directory Module(s) to migrate. Note if format-patch option is enabled, you have to provide only one module.
--format-patch -fp disabled by default Recover code, using git format-patch command.
--remote-name -rn Default: origin Name of the main remote, used by format-patch command.
--log-level -ll Default: INFO Possible value: DEBUG, INFO, WARNING, etc.
--log-path -lp Default: False If set, the logs will not be displayed at screen, but stored in a file
--no-commit -nc Default: commit If set the library will not git add and git commit changes.

Roadmap / Know issues

  • replacement of tag <openerp><data> by <odoo> will fail in the case where there are many <data> occurency. We could fix that, using lxml lib instead of regular expression.
  • Add a call to pre-commit run -a, if pre-commit is present in the repository.

Changes

0.3.2 (December 04, 2019)

  • [REM] Remove black call. (Add call to more generic tool pre-commit in the roadmap)
  • [IMP] Add --no-verify option in git commit to avoid to fail if pre-commit is present
  • [REF] Refactor _execute_shell function

0.2.0 (October 13, 2019)

  • Second release

0.1.4 (October 12, 2019)

  • Test

[ADD] test

  • framework

[ADD] --file-path option. [ADD] _DEPRECATED_MODULES syntax.

  • migration script

[FIX] Incorrect syntax of regular expression, to remove python 2 header [IMP] first release of all the steps from 8.0 to 13.0

0.1.3 (October 11, 2019)

  • framework

[ADD] --no-commit option that disable git add and git commit calls [FIX] do not commit many times if migration has many steps. [REF] remove useless commented code [REF] create _commit_changes() and _replace_in_file() functions

  • Meta

[FIX] github url of the project in setup.py [ADD] Travis file + links to coveralls [ADD] test_requirements.txt

  • migration script

[ADD] 12.0 to 13.0 and add a warning if reference to web_settings_dashboard are found. cortesy @yelizariev [ADD] bump version in manifest file [ADD] set installable to True

0.1.2 (October 10, 2019)

  • First release

Credits

Authors

Contributors

odoo-module-migrator's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

odoo-module-migrator's Issues

Refactoring: Scripts converted into classes

In this PR I propossed the refactoring to

* Module script is converted to instance of class
* It is easy to convert, only need move the current content of script to __init__ class
* All script have the run base function that operates with protected properties _TEXT_REPLACES, ERRORS, ETC. `def run(self, module_path, manifest_path, module_name, migration_steps, directory_path, commit_enabled)`
* Also have `def process_file(self, root, filename, extension, file_renames, directory_path, commit_enabled)` to process file one by one
* If a script fail you can know so easy. Only need review the stack trace.
* On this refactoring you could delete global functions. Only you need inherit run method and overwrite it
* I have maintain global_funtions

Since I intend to continue working on the tool and I prefer that the author give me his opinion, I am going to keep a branch in my fork and I will continue doing PR in the master of this repository. I await the code review. If they give me OK, when the time comes I will do a PR with the refactoring

[RFC] renaming the library

Ref of the thread :

https://odoo-community.org/groups/contributors-15/contributors-147949?mode=thread&date_begin=&date_end=

@pedrobaeza said :

I think we can move it to OCA, but a better name would be odoo-oca-module-migrator, as it avoids confusion with the migration process through OpenUpgrade or similar, and it also ties it to OCA specific process.

My Point of view :
OK to rename the lib. I'd prefer odoo-module-migrator because I'm not sure that adding oca is great, because this lib can be used for any modules. (not only OCA modules). But not a blocking point for my side.

Should i rename also the command ?

for the time being the installation is

pip install odoo-migrate

and the command is

odoo-migrate args....

thanks for your answers.

convert Odoo domain to python expression in xml data.

Is your feature request related to a problem?
If so, please provide clear and concise description of what the problem is.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context about the feature request here.

can ref. to odoo/odoo#104741 and @飞 https://github.com/Gorash/odoo-view-pocalypse-convert

requirements.txt lacks of minimum compatible version of lxml dependency

The requirements.txt does not dictate what version of the listed packages should be installed.

However, looking at lxml specifically, this project relies on lxml.etree providing an indent method, see for example:

This method however has been added to lxml only from version 4.5.0 (see Changelog)

To Reproduce

Latest odoo-module-migrator binary (0.4.0). Install a lxml version prior to 4.5.0 (I was using 4.2.3 but the exact version is not relevant).

While executing a migration:

Traceback (most recent call last):
  File "/home/are/.pyenv/versions/3.7.13/bin/odoo-module-migrate", line 8, in <module>
    sys.exit(main())
  File "/home/are/.pyenv/versions/3.7.13/lib/python3.7/site-packages/odoo_module_migrate/__main__.py", line 144, in main
    migration.run()
  File "/home/are/.pyenv/versions/3.7.13/lib/python3.7/site-packages/odoo_module_migrate/migration.py", line 221, in run
    module_migration.run()
  File "/home/are/.pyenv/versions/3.7.13/lib/python3.7/site-packages/odoo_module_migrate/module_migration.py", line 36, in run
    self._migration._commit_enabled
  File "/home/are/.pyenv/versions/3.7.13/lib/python3.7/site-packages/odoo_module_migrate/base_migration_script.py", line 152, in run
    tools=tools,
  File "/home/are/.pyenv/versions/3.7.13/lib/python3.7/site-packages/odoo_module_migrate/migration_scripts/migrate_130_140.py", line 144, in reformat_deprecated_tags
    reformatted_file = _reformat_file(file_path)
  File "/home/are/.pyenv/versions/3.7.13/lib/python3.7/site-packages/odoo_module_migrate/migration_scripts/migrate_130_140.py", line 106, in _reformat_file
    et.indent(tag, space=indent, level=1)
AttributeError: module 'lxml.etree' has no attribute 'indent'

I cannot provide the view that triggers this error because the module I was migrating is private, however the issue I'm pretty sure it does not depends from a particular view definition.

I suggest to add minimum required versions to the requirements.txt. That should be trivial to implement, and it would make the project more robust.

Affected versions:

0.4.0

Additional context

Tested on Linux (Ubuntu 18.04), Python 3.6.15 and 3.7.13.

Improving the library

Add error if the following patterns are found in python file, when migrating from 10.0 to 11.0 :

  • iteritems()

TODO : complete.

More switches to disable unneeded actions (version change, black, etc)

Hi,

Thanks for this interesting project) It is looking as a nice way to automate module migration routines, but i have the question.

Is it possible to avoid applying Black to code? The reason is, i would like to try to use this project in forward-ports workflow, but we try to do minimum changes during forwardport. So, Black may introduce a lot of unneeded changes in git. Also, forward-ports are repeatable actions, because 90% of our modules we develop on 11.0 branch and then periodically forward-port changes to 12.0 and 13.0 branches.

My expectations after running module migration is following:

  • No module version changed (in our forkflow it will be done by odoo-helper-scripts
  • No enforced formatting of code
  • Only changes related to migration of module to newer odoo serie have to be applied.

Parsing large XML takes infinite time

When migration is done from V10 to V11 it tires to search for color attribute inside tree view it tries to parse the xml and when it tries to parse xml in many cases it keeps parsing infinitely.

I can attach screenshots if required.
To be more exact replacing these lines "<tree(\n|.|\t)color="[^>]":
"color attribute is deprecated in tree view. Use decoration- instead.",
in file migrate_100_110.py skips the parsing.

Let me know if I need to put in more steps for replication of same.

PRs conflicts

Design of the tool doesn't fit well for contributing: if there are 5 pull request about the same migration (e.g 13 to 14) and all them target the same place of code, then merging one of them leads to conflicts in all others. Well, fixing a conflict not that difficult, but

  • We need to wait when the author has time to do it
  • Once it's done, another pull request may be merged, which means that the author has to fix conflict again
  • You cannot use git-aggregate tool to apply your unmerged PRs

To resolve this, I would suggest to collect rules in separate files, rather than in a dictionary like _TEXT_ERRORS. It may have the following structure:

/migration_scripts/13_14/text_errors/some_name.json
/migration_scripts/13_14/text_errors/another_name.json

Incoporate odoo-pylint conformance as always?

As we are dealing with proper code tree transformations, it might be considerable to incorporate odoo-pylint conformance scripts in order to facilitate to move legacy code base towards an OCA compliant state.

As we have pre-commit checks in place, since v13.0.

Autodetect modules

If --modules target omitted, auto scan the target directory recursively.

Can't fetch previous branch in shallow repository

When trying to migrate with the --format-patch option, you will sometimes get an error when trying to fetch the commits from the previous version due to a big difference in the commit history:

fatal: bad revision 'origin/14.0..origin/13.0'

That will cause the script to fail:

➤ odoo-module-migrate -m $MODULE --init-version-name 13.0 --target-version-name 14.0 --no-commit --format-patch
10:39:37   INFO        Creating new branch '14.0-mig-payment_redsys' ...
Switched to a new branch '14.0-mig-payment_redsys'
fatal: bad revision 'origin/14.0..origin/13.0'
10:39:37   WARNING     No valid module found for 'payment_redsys' in the directory '/home/jota/Documents/Tecnativa/plantillas/devel-14.0/odoo/custom/src/l10n-spain'
Traceback (most recent call last):
  File "/home/jota/.local/bin/odoo-module-migrate", line 8, in <module>
    sys.exit(main())
  File "/home/jota/.local/pipx/venvs/odoo-module-migrator/lib64/python3.9/site-packages/odoo_module_migrate/__main__.py", line 124, in main
    migration = Migration(
  File "/home/jota/.local/pipx/venvs/odoo-module-migrator/lib64/python3.9/site-packages/odoo_module_migrate/migration.py", line 87, in __init__
    raise ConfigException("No modules found to migrate. Exiting.")
odoo_module_migrate.exception.ConfigException: No modules found to migrate. Exiting.

This script could automatically detect that and perform a git fetch [--unshallow] previous to the format patch.

When you do it manually, it works:

jota@localhost:~/D/T/p/d/o/c/s/l10n-spain|14.0✓
➤ git fetch origin
remote: Enumerating objects: 56197, done.
remote: Counting objects: 100% (56197/56197), done.
remote: Compressing objects: 100% (17352/17352), done.
remote: Total 56197 (delta 37907), reused 55972 (delta 37683), pack-reused 0
Receiving objects: 100% (56197/56197), 71.00 MiB | 1.86 MiB/s, done.
Resolving deltas: 100% (37907/37907), done.
From https://github.com/OCA/l10n-spain
 * [new branch]        10.0       -> origin/10.0
 * [new branch]        11.0       -> origin/11.0
 * [new branch]        12.0       -> origin/12.0
 * [new branch]        13.0       -> origin/13.0
 * [new branch]        4.2        -> origin/4.2
 * [new branch]        5.0        -> origin/5.0
 * [new branch]        6.0        -> origin/6.0
 * [new branch]        6.1        -> origin/6.1
 * [new branch]        7.0        -> origin/7.0
 * [new branch]        8.0        -> origin/8.0
 * [new branch]        9.0        -> origin/9.0
jota@localhost:~/D/T/p/d/o/c/s/l10n-spain|14.0✓
➤ odoo-module-migrate -m $MODULE --init-version-name 13.0 --target-version-name 14.0 --no-commit --format-patch
10:48:14   INFO        Creating new branch '14.0-mig-payment_redsys' ...
Switched to a new branch '14.0-mig-payment_redsys'
10:48:14   INFO        [payment_redsys] Running migration from 13.0 to 14.0
10:48:14   INFO        Bump version to 14.0.1.0.0

Replacement of read_group by _read_group in v16 to v17 Migration is not actually expected

Upon reviewing the functionalities of both methods as per the latest Odoo documentation, it appears that this change might not be accurate due to the distinct purposes and functionalities that each method serves.

  1. Method Signatures and Purposes:
  • Model._read_group: This method is primarily designed for getting fields aggregations, specified by aggregates, and grouped by the given groupby fields where records are filtered by the domain. It supports advanced aggregations and is tailored for more complex group-by queries.

  • Model.read_group: This method focuses on retrieving a list of records in a list view, grouped by specified groupby fields. It is designed to be used for generating grouped views in the UI, supporting a simpler and more direct way of aggregating data for presentation.

  1. Functional Differences:
  • Parameters and Return Types: The Model._read_group method allows for a detailed specification of aggregates and having clauses, catering to complex SQL-like aggregations. In contrast, Model.read_group is structured around the list view's needs, providing a simpler interface for aggregating and displaying data.

  • Use Cases: The usage of _read_group is more aligned with backend data processing and complex data aggregation scenarios. On the other hand, read_group is tailored towards frontend data presentation, particularly in grouping data for display in Odoo's list views.

Given these distinctions, replacing read_group by _read_group can potentially disrupt the intended functionality of modules that rely on these methods for specific purposes. The direct replacement overlooks the nuanced differences in their applications, leading to possible issues in data presentation and aggregation in migrated modules.

[IMPORTANT] renaming done. (odoo-migrate -> odoo-module-migrator)

Hi all,

For your information, the library name, the command and the main folder has been renamed.
so pypi package is now pip install odoo-module-migrator

please update your code, cloning a fresh repository to avoid conflicts.

kind regards.

Note : I deleted on pypi, obsolete package.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.