Giter Club home page Giter Club logo

mergedeep's Introduction

PyPi release PyPi versions Downloads Conda Version Conda Downloads Documentation Status

A deep merge function for ๐Ÿ.

Check out the mergedeep docs

Installation

$ pip install mergedeep

Usage

merge(destination: MutableMapping, *sources: Mapping, strategy: Strategy = Strategy.REPLACE) -> MutableMapping

Deep merge without mutating the source dicts.

from mergedeep import merge

a = {"keyA": 1}
b = {"keyB": {"sub1": 10}}
c = {"keyB": {"sub2": 20}}

merged = merge({}, a, b, c) 

print(merged)
# {"keyA": 1, "keyB": {"sub1": 10, "sub2": 20}}

Deep merge into an existing dict.

from mergedeep import merge

a = {"keyA": 1}
b = {"keyB": {"sub1": 10}}
c = {"keyB": {"sub2": 20}}

merge(a, b, c) 

print(a)
# {"keyA": 1, "keyB": {"sub1": 10, "sub2": 20}}

Merge strategies:

  1. Replace (default)

Strategy.REPLACE

# When `destination` and `source` keys are the same, replace the `destination` value with one from `source` (default).

# Note: with multiple sources, the `last` (i.e. rightmost) source value will be what appears in the merged result. 

from mergedeep import merge, Strategy

dst = {"key": [1, 2]}
src = {"key": [3, 4]}

merge(dst, src, strategy=Strategy.REPLACE) 
# same as: merge(dst, src)

print(dst)
# {"key": [3, 4]}
  1. Additive

Strategy.ADDITIVE

# When `destination` and `source` values are both the same additive collection type, extend `destination` by adding values from `source`.
# Additive collection types include: `list`, `tuple`, `set`, and `Counter`

# Note: if the values are not additive collections of the same type, then fallback to a `REPLACE` merge.

from mergedeep import merge, Strategy

dst = {"key": [1, 2], "count": Counter({"a": 1, "b": 1})}
src = {"key": [3, 4], "count": Counter({"a": 1, "c": 1})}

merge(dst, src, strategy=Strategy.ADDITIVE) 

print(dst)
# {"key": [1, 2, 3, 4], "count": Counter({"a": 2, "b": 1, "c": 1})}
  1. Typesafe replace

Strategy.TYPESAFE_REPLACE or Strategy.TYPESAFE

# When `destination` and `source` values are of different types, raise `TypeError`. Otherwise, perform a `REPLACE` merge.

from mergedeep import merge, Strategy

dst = {"key": [1, 2]}
src = {"key": {3, 4}}

merge(dst, src, strategy=Strategy.TYPESAFE_REPLACE) # same as: `Strategy.TYPESAFE`  
# TypeError: destination type: <class 'list'> differs from source type: <class 'set'> for key: "key"
  1. Typesafe additive

Strategy.TYPESAFE_ADDITIVE

# When `destination` and `source` values are of different types, raise `TypeError`. Otherwise, perform a `ADDITIVE` merge.

from mergedeep import merge, Strategy

dst = {"key": [1, 2]}
src = {"key": {3, 4}}

merge(dst, src, strategy=Strategy.TYPESAFE_ADDITIVE) 
# TypeError: destination type: <class 'list'> differs from source type: <class 'set'> for key: "key"

License

MIT ยฉ Travis Clarke

mergedeep's People

Contributors

clarketm avatar dependabot[bot] avatar tdegeus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

mergedeep's Issues

1.3.4: sphinx 4.x warnings

Looks like latest sphinx shows some warning

+ /usr/bin/python3 setup.py build_sphinx -b man --build-dir build/sphinx
Warning: 'classifiers' should be a list, got type 'tuple'
running build_sphinx
Running Sphinx v4.1.2
making output directory... done
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 1 added, 0 changed, 0 removed
reading sources... [100%] index
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:3: WARNING: Unexpected indentation.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:1: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:1: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:6: WARNING: Explicit markup ends without a blank line; unexpected unindent.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:6: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:6: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:10: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:10: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:22: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:22: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:40: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:40: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:49: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:49: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:53: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:53: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:63: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:63: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:73: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:86: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:86: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:94: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:107: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:107: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:115: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:123: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:123: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:131: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:139: WARNING: Inline literal start-string without end-string.
/home/tkloczko/rpmbuild/BUILD/mergedeep-1.3.4/docs/source/index.md:139: WARNING: Inline interpreted text or phrase reference start-string without end-string.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... mergedeep.1 { } done
build succeeded, 30 warnings.

The manual pages are in build/sphinx/man.

Additive merge for dict with list values?

I have these dictionaries as a sample:

Y1 = {'migration': {'options': {'install_command': 'odoo2'},
  'versions': [{'version': 'setup',
    'operations': {'pre': ["echo 'pre-operation'"]},
    'addons': {'install': ['crm'], 'upgrade': ['note']}},
   {'version': '0.1.0',
    'operations': {'post': ["echo 'post-operation'"]},
    'addons': {'install': ['web', 'contacts']}}]}}
Y2 = {'migration': {'options': {'install_command': 'odoo'},
  'versions': [{'version': 'setup',
    'operations': {'pre': ["echo 'pre-operation-2'", "echo 'pre-operation'"]},
    'addons': {'install': ['account', 'crm'], 'upgrade': ['note', 'hr']}},
   {'version': '0.1.0',
    'operations': {'post': ["echo 'post-operation'",
      "echo 'post-operation-2'"]},
    'addons': {'install': ['purchase', 'sale']}}]}}
Y3 = {'migration': {'versions': [{'version': '0.2.0',
    'addons': {'install': ['mrp', 'stock']}}]}}

I merge it like this:

from mergedeep import merge, Strategy

d = merge({}, Y1, Y2, Y2, strategy=Strategy.ADDITIVE)
print(d)

And get this output:

{'migration': {'options': {'install_command': 'odoo'}, 'versions': [{'version': 'setup', 'operations': {'pre': ["echo 'pre-operation'"]}, 'addons': {'install': ['crm'], 'upgrade': ['note']}}, {'version': '0.1.0', 'operations': {'post': ["echo 'post-operation'"]}, 'addons': {'install': ['web', 'contacts']}}, {'version': 'setup', 'operations': {'pre': ["echo 'pre-operation-2'", "echo 'pre-operation'"]}, 'addons': {'install': ['account', 'crm'], 'upgrade': ['note', 'hr']}}, {'version': '0.1.0', 'operations': {'post': ["echo 'post-operation'", "echo 'post-operation-2'"]}, 'addons': {'install': ['purchase', 'sale']}}, {'version': 'setup', 'operations': {'pre': ["echo 'pre-operation-2'", "echo 'pre-operation'"]}, 'addons': {'install': ['account', 'crm'], 'upgrade': ['note', 'hr']}}, {'version': '0.1.0', 'operations': {'post': ["echo 'post-operation'", "echo 'post-operation-2'"]}, 'addons': {'install': ['purchase', 'sale']}}]}}

Yet I expect lists to be appended. For example from dicts Y1, Y2, install key in setup version, is ['crm'] and ['account', 'crm'], yet first one is kept values from Y2 is ignored. I expect it to become ['crm, 'account', 'crm']

Is my assumption is incorrect and mergedeep works as intended?

Issues with deepcopy

Hello,
I have some issues with deepcopy, if objects are not pickable, e.g.

Traceback (most recent call last):
  File "mappings.py", line 45, in merge_mapping
    return dict(merge(*(p.dict() for p in outputs_sequence), strategy=Strategy.ADDITIVE))
  File "/home/user/.virtualenvs/dt_launcher/lib/python3.8/site-packages/mergedeep/mergedeep.py", line 100, in merge
    return reduce(partial(_deepmerge, strategy=strategy), sources, destination)
  File "/home/user/.virtualenvs/dt_launcher/lib/python3.8/site-packages/mergedeep/mergedeep.py", line 87, in _deepmerge
    dst[key] = deepcopy(src[key])
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle '_thread.lock' object

I see two ways to approach that:

  • Use "normal" copy, i.e. target = source, which afaik uses references as default
  • Maybe add a parameter copy_fun with can default do deepcopy but can be overwritten by a function which uses shallow copy.

Support for Arrays

Are there any plans for support for complex arrays (ex: list of dict objects)?

Types are not used after installation of the package

No types for the package found:

>> ls -lA /venv/lib/python3.10/site-packages/mergedeep*

/venv/lib/python3.10/site-packages/mergedeep:
total 28K
-rw-r--r-- 1 user group  104 Dec 20 11:33 __init__.py
-rw-r--r-- 1 user group 4.3K Dec 20 11:33 mergedeep.py
-rw-r--r-- 1 user group  14K Dec 20 11:33 test_mergedeep.py

/venv/lib/python3.10/site-packages/mergedeep-1.3.4.dist-info:
total 28K
-rw-r--r-- 1 user group    4 Dec 20 11:33 INSTALLER
-rw-r--r-- 1 user group 1.2K Dec 20 11:33 LICENSE
-rw-r--r-- 1 user group 4.3K Dec 20 11:33 METADATA
-rw-r--r-- 1 user group  982 Dec 20 11:33 RECORD
-rw-r--r-- 1 user group    0 Dec 20 11:33 REQUESTED
-rw-r--r-- 1 user group   92 Dec 20 11:33 WHEEL
-rw-r--r-- 1 user group   10 Dec 20 11:33 top_level.txt

The `typesafe` merge strategy should support both replace and additive.

By default (assuming the source and target are of the same type), the typesafe strategy merges by using the replace strategy. It may be desirable to some users to also support an additive merge. The benefit of this over using the additive merge strategy directly is that the user gets both additive merging and type checking.

_handle_merge[Strategy.REPLACE](destination, source, key)

Proposal:

  1. Add Strategy.TYPESAFE_REPLACE and alias it to Strategy.TYPESAFE for backward compatibility.
  2. Add Strategy.TYPESAFE_ADDITIVE

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.