Giter Club home page Giter Club logo

pyecoregen's Introduction

PyEcore: A Pythonic Implementation of the Eclipse Modeling Framework

pypi-version master-build coverage code-quality license

PyEcore is a Model Driven Engineering (MDE) framework written for Python. Precisely, it is an implementation of EMF/Ecore for Python, and it tries to give an API which is compatible with the original EMF Java implementation.

PyEcore allows you to handle models and metamodels (structured data model), and gives the key you need for building MDE-based tools and other applications based on a structured data model. It supports out-of-the-box:

  • Data inheritance,
  • Two-ways relationship management (opposite references),
  • XMI (de)serialization,
  • JSON (de)serialization,
  • Notification system,
  • Reflexive API...

Let's see how we can create a very simple "dynamic" metamodel (as opposed to static ones, see the documentation for more details):

>>> from pyecore.ecore import EClass, EAttribute, EString, EObject
>>> Graph = EClass('Graph')  # We create a 'Graph' concept
>>> Node = EClass('Node')  # We create a 'Node' concept
>>>
>>> # We add a "name" attribute to the Graph concept
>>> Graph.eStructuralFeatures.append(EAttribute('name', EString,
                                                default_value='new_name'))
>>> # And one on the 'Node' concept
>>> Node.eStructuralFeatures.append(EAttribute('name', EString))
>>>
>>> # We now introduce a containment relation between Graph and Node
>>> contains_nodes = EReference('nodes', Node, upper=-1, containment=True)
>>> Graph.eStructuralFeatures.append(contains_nodes)
>>> # We add an opposite relation between Graph and Node
>>> Node.eStructuralFeatures.append(EReference('owned_by', Graph, eOpposite=contains_nodes))

With this code, we have defined two concepts: Graph and Node. Both have a name, and there exists a containment relationship between them. This relation is bi-directional, which means that each time a Node object is added to the nodes relationship of a Graph, the owned_by relation of the Node is also updated (it also works the other way around).

Let's create some instances of our freshly created metamodel:

>>> # We create a Graph
>>> g1 = Graph(name='Graph 1')
>>> g1
<pyecore.ecore.Graph at 0x7f0055554dd8>
>>>
>>> # And two node instances
>>> n1 = Node(name='Node 1')
>>> n2 = Node(name='Node 2')
>>> n1, n2
(<pyecore.ecore.Node at 0x7f0055550588>,
 <pyecore.ecore.Node at 0x7f00555502b0>)
>>>
>>> # We add them to the Graph
>>> g1.nodes.extend([n1, n2])
>>> g1.nodes
EOrderedSet([<pyecore.ecore.Node object at 0x7f0055550588>,
             <pyecore.ecore.Node object at 0x7f00555502b0>])
>>>
>>> # bi-directional references are updated
>>> n1.owned_by
<pyecore.ecore.Graph at 0x7f0055554dd8>

This example gives a quick overview of some of the features you get for free when using PyEcore.

The project slowly grows and it still requires more love.

Installation

PyEcore is available on pypi, you can simply install it using pip:

$ pip install pyecore

The installation can also be performed manually (better in a virtualenv):

$ python setup.py install

Documentation

You can read the documentation at this address:

https://pyecore.readthedocs.io/en/latest/

Dependencies

The dependencies required by pyecore are:

  • ordered-set which is used for the ordered and unique collections expressed in the metamodel,
  • lxml which is used for the XMI parsing.

These dependencies are directly installed if you choose to use pip.

Run the Tests

The tests use py.test and 'coverage'. Everything is driven by Tox, so in order to run the tests simply run:

$ tox

Liberty Regarding the Java EMF Implementation

  • There is some meta-property that could be missing inside PyEcore. If you see one missing, please open a new ticket!
  • Proxies are not "removed" once resolved as in the the Java version, instead they act as transparent proxies and redirect all calls to the 'proxied' object.
  • PyEcore is able to automatically load some model/metamodel dependencies on its own.

State

In the current state, the project implements:

  • the dynamic/static metamodel definitions,
  • reflexive API,
  • inheritance,
  • enumerations,
  • abstract metaclasses,
  • runtime typechecking,
  • attribute/reference creations,
  • collections (attribute/references with upper bound set to -1),
  • reference eopposite,
  • containment reference,
  • introspection,
  • select/reject on collections,
  • Eclipse XMI import (partially, only single root models),
  • Eclipse XMI export (partially, only single root models),
  • simple notification/Event system,
  • EOperations support,
  • code generator for the static part,
  • EMF proxies (first version),
  • object deletion (first version),
  • EMF commands (first version),
  • EMF basic command stack,
  • EMF very basic Editing Domain,
  • JSON import (simple JSON format),
  • JSON export (simple JSON format),
  • introduce behavior @runtime,
  • resources auto-load for some cross-references,
  • derived collections,
  • multiple roots resources,
  • xsi:schemaLocation support for XMI resources,
  • URI mapper like,
  • EGeneric support (first simple version),
  • URI converter like

The things that are in the roadmap:

  • new implementation of EOrderedSet, EList, ESet and EBag,
  • new implementation of EStringToStringMapEntry and EFeatureMapEntry,
  • improve documentation,
  • copy/paste (?).

Existing Projects

There aren't too many projects proposing to handle models and metamodels in Python. The only projects I found are:

PyEMOF proposes an implementation of the OMG's EMOF in Python. The project targets Python2, only supports Class/Primitive Types (no Enumeration), XMI import/export and does not provide a reflexion layer. The project didn't move since 2005.

EMF4CPP proposes a C++ implementation of EMF. This implementation also introduces Python scripts to call the generated C++ code from a Python environment. It seems that the EMF4CPP does not provide a reflexive layer either.

PyEMOFUC proposes, like PyEMOF, a pure Python implementation of the OMG's EMOF. If we stick to a kind of EMF terminology, PyEMOFUC only supports dynamic metamodels and seems to provide a reflexive layer. The project does not appear to have moved since a while.

Contributors

Thanks for making PyEcore better!

Additional Resources

  • This article on the blog of Professor Jordi Cabot gives more information and implementation details about PyEcore.

pyecoregen's People

Contributors

aranega avatar ferraith avatar moltob avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyecoregen's Issues

Derived collection generation

I'm currently adding the support for derived collections in PyEcore, which means that using pyecoregen (with some adjustments), it will be possible to have a dedicated Python UML implementation. The thing with the derived collection is that it will obviously requires to manually add code in order to implement their behaviors.

In the case of a simple generation (no mixins), there is no issue, the code could be generated in the module along with the EClass code and that's all.

However, I'm not sure where to insert the code for the derived collection in the case where the user code is generated in a mixin. For example, for UML superClass reference, which is a derived collection, each time a Class is added to the superClass collection, a Generalization must be created at the same time. A first naïve code would look like this:

class DerivedSuperClass(EDerivedCollection):
    # ... code for __len__ ...etc
    def insert(self, index, item):
        self.check(item)
        self.owner.generalization.append(Generalization(general=item))

    # ...

If the derived code is located with the mixins, it cannot make a import Generalization as Generalization is in uml and uml imports mixins. A way of 'breaking' the circular dependency could be to add the import directly in the method that needs it, but I'm not sure that would be the best solution. I find a little bit cumbersome to ask people of importing like this.

Import error when using freshly installed pyecoregen

After fresh installation of this library, the command line tool is throwing the following error message:

>pyecoregen -h
Traceback (most recent call last):
  File "C:\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Nina\ninaenv\Scripts\pyecoregen.exe\__main__.py", line 5, in <module>
  File "c:\nina\ninaenv\lib\site-packages\pyecoregen\cli.py", line 7, in <module>
    import pyecore.resources
  File "c:\nina\ninaenv\lib\site-packages\pyecore\resources\__init__.py", line 2, in <module>
    from . import xmi
  File "c:\nina\ninaenv\lib\site-packages\pyecore\resources\xmi.py", line 4, in <module>
    from lxml import etree
ImportError: DLL load failed: Die angegebene Prozedur wurde nicht gefunden.

Probably some dependency (lxml) is missing.

@varanega: I am a coworker of Mike...

Wrong code generated for enums

See pyecore/pyecore#126 (comment)

Quoting from there, for enums,

pyecoregen generates the wrong code:

for classif in otherClassifiers:
    eClassifiers[classif.name] = classif
    classif.ePackage = eClass

for classif in eClassifiers.values():
    eClass.eClassifiers.append(classif.eClass)

What happens is that, initially, eClassifiers is empty. Then, first, enums are added to eClassifiers, and then the eClass of each object in eClassifiers is added to the EPackage (that for some reason is called eClass). So we have the EEnum class added to the package we're defining! And, of course, it gets removed from the Ecore package, because eClassifiers is a containment relationship. I don't know the purpose of the second for loop, maybe it was to handle custom metaclasses or stuff like that, but it should filter the classes that it puts into eClass, or be removed altogether if it's not necessary.

multiple inheritance requires to **kwargs in any case

If using multiple inheritance than the **kwargs need to be generated independent from eSuperTypes

{%- macro generate_class_init_args(c) -%}
    {% if c.eStructuralFeatures | list  %}, *, {% endif -%}
    {{ c.eStructuralFeatures | map(attribute='name') | map('re_sub', '$', '=None') | join(', ') }}
    {%- if c.eSuperTypes %}, **kwargs{% endif %}
{%- endmacro %}

{%- macro generate_super_init_args(c, user_module=False) -%}
    {%- if user_module and c.eStructuralFeatures -%}
    {{ c.eStructuralFeatures | map(attribute='name') | map('re_sub', '(.+)', '\g<0>= \g<0>') | join(', ') }}
    {%- endif %}
    {%- if c.eSuperTypes %}{{', ' if user_module and c.eStructuralFeatures else ''}}**kwargs{% endif %}
{%- endmacro %}

Following use case:

class A
  aattr1
  aattr2

class B 
 battr1
 battr2

class C(A,B)


C(aattr1=1, battr1=2)

A.__init__ looks like this def __init__(self, *, aattr1=None, aattr2=None)
This lead to the problem that A.__init__ is called with a keyword argument (battr1) which is not known by the init function.

BR
Andreas

Enum newline break at wrong position.

Hi!

In our model we have an EEnum with quite a lot of options and in the latest version of pyecoregen the generated python code cannot compile due to a wrong line break.
It generates the following output (scroll to the right):

PhysicalQuantityEnum = EEnum('PhysicalQuantityEnum', literals=['UNDEFINED', 'ENERGY', 'POWER', 'VOLTAGE', 'PRESSURE', 'TEMPERATURE', 'EMISSION', 'COST', 'TIME', 'LENGTH', 'DISTANCE', 'IRRADIANCE', 'SPEED', 'STATE_OF_CHARGE', 'VOLUME', 'AREA', 'POWER_REACTIVE', 'COMPOSITION', 'FLOW', 'STATE', 'HEAD', 'POSITION', 'COEFFICIENT', 'WEIGHT', 'FORCE
                                                               ', 'CURRENT'])

It breaks before the ' instead of after the ,.

A different EEnum in our model has even more options, but is OK:

ProfileTypeEnum = EEnum('ProfileTypeEnum', literals=['UNDEFINED', 'SOLARIRRADIANCE_IN_W_PER_M2', 'WINDSPEED_IN_M_PER_S', 'STATEOFCHARGE_IN_WS', 'ENERGY_IN_WH', 'ENERGY_IN_KWH', 'ENERGY_IN_MWH', 'ENERGY_IN_GWH', 'ENERGY_IN_J', 'ENERGY_IN_KJ', 'ENERGY_IN_MJ', 'ENERGY_IN_GJ', 'ENERGY_IN_TJ',
                                                     'ENERGY_IN_PJ', 'TEMPERATURE_IN_C', 'TEMPERATURE_IN_K', 'POWER_IN_W', 'POWER_IN_KW', 'POWER_IN_MW', 'POWER_IN_GW', 'POWER_IN_TW', 'MONEY_IN_EUR', 'MONEY_IN_KEUR', 'MONEY_IN_MEUR', 'PERCENTAGE', 'MONEY_IN_EUR_PER_KW', 'MONEY_IN_EUR_PER_KWH', 'VOLUME_IN_M3', 'VOLUME_IN_LITERS'])

Any idea what causes this?
Thanks!

Documentation generation

Is it possible to generate documentation from an ecore model using this tool? If the feature is currently not available, could you give me some pointers to start?

Allow passing context data via generator

This is likely also a change in the base project pymultigen. Allow to pass in template context data from generators. Currently, contextual data is collected via tasks only. This does not work well with global data that e.g. is determined via user-given command line arguments. At that level, only the generator class is known, not the various tasks it utilizes.

from x import y vs. import x

Hi @aranega

I try to create static code from complex ecore models like capella. I always run into the problem of circular includes.
But I think this could be avoided if other modules are loaded by using import x instead of from x import y.
So if you have a module like this

module examlple
__init__.py
from .one import Foo
from .one import helper

one.py
from example import two 

class Foo(two.Bar):
    def __init__(self):
        super().__init__()
        print('Init Foo')

def helper():
    print('Help')

def main():
    foo = Foo()

if __name__ == '__main__':
    main()

two.py
from example import helper

class Bar:
    def __init__(self):
        print('Init Bar')
        helper()
    

As far as I understood, by using from x import y y is loaded into the local namespace. Therefore example above will not work.
but changing two.py to

two.py
import example

class Bar:
    def __init__(self):
        print('Init Bar')
        example.helper()
    

And it can be resolved. So instead loading single classifiers from different ecore files importing the whole package can maybe solve this issue.
Maybe I find some time to dive deeper and can prepare a patch and some example. Hope you have a rough idea of the problem.

Maybe you can give me a hint how I easily get the containing package from the classifier in the code gen template.

Best regards,
Andreas

Including PRs and some branches

Hi @moltob ,

I contact you this way because I was really certain that we exchanged emails in the past, but I was totally mistaken. There is some PR pending and some fix that are on some branches. Would it be ok if I include them myself?

Thanks,
Vincent

Generator not working on ecore model

I'm trying to use pyecoregen on an ecore model (which can be found here: https://github.com/EnergyTransition/ESDL/blob/master/esdl/model/esdl.ecore). The model is developed in Eclipse Photon.

I'm getting the following error when trying to generate the classes:

$ pyecoregen -vv -e C:\\Data\\git\\esdl.next\\model\\esdl.ecore -o gen
2018-10-03 15:20:56,926 INFO [multigen.generator] Generating code to 'gen'.
2018-10-03 15:20:56,926 DEBUG [multigen.generator] <pyecore.ecore.EPackage object at 0x6fffe7aa6d8> --> 'gen/esdl/__init__.py'
2018-10-03 15:20:57,566 DEBUG [multigen.generator] <pyecore.ecore.EPackage object at 0x6fffe7aa6d8> --> 'gen/esdl/esdl.py'
Traceback (most recent call last):
  File "/usr/bin/pyecoregen", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python3.6/site-packages/pyecoregen/cli.py", line 15, in main
    generate_from_cli(sys.argv[1:])  # nocover
  File "/usr/lib/python3.6/site-packages/pyecoregen/cli.py", line 62, in generate_from_cli
    ).generate(model, parsed_args.out_folder)
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 330, in generate
    super().generate(model, outfolder)
  File "/usr/lib/python3.6/site-packages/multigen/generator.py", line 39, in generate
    task.run(element, outfolder)
  File "/usr/lib/python3.6/site-packages/multigen/generator.py", line 65, in run
    self.generate_file(element, filepath)
  File "/usr/lib/python3.6/site-packages/multigen/jinja.py", line 43, in generate_file
    context = self.create_template_context(element=element)
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 118, in create_template_context
    imported_classifiers=self.imported_classifiers(element)
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 96, in imported_classifiers
    imported |= {t for t in attributes_types if t.ePackage not in {p, ecore.eClass, None}}
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 96, in <setcomp>
    imported |= {t for t in attributes_types if t.ePackage not in {p, ecore.eClass, None}}
  File "/usr/lib/python3.6/site-packages/pyecore/ecore.py", line 895, in __getattribute__
    self._wrapped = decoded.eClass
AttributeError: 'NoneType' object has no attribute 'eClass'

Any idea what goes wrong here?

Prevent generation of invalid identifiers

I am using the generator to create Python classes from a metamodel for Franca IDL. The ecore model defined by the project contains some classes used in an expression language, e.g. there is a meta-class else. Right now, pyecoregen will therefore produce invalid Python code since Python keywords must not be used as identifiers.

This enhancement will add a Jinja filter to ensure conflicting names are adapted during code generation. The Pythonic way of adaptation is to add a trailing underscore, which is what this filter would be doing.

PYPI tar file doesn't have template files

It appears that the tar file on PYPI (https://pypi.python.org/pypi/pyecoregen) doesn't have the templates included.

I tried to install it from the tar file and ran into errors that said the template files couldn't be found.

I looked through the tar file and didn't see them but when looked through the wheel file I found them.

I reinstalled using the wheel file and everything worked.

I couldn't figure out how to report the issue on PYPI so thought I would post here.

Thanks,
Ben

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.