pawamoy / dependenpy Goto Github PK

View Code? Open in Web Editor NEW

27.0 5.0 4.0 1.48 MB

Show the inter-dependencies between modules of Python packages.

Home Page: https://pawamoy.github.io/dependenpy/

License: ISC License

Python 98.11% Makefile 1.20% Shell 0.52% Dockerfile 0.17%

python dependencies analysis dsm matrix

dependenpy's Introduction

Dependenpy

Show the inter-dependencies between modules of Python packages.

dependenpy allows you to build a dependency matrix for a set of Python packages. To do this, it reads and searches the source code for import statements.

Installation

With pip:

pip install dependenpy

With pipx:

python3.8 -m pip install --user pipx
pipx install dependenpy

Usage (as a library)

from dependenpy import DSM

# create DSM
dsm = DSM('django')

# transform as matrix
matrix = dsm.as_matrix(depth=2)

# initialize with many packages
dsm = DSM('django', 'meerkat', 'appsettings', 'dependenpy', 'archan')
with open('output', 'w') as output:
    dsm.print(format='json', indent=2, output=output)

# access packages and modules
meerkat = dsm['meerkat']  # or dsm.get('meerkat')
finder = dsm['dependenpy.finder']  # or even dsm['dependenpy']['finder']

# instances of DSM and Package all have print, as_matrix, etc. methods
meerkat.print_matrix(depth=2)

This package was originally design to work in a Django project. The Django package django-meerkat uses it to display the matrices with Highcharts.

Usage (command-line)

usage: dependenpy [-d DEPTH] [-f {csv,json,text}] [-g] [-G] [-h]
                  [-i INDENT] [-l] [-m] [-o OUTPUT] [-t] [-v] 
                  [-z STRING] PACKAGES [PACKAGES ...]

Command line tool for dependenpy Python package.

positional arguments:
  PACKAGES              The package list. Can be a comma-separated list. Each
                        package must be either a valid path or a package in
                        PYTHONPATH.

optional arguments:
  -d DEPTH, --depth DEPTH
                        Specify matrix or graph depth. Default: best guess.
  -f {csv,json,text}, --format {csv,json,text}
                        Output format. Default: text.
  -g, --show-graph      Show the graph (no text format). Default: false.
  -G, --greedy          Explore subdirectories even if they do not contain an
                        __init__.py file. Can make execution slower. Default:
                        false.
  -h, --help            Show this help message and exit.
  -i INDENT, --indent INDENT
                        Specify output indentation. CSV will never be
                        indented. Text will always have new-lines. JSON can be
                        minified with a negative value. Default: best guess.
  -l, --show-dependencies-list
                        Show the dependencies list. Default: false.
  -m, --show-matrix     Show the matrix. Default: true unless -g, -l or -t.
  -o OUTPUT, --output OUTPUT
                        Output to given file. Default: stdout.
  -t, --show-treemap    Show the treemap (work in progress). Default: false.
  -v, --version         Show the current version of the program and exit.
  -z ZERO, --zero ZERO  Character to use for cells with value=0 (text matrix 
                        display only). Default: "0".

Example:

$ # running dependenpy on itself
$ dependenpy dependenpy -z=

                Module │ Id │0│1│2│3│4│5│6│7│8│
 ──────────────────────┼────┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
   dependenpy.__init__ │  0 │ │ │ │4│ │ │ │ │2│
   dependenpy.__main__ │  1 │ │ │1│ │ │ │ │ │ │
        dependenpy.cli │  2 │1│ │ │1│ │4│ │ │ │
        dependenpy.dsm │  3 │ │ │ │ │2│1│3│ │ │
     dependenpy.finder │  4 │ │ │ │ │ │ │ │ │ │
    dependenpy.helpers │  5 │ │ │ │ │ │ │ │ │ │
       dependenpy.node │  6 │ │ │ │ │ │ │ │ │3│
    dependenpy.plugins │  7 │ │ │ │1│ │1│ │ │ │
 dependenpy.structures │  8 │ │ │ │ │ │1│ │ │ │

dependenpy's People

Contributors

Stargazers

Watchers

Forkers

pombredanne gitter-badger vladdu

dependenpy's Issues

Add option to reference a module path from another

Example: os.path = posixpath

Fix external deps being considered as internal

See FIXME in node.RootNode._get_target.
Reproduce with django.contrib.auth.forms: it is importing forms from django.

Solution: when parsing files with ast, record what objects are defined in the module. Then check here if the given part is one of these objects.

Export as image

It would be useful to be able to export matrices/treemaps/graphs as images (PNG or SVG). This will also allow for more colors and gradients. Graphs can use Graphviz, but for the others it needs to be implemented.

Maybe the presentations should be separated from the core functionality, and implemented as some kind of plugins. This way, people could add their own ways of displaying the results. But this is probably a different feature.

Improve documentation

Add limitations details (only from import, not import, only internal imports...)
Write technical doc (algorithms, ...)

No module named 'mkdocstrings.handlers.python'

Describe the bug
When running "make check", I get this error:

  > mkdocs build -s
  INFO     -  Cleaning site directory
  INFO     -  Building documentation to directory:
              /home/vdumitre/projects/dependenpy/site
  Package(s) not found: atomicwrites, cached-property, typed-ast
  INFO     -  The following pages exist in the docs directory, but are not
              included in the "nav" configuration:
                - usage.md
  ERROR    -  Error reading page 'reference/__init__.md': No module named
              'mkdocstrings.handlers.python'

To Reproduce
Steps to reproduce the behavior:

Run make check
See error

System (please complete the following information):

Dependenpy version: 3.3.1.dev11
Python version: 3.8
OS: Linux

Question

I'm looking for a similar tool that would determine the dependencies between classes (to compute coupling / cohesion metrics). Are you aware of such a tool?

Project status?

Hi, is the project maintained? The last commit was almost 2 years ago, and there are many things that would have to be updated to keep up with the times (python versions, dependency versions)

import a.b.c.d as utils is not counted

I have two modules

a.b
a.x
Within submodule a.x.y.z, there is an "import a.b.c.d as utils"

Hence I would expect a "uses' relation in the matrix.
However, "MatrixBuilder(['a.b','a.x']).build().get_matrix(2)" returns all zeros

Changing the import to eg "from a.b.c import d as utils" works fine

[Question] Sorry how does dependenpy, archan, and meerkat related to one another?

I was googling about design structure matrices and python when I stumble upon this project.

I like to know how the 3 projects are related to one another and how to use them to improve the architecture of code written in a Django app. Assume LTS 2.2 and above

Heat matrix

With one color gradient

Group sorts

Implement group sort methods (normal is done by default at creation, we need to implement reverse)

Refactor to be more object oriented

Currently only 2 classes: Matrix and MatrixBuilder.

Refactor with classes like Module and Import.

Add colors

Related to #44. As an alternative to changing the "zero deps" character, we could add colors to the output to highlight the cells where deps are above 0, maybe with a gradient (cold, warm) on a scale from min_deps to max_deps.

Path resolver not used

A given custom path resolver might not be used when building the data.

Clue: when pallamidessi tried to build the dependency matrix in his virtualenv through the non-yet-public django-dpdpy app, wrong data were included.

Apparently the custom django path resolver has not been used and the default one included the main directory of the virtualenv called genida, instead of the package called genida (inside the project, not the virtualenv). This is understandable as the virtualenv is before the project root in sys.path. But sys.path should not have been used.

A first fix for this would be to check, when looping over sys.path, if the found directory that matches the current searched package name, actually is a package and not just a directory (i.e.: contains an init.py file), to avoid adding directories that are not packages.

Add as_graph method

A dependency or adjacency matrix can be transformed as a graph. Add a method in RootNode.

Initial Update

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create seperate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

Implement TreeMap structure

As well as methods to output it as text, json and csv.

Implement text output for graph objects

We already have JSON and CSV for graphs, now we need to be able to output it as text. Maybe see graph-easy (Perl module) ASCII renderer.

Add option to choose character when zero deps

Instead of printing zeroes everywhere in text format, user should be able to choose a space a or dash. It would make the output more readable.

add poetry.lock back

Describe the bug
The poetry-lock file should be committed to version control, so that all contributors use the same versions.

I have cloned the repo, followed the instructions, but "make check" crashes due to some version mismatch.

System (please complete the following information):

Dependenpy version: 3.3.0
Python version: 3.8
OS: Linux

DSM as numpy array or 2D list

First of all thank you for maintaining this. I would like to use the dependency matrix obtained with dependenpy in a personal project, and for that I would like to have the matrix returned as a 2D list or array. Inspecting the code I haven't found a trivial way of doing it, although it's likely that I have missed it.

Is it possible at the moment? I have no problem editing the code myself, but I would need to understand how this information is stored. I was initially looking for the matrix.print() function (expecting to see some nested for loop), but I am a bit confused about it.

Thanks.