Giter Club home page Giter Club logo

guppy-pe's Introduction

This is Guppy-PE version 0.1.11

A Python Programming Environment with Heapy heap analysis toolset.

Note that his package is for Python2 only. There is a fork that is
ported and recommended for Python3 at:

https://github.com/zhuyifei1999/guppy3

CONTENTS

This distribution provides a root package, guppy, with the following
subpackages:

doc	Documentation data files: *.html, *.jpg, and help support.

etc	Support modules. Contains especially the Glue protocol module.

gsl	The Guppy Specification Language implementation. This can
	be used to create documents and tests from a common source.

heapy	The heap analysis toolset. It can be used to find information
	about the objects in the heap and display the information
	in various ways.

sets	Bitsets and 'nodesets' implemented in C.

The following files are not in packages but on the toplevel:

gsl-mode-0.1.el		Emacs mode for GSL
specs/*.gsl		Specifications
specs/genguppydoc.py	Regenerates the doc/*.html files from specs/*.gsl

REQUIREMENTS

You should have Python 2.3, 2.4, 2.5, 2.6 or 2.7

To build the system so you can use Heapy, you should have what is
needed to build extension modules from source code. The extension
modules are used not only in Heapy but also in GSL in some
situations. (The test generator uses bitsets to speed up an
algorithm.)

To use the graphical browser, Tkinter is needed.
To use the remote monitor, threading must be available.

INSTALLATION

Extract the files from the .tar.gz file, in Linux for example by:

tar xzf guppy-x.y.z.tar.gz

where x.y.z is the current version number. This will create the
directory guppy-x.y.z .  You should then cd into this directory.  You
can then compile the extension modules by:

python setup.py build

And to install you can do, for example, as follows to install to the
default location (when you are super-user):

python setup.py install

NOTE that you typically must LEAVE the guppy-x.y.z directory to use
Heapy, since the current directory is usually in the Python search
path, and then Python will find the guppy directory there FIRST and
will NOT find the extension modules. This may be a somewhat common
problem -- so I am noting it here for lack of a suitable fix at the
moment.

I also note that the documentation files and emacs-mode are not
automatically installed.

HEAPY USAGE EXAMPLE

The following example shows

1. How to create the session context: h=hpy()
2. How to show the reachable objects in the heap: h.heap()
3. How to create and show a set of objects: h.iso(1,[],{})
4. How to show the shortest paths from the root to x: h.iso(x).sp

>>> from guppy import hpy; h=hpy()
>>> h.heap()
Partition of a set of 48477 objects. Total size = 3265516 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  25773  53  1612820  49   1612820  49 str
     1  11699  24   483960  15   2096780  64 tuple
     2    174   0   241584   7   2338364  72 dict of module
     3   3478   7   222592   7   2560956  78 types.CodeType
     4   3296   7   184576   6   2745532  84 function
     5    401   1   175112   5   2920644  89 dict of class
     6    108   0    81888   3   3002532  92 dict (no owner)
     7    114   0    79632   2   3082164  94 dict of type
     8    117   0    51336   2   3133500  96 type
     9    667   1    24012   1   3157512  97 __builtin__.wrapper_descriptor
<76 more rows. Type e.g. '_.more' to view.>
>>> h.iso(1,[],{})
Partition of a set of 3 objects. Total size = 176 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  33      136  77       136  77 dict (no owner)
     1      1  33       28  16       164  93 list
     2      1  33       12   7       176 100 int
>>> x=[]
>>> h.iso(x).sp
 0: h.Root.i0_modules['__main__'].__dict__['x']
>>> 

TEST

To test if the heapy build and installation was ok, you can do:

>>> from guppy import hpy
>>> hpy().test()
Testing sets
Test #0
Test #1
Test #2
...

There will be several more tests. Some tests may take a while.

Caveat 1: Some tests may be somewhat picky, and may have to be relaxed
to pass in different installations.

Caveat 2: All tests don't currently work always in Python 2.7

LICENSE

Copyright (C) 2005 -- 2019 Sverker Nilsson, S. Nilsson Computer System AB

The right is granted to copy, use, modify and redistribute this code
according to the rules in what is commonly referred to as an MIT
license.

*** USE AT YOUR OWN RISK AND BE AWARE THAT THIS IS AN EARLY RELEASE ***

CONTACT INFORMATION

Sverker Nilsson <[email protected]> (Homepage: http://sncs.se)
Guppy-PE homepage: https://svenil.github.io/guppy-pe/

guppy-pe's People

Contributors

svenil avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

guppy-pe's Issues

Bug in "dominos" or "size" (or my understanding thereof)?

Currently, I try to analyze an apparent memory leak (under Linux) using different types of analysis, among them the following based on heappy:

class GcHpy(SnapshotData):
  """Statistics based on `gc` and `heapy` (ATTENTION: expensive and blocking)."""
  # @gc_collect_after
  def do_sample(self, view):
    gc.collect()
    h = hpy()
    objs = h.idset(gc.get_objects()).dominos
    return dict(
      count=objs.count,
      size=objs.size,
      garbage=len(gc.garbage)
      )

The idea is to let gc determine the gc managed "containers" and from them determine all dominos (for unknown reasons is this much faster then hpy.heap()) and their size.

For a concrete observation, this approach reports about 1.3 GB as size while at the same time mallinfo and malloc_info (they report the status of the "libc" memory manager) indicate that the "libc" memory manager has delivered less than 0.8 GB to the application (even requested less than that amount from the operating system).

Obviously, there is an error somewhere. Have you an idea?

Segmentation fault on long list (Stack overflow?)

This bug was first reported at:
https://sourceforge.net/p/guppy-pe/bugs/20/

Can we do something about it? It is probably a stack overflow. It would be better if we could check for stack size and report an exception instead of trigging a segmentation fault.

#!/usr/bin/env python

from guppy import hpy

class node(object):
    def __init__(self, value, _next=None):
        self.value = value
        self.next = _next


llist = node(0)
curnode = llist
for i in xrange(int(1e6)):
    curnode.next = node(0)
    curnode = curnode.next

heap = hpy().heap()

"heapy.UniSet.IdentitySetMulti" creates cyclic data structures

IdentitySetMulti apparently creates a cyclic data structure with references to all contained objects. As a consequence, deleting the IdentitySetMultii is not sufficient to (immediately) "free" the references to those objects; they are freed only at the next garbage collection. This may confuse memory leak analysis as temporary objects still seem alive.

Demonstration:

>>> from weakref import ref
>>> from sys import getrefcount
>>> from gc import collect
>>> from guppy import hpy; h=hpy()
>>> 
>>> class C(object):
...   def __init__(self):
...     self.i = 0
... 
>>> d = {"c": C()}
>>> cr = ref(d["c"])
>>> dd = h.iso(d).dominos
>>> d.clear() # only reference to C instance in dd
>>> cr()
<__main__.C object at 0x7fc6e0d19550>
>>> del dd # last reference to C should have disappeared
>>> cr() # but it still exists
<__main__.C object at 0x7fc6e0d19550>
>>> collect()
21
>>> cr() # only now has it disappeared

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.