Giter Club home page Giter Club logo

python-xdis's Introduction

CircleCI PyPI Installs Latest Version Supported Python Versions

Packaging Status

xdis

A Cross-Python bytecode disassembler, bytecode/wordcode and magic-number manipulation library/package.

Introduction

The Python dis module allows you to disassemble bytecode from the same version of Python that you are running on. But what about bytecode from different versions?

That's what this package is for. It can "marshal load" Python bytecodes from different versions of Python. The command-line routine pydisasm will show disassembly output using the most modern Python disassembly conventions in a variety of user-specified formats. Some of these formats like extended and extended-format are the most advanced of any Python disassembler I know of because they can show expression-tree on operators. See the [Disassembler Example][#disassembler-example] below.

Also, if you need to modify and write bytecode, the routines here can be of help. There are routines to pack and unpack the read-only tuples in Python's Code type. For interoperability between the changes over the years to Python CodeType, provide our own versions of the Code Type to allow interoperability, and we provide routines to reduce the tedium in writing a bytecode file.

This package also has an extensive knowledge of Python bytecode magic numbers, including PyPy and others, and how to translate from sys.sys_info major, minor, and release numbers to the corresponding magic value.

So if you want to write a cross-version assembler, bytecode-level analyzer, or optimizer this package may also be useful. In addition to the kinds of instruction categorization that dis offers, we have additional categories for things that would be useful in such a bytecode assembler, optimizer, or decompiler.

The programs here accept bytecodes from Python version 1.0 to 3.11 or so. The code requires Python 2.4 or later and has been tested on Python running lots of Python versions.

When installing, except for the most recent versions of Python, use the Python egg or wheel that matches that version, e.g. xdis-6.0.2-py3.3.egg, xdis-6.0.2-py33-none-any.whl. Of course for versions that pre-date wheel's, like Python 2.6, you will have to use eggs.

To install older versions for from source in git use the branch python-2.4-to-2.7 for Python versions from 2.4 to 2.7, python-3.1-to-3.2 for Python versions from 3.1 to 3.2, python-3.3-to-3.5 for Python versions from 3.3 to 3.5. The master branch handles Python 3.6 and later.

Installation

The standard Python routine:

$ pip install -e .
$ pip install -r requirements-dev.txt

A GNU makefile is also provided so make install (possibly as root or sudo) will do the steps above.

Disassembler Example

The cross-version disassembler that is packaged here, can produce assembly listing that are superior to those typically found in Python's dis module. Here is an example:

pydisasm -S -F extended bytecode_3.8/pydisasm-example.pyc
# pydisasm version 6.1.1.dev0
# Python bytecode 3.8.0 (3413)
# Disassembled from Python 3.11.8 (main, Feb 14 2024, 04:47:01) [GCC 13.2.0]
# Timestamp in code: 1693155156 (2023-08-27 12:52:36)
# Source code size mod 2**32: 320 bytes
# Method Name:       <module>
# Filename:          simple_source/pydisasm-example.py
# Argument count:    0
# Position-only argument count: 0
# Keyword-only arguments: 0
# Number of locals:  0
# Stack size:        3
# Flags:             0x00000040 (NOFREE)
# First Line:        4
# Constants:
#    0: 0
#    1: None
#    2: ('version_info',)
#    3: 1
#    4: (2, 4)
#    5: 'Is small power of two'
# Names:
#    0: sys
#    1: version_info
#    2: print
#    3: version
#    4: len
#    5: major
#    6: power_of_two
             # import sys
  4:           0 LOAD_CONST           (0) ; TOS = 0
               2 LOAD_CONST           (None) ; TOS = None
               4 IMPORT_NAME          (sys) ; TOS = import_module(sys)
               6 STORE_NAME           (sys) ; sys = import_module(sys)

             # from sys import version_info
  5:           8 LOAD_CONST           (0) ; TOS = 0
              10 LOAD_CONST           (('version_info',)) ; TOS = ('version_info',)
              12 IMPORT_NAME          (sys) ; TOS = import_module(sys)
              14 IMPORT_FROM          (version_info) ; TOS = from sys import version_info
              16 STORE_NAME           (version_info) ; version_info = from sys import version_info
              18 POP_TOP

             # print(sys.version)
  7:          20 LOAD_NAME            (print) ; TOS = print
              22 LOAD_NAME            (sys) ; TOS = sys
              24 LOAD_ATTR            (version) ; TOS = sys.version
              26 CALL_FUNCTION        (1 positional argument) ; TOS = print(sys.version)
              28 POP_TOP

             # print(len(version_info))
  8:          30 LOAD_NAME            (print) ; TOS = print
              32 LOAD_NAME            (len) ; TOS = len
              34 LOAD_NAME            (version_info) ; TOS = version_info
              36 CALL_FUNCTION        (1 positional argument) ; TOS = len(version_info)
              38 CALL_FUNCTION        (1 positional argument) ; TOS = print(len(version_info))
              40 POP_TOP

             # major = sys.version_info[0]
  9:          42 LOAD_NAME            (sys) ; TOS = sys
              44 LOAD_ATTR            (version_info) ; TOS = sys.version_info
              46 LOAD_CONST           (0) ; TOS = 0
              48 BINARY_SUBSCR        TOS = sys.version_info[0]
              50 STORE_NAME           (major) ; major = sys.version_info[0]

             # power_of_two = major & (major - 1)
 10:          52 LOAD_NAME            (major) ; TOS = major
              54 LOAD_NAME            (major) ; TOS = major
              56 LOAD_CONST           (1) ; TOS = 1
              58 BINARY_SUBTRACT      TOS = major - (1)
              60 BINARY_AND           TOS = major & (major - (1))
              62 STORE_NAME           (power_of_two) ; power_of_two = major & (major - (1))

             # if power_of_two in (2, 4):
 11:          64 LOAD_NAME            (power_of_two) ; TOS = power_of_two
              66 LOAD_CONST           ((2, 4)) ; TOS = (2, 4)
              68 COMPARE_OP           (in) ; TOS = power_of_two in ((2, 4))
              70 POP_JUMP_IF_FALSE    (to 80)

             # print("Is small power of two")
 12:          72 LOAD_NAME            (print) ; TOS = print
              74 LOAD_CONST           ("Is small power of two") ; TOS = "Is small power of two"
              76 CALL_FUNCTION        (1 positional argument) ; TOS = print("Is small power of two")
              78 POP_TOP
         >>   80 LOAD_CONST           (None) ; TOS = None
              82 RETURN_VALUE         return None

Note in the above that some operand interpretation is done on items that are in the stack. For example in

24 LOAD_ATTR            (version) | sys.version

from the instruction see that sys.version is the resolved attribute that is loaded.

Similarly in:

68 COMPARE_OP           (in) | power_of_two in (2, 4)

we see that we can resolve the two arguments of the in operation. Finally in some CALL_FUNCTIONS we can figure out the name of the function and arguments passed to it.

Testing

$ make check

A GNU makefile has been added to smooth over setting running the right command, and running tests from fastest to slowest.

If you have remake installed, you can see the list of all tasks including tests via remake --tasks.

Usage

Run

$ ./bin/pydisasm -h

for usage help.

As a drop-in replacement for dis

xdis also provides some support as a drop in replacement for the the Python library dis module. This is may be desirable when you want to use the improved API from Python 3.4 or later from an earlier Python version.

For example:

>>> # works in Python 2 and 3
>>> import xdis.std as dis
>>> [x.opname for x in dis.Bytecode('a = 10')]
['LOAD_CONST', 'STORE_NAME', 'LOAD_CONST', 'RETURN_VALUE']

There may some small differences in output produced for formatted disassembly or how we show compiler flags. We expect you'll find the xdis output more informative though.

See Also

python-xdis's People

Contributors

2elli avatar arusekk avatar bmwiedemann avatar caandt avatar dperret avatar jdw170000 avatar jeffenstein avatar julionc avatar jwilk avatar laike9m avatar licy183 avatar lulzx avatar matmaus avatar moagstar avatar painor avatar rakovskij-stanislav avatar rklyne avatar rocky avatar stredaa avatar thurask avatar toddrme2178 avatar x746e avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-xdis's Issues

Invalid Syntax in unmarshal.py for Python 2.7

Occurs at Line 98 in unmarshal.py. Using Python2.7 on MacOs.

Traceback (most recent call last):
  File "/Users/****/Library/Python/2.7/bin//pydisasm", line 5, in <module>
    from xdis.bin.pydisasm import main
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/xdis/__init__.py", line 73, in <module>
    from xdis.load import (
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/xdis/load.py", line 22, in <module>
    import xdis.unmarshal
  File "/Users/****/Library/Python/2.7/lib/python/site-packages/xdis/unmarshal.py", line 98
    def compat_str(s: str) -> str:
                    ^
SyntaxError: invalid syntax

Basically a Python interpreter version and a list of code object, unable to generate the correct pyc file

In stackoverflow, there is the code:

import xdis
from xdis import PYTHON3
from xdis.magics import magics
from xdis.marsh import dumps
from struct import pack
import time


def write_pycfile(pyc_file, code_list, version=xdis.PYTHON_VERSION):
    if PYTHON3:
        file_mode = 'wb'
    else:
        file_mode = 'w'

    with open(pyc_file, file_mode) as fp:
        fp.write(magics[version])
        timestamp = int(time.time())
        fp.write(pack('I', timestamp))
        if version > 3.2:
            fp.write(pack('I', 0))
        for co in code_list:
            try:
                co_obj = dumps(co, python_version=str(version))
                if PYTHON3 and version < 3.0:
                    co_obj = str.encode(co_obj)
                    pass

                fp.write(co_obj)
            except:
                pass
            pass
    print("Wrote %s" % pyc_file)

write_pycfile("D:\Person_Program\Python_Decompile\test.pyc", [write_pycfile.__code__])

then, I run this:

pydisasm --format xasm "D:\Person_Program\Python_Decompile\test.pyc"

The console shows that the target file is not a pyc type file

File name: 'D:\Person_Program\Python_Decompile\test.pyc (12 bytes)' is too short to be a valid pyc file

My python version is 3.7. does this question have anything to do with my python version?

Incorrect setup_requires

s/setup_requires/tests_require/

Nose is a test only dependency. Using setup_requires forces it to be installed.
It should be a tests_require setting.

Cannot install under Python 2.7

Description

Can no longer install under Python 2.7 even though the trove classifiers indicate this package is compatible with 2.7.

How to Reproduce

$ docker run -it --rm python:2.7-slim-buster /bin/bash

root@76ae375f4fb4:/# python2 -m pip install xdis
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting xdis
  Downloading xdis-5.0.13.tar.gz (293 kB)
     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 293 kB 1.9 MB/s 
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python2 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9TfIIi/xdis/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9TfIIi/xdis/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-9TfIIi/xdis/pip-egg-info
         cwd: /tmp/pip-install-9TfIIi/xdis/
    Complete output (12 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-9TfIIi/xdis/setup.py", line 3, in <module>
        from xdis.version import __version__
      File "xdis/__init__.py", line 73, in <module>
        from xdis.load import (
      File "xdis/load.py", line 22, in <module>
        import xdis.unmarshal
      File "xdis/unmarshal.py", line 98
        def compat_str(s: str) -> str:
                        ^
    SyntaxError: invalid syntax
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 20.3.4 is available.
You should consider upgrading via the '/usr/local/bin/python2 -m pip install --upgrade pip' command.
root@76ae375f4fb4:/# 

Expected behavior

It to install correctly.

Environment

Above I reproduced in a Python 2.7 docker image, specifically: python:2.7-slim-buster

Additional Environment or Context

It looks like there's a type annotation in unmarshal.py, which would make the code not compatible with anything pre-3.5 (I think?)

Note that I can install 5.0.12 successfully, but not 5.0.13 (same issue), so looks like it was introduced as part of 5.0.13

Bug somewhere in deparse_code caused by ea8649d

One of two bugs found in Hypothesis CI, Iโ€™m afraid. This comes from HypothesisWorks/hypothesis#382.

We have two tests failing, on PyPy only, that are caused by something in xdis 3.2.0.

Environment config

$ pip freeze
cffi==1.3.0
enum34==1.1.6
greenlet==0.4.9
hypothesis==3.5.3
readline==6.2.4.1
spark-parser==1.4.0
uncompyle6==2.9.3
wheel==0.24.0
xdis==3.2.0

$ python --version
Python 2.7.10 (850edf14b2c75573720f59e95767335fb1affe55, Oct 30 2015, 00:18:28)
[PyPy 4.0.0 with GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)]

This snippet reproduces one of the issues:

from hypothesis import strategies as st
from io import BytesIO
import sys

from uncompyle6 import deparse_code

f = lambda right: [].map(lambda length:())
out = BytesIO()
deparse_code(2.7, f.__code__, out=out, is_pypy=True)
print(out.getvalue())

Our test expects this to return

return [].map(lambda length: ())

but in fact we get

return [].None(lambda length: ())

Playing around with git bisect, it looks as if this caused by a change in xdis, specifically ea8649d. Unfortunately I donโ€™t know enough about Python bytecode parsing to say what in this commit might be causing the issue.

As with the previous issue, we can get around this by pinning versions, but it would be nice if that wasnโ€™t necessary.

This is one of two bugs our CI spotted. I suspect they have the same root cause, as the symptoms are the same; I havenโ€™t had time to get a minimal example for the second failure yet.

Warn or fix whitespace in variable names

Continuing the discussion from here: rocky/python-uncompyle6#312

@rocky directed me to post here instead.

It's possible to easily fool a lot of decompilers by renaming variables to names containing whitespace. For example, running this little script:

$ git clone https://github.com/ZetaTwo/python-obfuscator
$ cd python-obfuscator
$ ./test.sh

Will demonstrate how uncompyle6 can be tricked into decompiling the bytecode into valid code that does not reflect the actual bytecode.

My proposal is to add a small check that will at least warn about the presence of variables with invalid names and possinly even replace the invalid characters with for example "_".

I'm willing to make a PR with these changes but I would first just like to hear if I'm in the right place and if you have opinions on where/how this should be implemented?

Issue about imp.get_magic()

Hi rocky, I noticed imp.get_magic() is invokded in 4 places in the codebase, each given a different name. Since imp is deprecated, I would like to replace it with importlib.util.MAGIC_NUMBER, meanwhile put them into one single place. Is there any file that you suggest me to put the magic number into, or you prefer to keep it in multiple places?

Error message format

A space is missing from here

"https://github.com/kholia/dedrop"
, which would cause the link https://github.com/kholia/dedropfor to be displayed instead of the correct https://github.com/kholia/dedrop. This confused me for a few minutes.

Yo Just Became A Sponsor

I want to decompile a pyinstaller file thats compiled with version 3.11
and how to get to you
need it to be decompiled urgetly

thanks

Please issue an xdis release

Description

Request to please issue a release of xdis, particularly since the latest in master can be installed on systems using Python 3.11 and 3.12

Priority

Believe this is ideal for the community to enable using as a pinned dependency

Error when using pyc-xasm

return tuple([int(v) for v in python_version.split(".")[:len]])
AttributeError: 'NoneType' object has no attribute 'split'

python version 3.8.8

Python 3.12 support

Description

Does python-xdis support python 3.12?

Background

Right now it shows an error when trying to run it on python 3.12, eg:

c:\Prg\>python
Python 3.12.0 (main, Oct 10 2023, 10:37:17) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import xdis.std as dis
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python312\Lib\site-packages\xdis\std.py", line 220, in <module>
    _std_api = make_std_api()
               ^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\xdis\std.py", line 218, in make_std_api
    return _StdApi(python_version, variant)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\xdis\std.py", line 73, in __init__
    self.opc = opc = get_opcode_module(python_version, variant)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\xdis\op_imports.py", line 174, in get_opcode_module
    return op_imports[canonic_python_version[vers_str]]
                      ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: '3.12.0'
>>>

Possible wrong magic number for python 2.4a0

Hello,

First of all, that you for your effort on this. It is really a great tool!

It seems that magic number for python 2.4a0 is possibly declared wrong.

add_magic_from_int(62041, "2.4a0")

Had to decompile some .pyc files of that version and I had some issues which were resovled by changing to

add_magic_from_int(62031, "2.4a0")

And it works perfectly.

MIT or GPL License

The license listed in the LICENSE file is MIT. However, the license of the code in the xdis/dropbox folder is GPL-2.0. How does that affect the overall license of the xdis package?

KeyError: '3.8.6' when importing uncompyle6

Traceback (most recent call last): File "", line 1, in
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/uncompyle6/init.py", line 53, in import uncompyle6.semantics.pysource
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/uncompyle6/semantics/pysource.py", line 141, in from uncompyle6.parsers.treenode import SyntaxTree
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/uncompyle6/parsers/treenode.py", line 3, in from uncompyle6.scanners.tok import NoneToken
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 200, in NoneToken = Token("LOAD_CONST", offset=-1, attr=None, pattr=None)
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 86, in init
from xdis.std import _std_api
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis/std.py", line 220, in
_std_api = make_std_api()
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis/std.py", line 218, in make_std_api
return _StdApi(python_version, variant)
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis/std.py", line 73, in init
self.opc = opc = get_opcode_module(python_version, variant)
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis/op_imports.py", line 169, in get_opcode_module
return op_imports[canonic_python_version[vers_str]]
KeyError: '3.8.6'

xdis is requiring xdis itself

Description

xdis is requiring xdis itself.
Is there a good reason to do so or caused by a bug?

How to Reproduce

pip show xdis

Output Given

$ pip show xdis
Name: xdis
Version: 6.1.0
Summary: Python cross-version byte-code library and disassembler
Home-page: https://github.com/rocky/python-xdis
Author: 
Author-email: Rocky Bernstein <[email protected]>
License: GPL
Location: /usr/local/lib/python3.12/site-packages
Requires: click, pygments, six, term-background, xdis
Required-by: xdis

Expected behavior

xdis should not appears in the Requires line of xdis.

Environment

Python 3.12.0
Debian 11.7

Workarounds

Priority

Additional Context

get_opcode_module fails to detect 3-digit versions

Unclear how it is supposed to work.
get_opcode_module fails to handle 3.6.12|3.8.6, and likely every 3.6.n|3.8.n.

$ uncompyle6 -h
Traceback (most recent call last):
  File "/usr/bin/uncompyle6", line 11, in <module>
    load_entry_point('uncompyle6==3.7.2', 'console_scripts', 'uncompyle6')()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2852, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.8/site-packages/uncompyle6/__init__.py", line 48, in <module>
    import uncompyle6.semantics.pysource
  File "/usr/lib/python3.8/site-packages/uncompyle6/semantics/pysource.py", line 141, in <module>
    from uncompyle6.parsers.treenode import SyntaxTree
  File "/usr/lib/python3.8/site-packages/uncompyle6/parsers/treenode.py", line 3, in <module>
    from uncompyle6.scanners.tok import NoneToken
  File "/usr/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 200, in <module>
    NoneToken = Token("LOAD_CONST", offset=-1, attr=None, pattr=None)
  File "/usr/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 86, in __init__
    from xdis.std import _std_api
  File "/usr/lib/python3.8/site-packages/xdis/std.py", line 220, in <module>
    _std_api = make_std_api()
  File "/usr/lib/python3.8/site-packages/xdis/std.py", line 218, in make_std_api
    return _StdApi(python_version, variant)
  File "/usr/lib/python3.8/site-packages/xdis/std.py", line 73, in __init__
    self.opc = opc = get_opcode_module(python_version, variant)
  File "/usr/lib/python3.8/site-packages/xdis/op_imports.py", line 169, in get_opcode_module
    return op_imports[canonic_python_version[vers_str]]
KeyError: '3.8.6'

Workaround is version_info[0:2] to find a match.

Unclear what the openSUSE Leap and Tumbleweed packages do wrong.

Stopping at the middle of the decompilation

Hey when i do this:

loaded = xdis.load_module_from_file_object(io.BytesIO(v))
insts: list[dis.Instruction] = xdis.Bytecode(loaded[3], xdis.get_opcode(loaded[0], False)).dis(loaded[3])
print(insts)

i get the dis of "v" but its not complete like stop at the middle of the decompilation:

             860 LOAD_CONST           (None)
             862 RETURN_VALUE
         >>  864 LOAD_CONST           (None)
             866 RETURN_VALUE
         >>  868 LOAD_CONST           (None)
             870 RETURN_VALUE

End here

but when i use pydisasm command or dis module i get it completely can i get help?

Need more investigations: `xdis.unmarshal._VersionIndependentUnmarshaller` method `t_object_reference`

Method t_object_reference of xdis.unmarshal._VersionIndependentUnmarshaller probably wants internObjects[refnum-1], not internObjects[refnum]

Found this bug while unmarshalling py2exe bootstrap code object (it's safe despite it was got from malware sample, but anyway be careful).

code object in base64:

YwAAAAAAAAAAAAAAAAAAAAADAAAAQAAAAHMuAAAAZABkAWwAWgBkAGQBbAFaAWUAagKgA2UBagShAQQAZQFfBWUBXwZbAFsBZAFTACkC6QAAAABOKQfaAm9z2gNzeXPaBHBhdGjaB2Rpcm5hbWXaCmV4ZWN1dGFibGXaC2Jhc2VfcHJlZml42gZwcmVmaXipAHIJAAAAcgkAAAD6DDxib290c3RyYXAyPtoIPG1vZHVsZT4BAAAAcwQAAAAQABYA

On the screenshot:
internObjects[refnum] => internObjects[9] => Exception.

This code object got from normal-working file, this way the problem in _VersionIndependentUnmarshaller

The fix is to use refnum-1, but I want to make sure that this fix is proper.

xdis.std.Instruction should not take "has_arg" parameter #86

Although we want has_stdarg in the underlying bytecode instruction, in in std.Instruction() it shouldn't be passed as a parameter but computed based on the opc, e.g. has_arg = opname >= opc.HAVE_ARGUMENT.

In an ideal world, fields argrepr and opname wouldn't need to be passed in std.Instruction(). We could either create another function or have setting these to be None to mean that they get filled in. And opname probably should be checked against opcode.

Python3.6 compatibility

I get the following errors on Python3.6

nose:

Basic test of load_file, check_object_path and load_module ... ERROR
test_basic (test_magic.TestOpcodes)
Basic test of magic numbers ... ERROR
test_basic (test_marsh.TestMarshal)
Tests xdis.load.load_module ... ok
Failure: AssertionError () ... FAIL

======================================================================
ERROR: test_basic (test_load.TestLoad)
Basic test of load_file, check_object_path and load_module
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1/xdis/load.py", line 95, in load_module
    version = float(magics.versions[magic][:3])
KeyError: b'3\r\r\n'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1/test_unit/test_load.py", line 16, in test_basic
    version, timestamp, magic_int, co2, is_pypy = load_module(obj_path)
  File "/tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1/xdis/load.py", line 100, in load_module
    (ord(magic[0])+256*ord(magic[1]), filename))
TypeError: ord() expected string of length 1, but int found

======================================================================
ERROR: test_basic (test_magic.TestOpcodes)
Basic test of magic numbers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1/test_unit/test_magic.py", line 10, in test_basic
    python_via_magic = magics.by_magic[ current ]
KeyError: b'3\r\r\n'

======================================================================
FAIL: Failure: AssertionError ()
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nix/store/hza739g9wc5ph8zgbab7iqyc711gz3wr-python3.6-nose-1.3.7/lib/python3.6/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/nix/store/hza739g9wc5ph8zgbab7iqyc711gz3wr-python3.6-nose-1.3.7/lib/python3.6/site-packages/nose/loader.py", line 417, in loadTestsFromName
    addr.filename, addr.module)
  File "/nix/store/hza739g9wc5ph8zgbab7iqyc711gz3wr-python3.6-nose-1.3.7/lib/python3.6/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/nix/store/hza739g9wc5ph8zgbab7iqyc711gz3wr-python3.6-nose-1.3.7/lib/python3.6/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/nix/store/jipa3vzflsc4lwhdplc274ps4frz5gqv-python3-3.6.0/lib/python3.6/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/nix/store/jipa3vzflsc4lwhdplc274ps4frz5gqv-python3-3.6.0/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 675, in _load
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1/test_unit/test_opcode.py", line 4, in <module>
    from xdis.opcodes import (opcode_23, opcode_24, opcode_25,
  File "/tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1/xdis/opcodes/opcode_36.py", line 94, in <module>
    assert all(item in opmap.items() for item in dis.opmap.items())
AssertionError

----------------------------------------------------------------------
Ran 4 tests in 0.072s

FAILED (failures=1, errors=2)

py.test:

============================= test session starts ==============================
platform linux -- Python 3.6.0, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /tmp/nix-build-python3.6-xdis-2.3.1.drv-0/xdis-2.3.1, inifile: 
collected 3 items / 2 errors

test_unit/test_load.py .
test_unit/test_magic.py F
test_unit/test_marsh.py .

==================================== ERRORS ====================================
__________________ ERROR collecting test_unit/test_opcode.py ___________________
test_unit/test_opcode.py:4: in <module>
    from xdis.opcodes import (opcode_23, opcode_24, opcode_25,
/nix/store/pwmgs44d1pa8p2jw9wbr7v3aw4y1cd2z-python3.6-xdis-2.3.1/lib/python3.6/site-packages/xdis/opcodes/opcode_36.py:94: in <module>
    assert all(item in opmap.items() for item in dis.opmap.items())
E   assert all(<generator object <genexpr> at 0x7ffff22dd2b0>)
__________________ ERROR collecting test_unit/3.3/test_dis.py __________________
test_unit/3.3/test_dis.py:394: in <module>
    class CodeInfoTests(unittest.TestCase):
test_unit/3.3/test_dis.py:396: in CodeInfoTests
    (dis.code_info, code_info_code_info),
E   NameError: name 'dis' is not defined
=================================== FAILURES ===================================
____________________________ TestOpcodes.test_basic ____________________________

self = <test_magic.TestOpcodes testMethod=test_basic>

    def test_basic(self):
        """Basic test of magic numbers"""
        current = imp.get_magic()
>       python_via_magic = magics.by_magic[ current ]
E       KeyError: b'3\r\r\n'

test_unit/test_magic.py:10: KeyError
================= 1 failed, 2 passed, 2 error in 0.16 seconds ==================

Python2.7 tests fail

was bumping this package from 4.0.1 to 4.0.3, and noticed that the python tests for python2.7 fail.

system info:

$ python --version
Python 2.7.16
$ uname -a
Linux jon-workstation 4.19.64 #1-NixOS SMP Sun Aug 4 07:30:58 UTC 2019 x86_64 GNU/Linux

Failure:

======================================================================
FAIL: test_bug_708901 (test_dis27.DisTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/xdis-4.0.3/test_unit/test_dis27.py", line 161, in test_bug_708901
    self.do_disassembly_test(bug708901, dis_bug708901)
  File "/build/xdis-4.0.3/test_unit/test_dis27.py", line 137, in do_disassembly_test
    lines)))
AssertionError: events did not match expectation:
   19:           0 SETUP_LOOP               23 (to 26)
                 3 LOAD_GLOBAL               0 (range)
                 6 LOAD_CONST                1 (1)

   20:           9 LOAD_CONST                2 (10)
-               12 CALL_FUNCTION             2 (2 positional, 0 keyword pair)
?                                                               ^ ---- -----

+               12 CALL_FUNCTION             2 (2 positional, 0 named)
?                                                               ^^^

                15 GET_ITER
           >>   16 FOR_ITER                  6 (to 25)
                19 STORE_FAST                0 (res)

   21:          22 JUMP_ABSOLUTE            16 (to 16)
           >>   25 POP_BLOCK
           >>   26 LOAD_CONST                0 (None)
                29 RETURN_VALUE



======================================================================
FAIL: test_dis (test_dis27.DisTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/xdis-4.0.3/test_unit/test_dis27.py", line 155, in test_dis
    self.do_disassembly_test(_f, dis_f)
  File "/build/xdis-4.0.3/test_unit/test_dis27.py", line 137, in do_disassembly_test
    lines)))
AssertionError: events did not match expectation:
   30:           0 LOAD_GLOBAL               0 (print)
                 3 LOAD_FAST                 0 (a)
-                6 CALL_FUNCTION             1 (1 positional, 0 keyword pair)
?                                                               ^ ---- -----

+                6 CALL_FUNCTION             1 (1 positional, 0 named)
?                                                               ^^^

                 9 POP_TOP

   31:          10 LOAD_CONST                1 (1)
                13 RETURN_VALUE



----------------------------------------------------------------------
Ran 7 tests in 0.199s

FAILED (failures=2, skipped=1)
================ 3 failed, 11 passed, 1 skipped in 1.29 seconds ================

key error!!!!

$ uncompyle6 -h
Traceback (most recent call last):
File "/home/ahmedmoselhi/.local/bin/uncompyle6", line 5, in
from uncompyle6.bin.uncompile import main_bin
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/uncompyle6/init.py", line 53, in
import uncompyle6.semantics.pysource
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/uncompyle6/semantics/pysource.py", line 141, in
from uncompyle6.parsers.treenode import SyntaxTree
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/uncompyle6/parsers/treenode.py", line 3, in
from uncompyle6.scanners.tok import NoneToken
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/uncompyle6/scanners/tok.py", line 200, in
NoneToken = Token("LOAD_CONST", offset=-1, attr=None, pattr=None)
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/uncompyle6/scanners/tok.py", line 86, in init
from xdis.std import _std_api
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/xdis/std.py", line 220, in
_std_api = make_std_api()
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/xdis/std.py", line 218, in make_std_api
return _StdApi(python_version, variant)
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/xdis/std.py", line 73, in init
self.opc = opc = get_opcode_module(python_version, variant)
File "/home/ahmedmoselhi/.local/lib/python2.7/site-packages/xdis/op_imports.py", line 169, in get_opcode_module
return op_imports[canonic_python_version[vers_str]]
KeyError: '2.7.18candidate1'

xdis-3.6.0.tar.gz on pypi, tests fail

As you know I've had problems with packaging xdis for Gentoo for some time, getting test fails.
I was trying to add the 3.6.0 release today and was getting stuck on several python-2.7 test fails.
I updated my checkout of this repo, ran the tests from it. All tests pass for all python versions we currently support.
I also noticed that there were more tests ran from the local checkout than from the install tarball downloaded from pypi.

So, I made a new tarball using setup.py sdist, re-generated our ebuild manifest for the new tarball and it runs all tests without failure. Plus it runs more tests than did the pypi tarball.
Github archives and release also work correctly.

P.S. the only patch applied is to remove setup_requires from setup.py

keyerror x(

it was working greatly on 3.9.5 after I edited python xdis scanners 37 & 37 and base 37 but now it make me another error since i upgraded on 3.9.6 KeyError: '3.9.6' on module xdis so i tried to add in python versions in xdis module but it does not works. Thanks for any help

Support standard dis api

I think it would be nice to support the standard dis api:

https://docs.python.org/2/library/dis.html

I think in terms of the principal of least surprise it would be nice to be able to use xdis as a drop in replacement for the dis module, but then with the supercharged power of cross version + the better semantics of python 3 dissasembly (i.e. being able to iterate through Instructions rather than just getting a string)

@rocky What are your thoughts?

TypeError: an integer is required (got type bytes)

I get errors like this when running in python 3.8, but when I run for python2.7 it always works, How do I fix this, sir?

$ pydisasm --asm tes.cpython-38.pyc
$# pydisasm version 4.2.2
$# Python bytecode 3.8 (3413)
$# Disassembled from Python 3.8.1 (default, Jan 15 2020, 23:19:07)
$# [Clang 8.0.7 (https://android.googlesource.com/toolchain/clang b55f2d4ebfd35bf6
$# Source code size mod 2**32: 17806 bytes
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/bin/pydisasm", line 11, in
load_entry_point('xdis==4.2.2', 'console_scripts', 'pydisasm')() File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs) File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs) File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis-4.2.2-py3.8.egg/xdis/bin/pydisasm.py", line 75, in main
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis-4.2.2-py3.8.egg/xdis/main.py", line 283, in disassemble_file
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis-4.2.2-py3.8.egg/xdis/main.py", line 153, in disco
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis-4.2.2-py3.8.egg/xdis/main.py", line 212, in disco_loop_asm_format
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis-4.2.2-py3.8.egg/xdis/main.py", line 249, in disco_loop_asm_format
File "/data/data/com.termux/files/usr/lib/python3.8/site-packages/xdis-4.2.2-py3.8.egg/xdis/code.py", line 130, in freeze
TypeError: an integer is required (got type bytes)

Wrong variable name version in _Marshaller

so the code for _Marshaller looks like this https://github.com/rocky/python-xdis/blob/master/xdis/marsh.py#L97

def __init__(self, writefunc, python_version=None):
    self._write = writefunc
    self.python_version = python_version

but it's then checking for self.version instead of self.python_version un dump

 if isinstance(x, types.CodeType) and PYTHON_VERSION != self.version:

which causes the following error

   if isinstance(x, types.CodeType) and PYTHON_VERSION != self.version:
AttributeError: '_Marshaller' object has no attribute 'version'

Unknown magic number: 62215

Dropbox seems to be using a magic number that isn't documented anywhere, as far as I can tell. It's likely that they've modified the interpreter.

/Applications/Dropbox.app/Contents/MacOS/python -V                                                                  :(
Python 2.7.11

When attempting to open one of Dropbox's pyo files with vanilla Python 2.7.12, I get:

python2.7 site.pyo
RuntimeError: Bad magic number in .pyc file

Obviously, there's no expectation for proprietary modifications to be supported, but if it's something as simple as adding the magic number, I'm not the only one who would appreciate it. This is particularly relevant given all the recent commotion surrounding Dropbox's unusual security practices on macOS; many questions could be answered if the pyo files could be decompiled.

XDis/decompyle3 throws warning when using python 3.8.18

Description

xdis throws warning when using python 3.8.18

How to Reproduce

  1. Install conda
  2. Create env with python=3.8, which will resolve to 3.8.18 (as of 19.10.2023)
  3. Install xdis in this env.
  4. Try using xdis

Output Given

I don't know about Python version '3.8.18' yet.
xdis might need to be informed about version '3.8.18'

Expected behavior

No warning

Environment

Windows 10
Conda 23.9.0
xdis 6.0.5
decompyle3 3.9.0

xdis-3.3.0 release tarball errors in setup.py

Seems this code was not committed, but was in the release tarball. Looks like a copy/paste error.
You should likely re-release as 3.3.1 with the fixed code.

/usr/bin/python3.5 setup.py build
File "setup.py", line 25
tests_require = tests_require,
^
SyntaxError: keyword argument repeated

"""Setup script for the 'xdis' distribution."""

from pkginfo import
author, author_email,
license, long_description, classifiers,
modname, py_modules, tests_require,
scripts, short_desc, tests_require,
VERSION, web, zip_safe

from setuptools import setup, find_packages
setup(
author = author,
author_email = author_email,
classifiers = classifiers,
description = short_desc,
license = license,
long_description = long_description,
name = modname,
packages = find_packages(),
py_modules = py_modules,
tests_require = setup_requires,
scripts = scripts,
tests_require = tests_require,
url = web,
version = VERSION,
zip_safe = zip_safe)

bytecode not working after disasm and xasm

for this pyc file (from a CTF problem):
https://hackme.inndy.tw/static/pyyy.pyc

$ pydisasm --asm pyyy.pyc > pyyy_disasm.pyasm

$ pyc-xasm pyyy_disasm.pyasm                
Wrote pyyy_disasm.pyc

$ python pyyy_disasm.pyc       
Traceback (most recent call last):
  File "pyyy.py", line 16, in <module>
  File "pyyy.py", line 25, in <lambda>
TypeError: unsupported operand type(s) for +: 'function' and 'function'

Add a more flexible way to adjust Magic or deal with new/alternate magic numbers.

Right now xdis needs to be update every time a new Python release comes out. This is by design because we do want to have this program be a reference on valid Python's rather than hypothetical ones.

However one proble is that right now we can only handle one magic number per Python version and sometimes for a given Python version there can be more than one. See for example #69.

Propose and implement a way to accomodate these problems.

One possibility is to add a parameter to override the built-in magic to Python mappings. And an option to pydisasm could allow a magic number override.

No doubt there are a number of other possibilityes.

Installed via pip, can't execute pydisasm

I'm using the Ubuntu 18.04 supplied Python 3.6.6 and the latest pip to install xdis globally:

$ python3 --version
Python 3.6.6
$ python3 -m pip --version
pip 18.0 from /home/bj/.local/lib/python3.6/site-packages/pip (python 3.6)
$ pydisasm
bash: /usr/local/bin/pydisasm: /home/rocky/.pyenv/versions/3.6.6/bin/python: bad interpreter: No such file or directory
$ head -1 /usr/local/bin/pydisasm
#!/home/rocky/.pyenv/versions/3.6.6/bin/python

That she bang line is already in the wheel file and definately not portable. :-)

KeyError: '3.7.5' when importing uncompyle6

Platform: Windows 10, Python 3.7.5

>>> import uncompyle6
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\uncompyle6\__init__.py", line 48, in <module>
    import uncompyle6.semantics.pysource
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\uncompyle6\semantics\pysource.py", line 136, in <module>
    from uncompyle6.parsers.treenode import SyntaxTree
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\uncompyle6\parsers\treenode.py", line 3, in <module>
    from uncompyle6.scanners.tok import NoneToken
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\uncompyle6\scanners\tok.py", line 163, in <module>
    NoneToken = Token("LOAD_CONST", offset=-1, attr=None, pattr=None)
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\uncompyle6\scanners\tok.py", line 59, in __init__
    from xdis.std import _std_api
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\xdis\std.py", line 214, in <module>
    _std_api = make_std_api()
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\xdis\std.py", line 212, in make_std_api
    return _StdApi(python_version, variant)
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\xdis\std.py", line 72, in __init__
    self.opc = opc = get_opcode_module(python_version, variant)
  File "c:\users\laike9m\appdata\local\programs\python\python37\lib\site-packages\xdis\op_imports.py", line 156, in get_opcode_module
    return op_imports[canonic_python_version[vers_str]]
KeyError: '3.7.5'

Self-referential pyproject.toml

Description

Self-referential pyproject.toml breaks Poetry dependency manager.

How to Reproduce

  • have pyproject.toml with xdis = "^6.1.0"
  • run poetry lock

Output Given

Package 'xdis' is listed as a dependency of itself.

Expected behavior

Successful project dependency resolution.

Environment

Windows 10/Ubuntu, poetry 1.8.2

Workarounds

Rebuild xdis without line 17 in pyproject.toml.

make_std_api has examples showing float input; requires sys.version_info

From the code:

    from xdis.std import make_std_api
    dis = make_std_api(2.4)
    # dis can now disassemble code objects from python 2.4

In practice:

>>> from xdis.std import make_std_api
>>> dis = make_std_api(2.4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/6dsfg70znajh7h5x80pgk7adc5v4ayw4-python2.7-xdis-3.8.7/lib/python2.7/site-packages/xdis/std.py", line 208, in make_std_api
    return _StdApi(python_version, variant)
  File "/nix/store/6dsfg70znajh7h5x80pgk7adc5v4ayw4-python2.7-xdis-3.8.7/lib/python2.7/site-packages/xdis/std.py", line 72, in __init__
    self.opc = opc = get_opcode_module(python_version, variant)
  File "/nix/store/6dsfg70znajh7h5x80pgk7adc5v4ayw4-python2.7-xdis-3.8.7/lib/python2.7/site-packages/xdis/op_imports.py", line 116, in get_opcode_module
    vers_str = '.'.join([str(v) for v in version_info[0:3]])
TypeError: 'float' object has no attribute '__getitem__'

op_imports.get_opcode_module raises KeyError on Python 3.8.5

Attempting to decompile any pyc file on Python 3.8.5 using uncompyle6 errors with this message.

Traceback (most recent call last):
  File "/usr/local/bin/uncompyle6", line 5, in <module>
    from uncompyle6.bin.uncompile import main_bin
  File "/usr/local/lib/python3.8/site-packages/uncompyle6/__init__.py", line 53, in <module>
    import uncompyle6.semantics.pysource
  File "/usr/local/lib/python3.8/site-packages/uncompyle6/semantics/pysource.py", line 141, in <module>
    from uncompyle6.parsers.treenode import SyntaxTree
  File "/usr/local/lib/python3.8/site-packages/uncompyle6/parsers/treenode.py", line 3, in <module>
    from uncompyle6.scanners.tok import NoneToken
  File "/usr/local/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 200, in <module>
    NoneToken = Token("LOAD_CONST", offset=-1, attr=None, pattr=None)
  File "/usr/local/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 86, in __init__
    from xdis.std import _std_api
  File "/usr/local/lib/python3.8/site-packages/xdis/std.py", line 220, in <module>
    _std_api = make_std_api()
  File "/usr/local/lib/python3.8/site-packages/xdis/std.py", line 218, in make_std_api
    return _StdApi(python_version, variant)
  File "/usr/local/lib/python3.8/site-packages/xdis/std.py", line 73, in __init__
    self.opc = opc = get_opcode_module(python_version, variant)
  File "/usr/local/lib/python3.8/site-packages/xdis/op_imports.py", line 167, in get_opcode_module
    return op_imports[canonic_python_version[vers_str]]
KeyError: '3.8.5'

get_opcode_module fails to handle versions with 3 numbers

Unclear how it is supposed to work.
get_opcode_module fails to handle 3.6.12|3.8.6, and likely every 3.6.n|3.8.n.

$ uncompyle6 -h
Traceback (most recent call last):
  File "/usr/bin/uncompyle6", line 11, in <module>
    load_entry_point('uncompyle6==3.7.2', 'console_scripts', 'uncompyle6')()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2852, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.8/site-packages/uncompyle6/__init__.py", line 48, in <module>
    import uncompyle6.semantics.pysource
  File "/usr/lib/python3.8/site-packages/uncompyle6/semantics/pysource.py", line 141, in <module>
    from uncompyle6.parsers.treenode import SyntaxTree
  File "/usr/lib/python3.8/site-packages/uncompyle6/parsers/treenode.py", line 3, in <module>
    from uncompyle6.scanners.tok import NoneToken
  File "/usr/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 200, in <module>
    NoneToken = Token("LOAD_CONST", offset=-1, attr=None, pattr=None)
  File "/usr/lib/python3.8/site-packages/uncompyle6/scanners/tok.py", line 86, in __init__
    from xdis.std import _std_api
  File "/usr/lib/python3.8/site-packages/xdis/std.py", line 220, in <module>
    _std_api = make_std_api()
  File "/usr/lib/python3.8/site-packages/xdis/std.py", line 218, in make_std_api
    return _StdApi(python_version, variant)
  File "/usr/lib/python3.8/site-packages/xdis/std.py", line 73, in __init__
    self.opc = opc = get_opcode_module(python_version, variant)
  File "/usr/lib/python3.8/site-packages/xdis/op_imports.py", line 169, in get_opcode_module
    return op_imports[canonic_python_version[vers_str]]
KeyError: '3.8.6'

Workaround is version_info[0:2] to find a match.

Unclear what the openSUSE Leap and Tumbleweed packages do wrong.

Documentation for using xdis as a library?

Description

Is there any API documentation for xdis? If not it would be useful to provide some information about how to use it as a library.

Background

I looked for API documentation but couldn't find any. Apologies in advance if I missed something obvious.

pytest/test_disasm.py is broken under Python 2.7

 % python setup.py test                                                                                                                                      test-tweaks
running pytest
running egg_info
writing dependency_links to xdis.egg-info/dependency_links.txt
writing xdis.egg-info/PKG-INFO
writing top-level names to xdis.egg-info/top_level.txt
reading manifest file 'xdis.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'xdis.egg-info/SOURCES.txt'
running build_ext
========================================================================== test session starts ==========================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /usr/local/google/home/ksp/src/python-xdis, inifile: 
collected 13 items 

pytest/test_bytecode.py .
pytest/test_disasm.py .F.
pytest/test_load_file.py .
test_unit/test_dis.py ....
test_unit/test_load.py .
test_unit/test_magic.py .
test_unit/test_marsh.py .
test_unit/test_opcode.py F

=============================================================================== FAILURES ================================================================================
____________________________________________________________ test_funcoutput[test_tuple1-function_to_test1] _____________________________________________________________

capfd = <_pytest.capture.CaptureFixture instance at 0x7f9c9c4446c8>, test_tuple = ('../test/bytecode_3.0/04_raise.pyc', 'testdata/raise-3.0.right')
function_to_test = <function disassemble_file at 0x7f9c9c527578>

    @pytest.mark.parametrize(("test_tuple", "function_to_test"), [
        (
            ('../test/bytecode_3.6/01_fstring.pyc', 'testdata/fstring-3.6.right',),
            disassemble_file,
        ),
        (
            ('../test/bytecode_3.0/04_raise.pyc', 'testdata/raise-3.0.right',),
            disassemble_file,
        ),
        (
            ('../test/bytecode_pypy2.7/04_pypy_lambda.pyc', 'testdata/pypy_lambda.right',),
            disassemble_file,
        ),
    ])
    
    def test_funcoutput(capfd, test_tuple, function_to_test):
        in_file, filename_expected = test_tuple
        resout = StringIO()
        function_to_test(in_file, resout)
        expected = "".join(open(filename_expected, "r").readlines())
        got_lines = resout.getvalue().split("\n")
        got_lines = [re.sub(' at 0x[0-9a-f]+, file', ' at 0xdeadbeef0000, file', line)
                     for line in got_lines]
        got = "\n".join(got_lines[5:])
    
        if got != expected:
            with open(filename_expected + ".got", "w") as out:
                out.write(got)
>       assert got == expected
E       assert '# Method Nam...URN_VALUE\n\n' == '# Method Name...URN_VALUE\n\n'
E         Skipping 228 identical leading characters in diff, use -v to show
E           
E         - #    0: <xdis.code.Code3 instance at 0x7f9c9c4444d0>
E         + #    0: <code object compact_traceback at 0xdeadbeef0000, file "simple_source/stmts/04_raise.py", line 6>
E           #    1: None
E           # Names:
E           #    0: compact_traceback
E         -   6           0 LOAD_CONST               0 (<xdis.code.Code3 instance at 0x7f9c9c4444d0>)
E         +   6           0 LOAD_CONST               0 (<code object compact_traceback at 0xdeadbeef0000, file "simple_source/stmts/04_raise.py", line 6>)
E         Detailed information truncated (38 more lines), use "-vv" to show

test_disasm.py:49: AssertionError

Under Python 3.4 the test is working fine.

Dissasembling failing in xasm format

Hi,

I'm currently trying to extract the bytecode, edit a few strings and assemble it back to a .pyc file.
Pydisasm without any flags work just fine but as soon as I try to dissamble the file with Pydisasm -F xasm ./file.pyc it fails with the following traceback:

Traceback (most recent call last):
  File "/usr/local/bin/pydisasm", line 33, in <module>
    sys.exit(load_entry_point('xdis', 'console_scripts', 'pydisasm')())
  File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/root/python-xdis/xdis/bin/pydisasm.py", line 72, in main
    disassemble_file(path, sys.stdout, format)
  File "/root/python-xdis/xdis/disasm.py", line 329, in disassemble_file
    disco(
  File "/root/python-xdis/xdis/disasm.py", line 160, in disco
    disco_loop_asm_format(opc, version_tuple, co, real_out, {}, set([]))
  File "/root/python-xdis/xdis/disasm.py", line 220, in disco_loop_asm_format
    disco_loop_asm_format(
  File "/root/python-xdis/xdis/disasm.py", line 220, in disco_loop_asm_format
    disco_loop_asm_format(
  File "/root/python-xdis/xdis/disasm.py", line 249, in disco_loop_asm_format
    assert mapped_name not in fn_name_map
AssertionError

I also printed out the vars from the assert:

mapped_name='listcomp_0x7f3d301932f0'

fn_name_map={'listcomp_0x7f3d30192ff0': 'listcomp', 'listcomp_0x7f3d301932f0': 'listcomp'}

Python 3.10 support

Most of the changes are simple enough and are in branch 3.10-adjust.

The is a long-standing design flaw in converting a python version into a floating point number and using that in comparisons to check whether we are running or disassembling particular sets of versions. With 3.10 this no longer works because 3.10 is treated as the same as 3.1.

The change to use a tuple instead of a float is pretty straight-forward. However it is pervasive.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.