vector35 / dwarf_import Goto Github PK

This loads DWARF info from an open binary and propagates function names, arguments, and type info

License: MIT License

Python 100.00%

binary-reversing binaryninja binaryninja-plugin debug debugging debugging-tool debugging-tools debugview dwarf elf-binaries elf-format elf-loader elf-parser reverse-engineering reversing

dwarf_import's People

Contributors

Stargazers

Watchers

Forkers

whitequark endeav0r gengrill hack-different matteyeux

dwarf_import's Issues

dwarf_export

Hello.

BN - one of the best platform for linux pwn and reverse engineering.
I spent much time trying to make something like "reverse unstripping" for elf binary.

Both ghidra and ida have solutions - ghidra2dwarf and dwarfexport respectively.
Firs one doesn't handle large (7mb) elfs, second - doesn't support last versions of IDA.
I was able to made adaptation of last one for idasdk 7.5, but only for linux. I don't have IDA PRO for linux.

So, fact: the best decompiled, human readable code I achieve with BN. Speaking true - I do some cheating with importing some C++ symbols from IDA to BN, but finally - BN gives me the best result.

So, my question is: how to EXPORT (generate) dwarf from BN project?

With respect.

AssertionError on a global variable with no type attribute

I was trying to apply DWARF info for the attached binary and the plugin suffered an assertion failure when importing a global variable. It was trying to import a DW_TAG_variable die with no DW_AT_type attribute. In the source for this binary, the variable that it was trying to import is an extern const char *.

Here's the output of llvm-dwarfdump. Based on added logging in the plugin, the assertion error happened when it was trying to inspect the DW_TAG_variable at 0x25.

0x0000001d:   DW_TAG_variable
                DW_AT_name  ("buildstring")
                DW_AT_decl_file ("/tmp/Darkplaces.build.linux64/build-obj/release/darkplaces-dedicated/../../../builddate.c")
                DW_AT_decl_line (4)
                DW_AT_decl_column   (0x14)
                DW_AT_external  (true)
                DW_AT_declaration   (true)

0x00000025:   DW_TAG_variable
                DW_AT_specification (0x0000001d "buildstring")
                DW_AT_decl_line (5)
                DW_AT_decl_column   (0x0d)
                DW_AT_location  (DW_OP_addr 0x4d0490)

Traceback for the plugin:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.9.13_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/Applications/Binary Ninja.app/Contents/MacOS/plugins/../../Resources/python/binaryninja/plugin.py", line 928, in run
    self.task.run()
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/__init__.py", line 41, in run
    analysis_session = AnalysisSession(binary_view = self.view, debug_file = self.debug_file)
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/mapped_model.py", line 67, in __init__
    m = AnalysisModel.from_dwarf(debug_source, self.debug_root, name=binary_name, logger=logger)
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/model/analysis_model.py", line 297, in from_dwarf
    import_ELF_DWARF_into_model(elf_file, model, debug_root=debug_root, logger=logger)
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 1468, in import_ELF_DWARF_into_model
    importer.import_debug_info()
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 671, in import_debug_info
    self.import_compilation_unit(cu_die)
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 689, in import_compilation_unit
    self.import_global_variable(child)
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 765, in import_global_variable
    assert(type_die)
AssertionError
Traceback (most recent call last):
  File "/Applications/Binary Ninja.app/Contents/MacOS/plugins/../../Resources/python/binaryninja/plugin.py", line 229, in _default_is_valid
    return is_valid(view_obj)
  File "/Users/scott/Library/Application Support/Binary Ninja/plugins/dwarf_import/__init__.py", line 92, in is_valid
    reader = BinaryReader(bv.file.raw)
  File "/Applications/Binary Ninja.app/Contents/MacOS/plugins/../../Resources/python/binaryninja/binaryview.py", line 7908, in __init__
    _handle = core.BNCreateBinaryReader(view.handle)
AttributeError: 'NoneType' object has no attribute 'handle'

Another issue I noticed is that when the plugin failed, bv.file.raw becomes a None type not a BinaryView as intended. The AttributeError in the provided traceback shows this.

Binary in question: xonotic-linux64-dedicated.zip

remove deprecation warning

When loading a binary in Binary Ninja after installing this plugin:

/home/lockbox/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/__init__.py:85: DeprecatedWarning: __len__ is deprecated. Use .length instead. Python disallows the length of an object to >= 0x8000000000000000. See https://bugs.python.org/issue21444.
  if not bv.parent_view:

changing this line to

if not bv.parent_view.length:

removes this issue

support for line numbers

support for comments

Errors when parsing binary compiled with DWARFv5

Sample Binary
test.tar.gz
Error:

During parsing:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Applications/Binary Ninja.app/Contents/MacOS/plugins/../../Resources/python/binaryninja/plugin.py", line 860, in run
    self.task.run()
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/__init__.py", line 37, in run
    analysis_session = AnalysisSession(binary_view = self.view, debug_file = self.debug_file)
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/mapped_model.py", line 65, in __init__
    m = AnalysisModel.from_dwarf(debug_source, self.debug_root, name=binary_name, logger=logger)
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/model/analysis_model.py", line 271, in from_dwarf
    import_ELF_DWARF_into_model(elf_file, model, debug_root=debug_root, logger=logger)
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 1486, in import_ELF_DWARF_into_model
    dwarf_db = DWARFDB(elf_file, debug_root)
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 263, in __init__
    self._pri = DWARFData(elf_file, debug_root)
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 182, in __init__
    self._index()
  File "/Users/kyle/Library/Application Support/Binary Ninja/plugins/dwarf_import/io/dwarf_import.py", line 185, in _index
    for cu in self._dwarf_info.iter_CUs():
  File "/Users/kyle/Library/Python/3.8/lib/python/site-packages/elftools/dwarf/dwarfinfo.py", line 353, in _parse_CUs_iter
    cu = self._cached_CU_at_offset(offset)
  File "/Users/kyle/Library/Python/3.8/lib/python/site-packages/elftools/dwarf/dwarfinfo.py", line 383, in _cached_CU_at_offset
    cu = self._parse_CU_at_offset(offset)
  File "/Users/kyle/Library/Python/3.8/lib/python/site-packages/elftools/dwarf/dwarfinfo.py", line 417, in _parse_CU_at_offset
    cu_structs = DWARFStructs(
  File "/Users/kyle/Library/Python/3.8/lib/python/site-packages/elftools/dwarf/structs.py", line 92, in __init__
    assert address_size == 8 or address_size == 4
AssertionError

After parsing (while the UI checks if the callback is valid or not):

Traceback (most recent call last):
  File "/home/user/binaryninja/plugins/../python/binaryninja/plugin.py", line 225, in _default_is_valid
    return is_valid(view_obj)
  File "/home/user/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/__init__.py", line 81, in is_valid
    for view in bv.file.raw.available_view_types:
AttributeError: 'NoneType' object has no attribute 'available_view_types'

Feature: dSYM support

For macOS platforms dSYM bundles are just pretty wrappers on DWARF files - which you can use with the full path, but difficult from the UI as the file picker does not show the contents of the bundle.

If the selected path is a folder ending .dSYM appending the path Contents/Resources/DWARF/<file> would be the path for the file.

In another universe it could also be really interesting to provide connection to the files in the python directory as they are lldb extensions that provide additional debugging functionality that can be used

Feature request: Mach-O support

Right now the plugin only supports ELF binaries, but Mach-O binaries can also contain DWARF debug info. It would be useful to support those as well.

AttributeError trying to apply on latest dev

Dev 3.5.4286, stock Ubuntu 20.04 and installed from plugin manager

Trying to use DWARF Import > Load DWARF Symbols on a simple C Linux ELF file throws a bunch of errors in the python console and fails to work. Binary available upon request

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/vector35/binaryninja/plugins/../python/binaryninja/plugin.py", line 928, in run
    self.task.run()
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/__init__.py", line 41, in run
    analysis_session = AnalysisSession(binary_view = self.view, debug_file = self.debug_file)
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/mapped_model.py", line 67, in __init__
    m = AnalysisModel.from_dwarf(debug_source, self.debug_root, name=binary_name, logger=logger)
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/model/analysis_model.py", line 297, in from_dwarf
    import_ELF_DWARF_into_model(elf_file, model, debug_root=debug_root, logger=logger)
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/io/dwarf_import.py", line 1466, in import_ELF_DWARF_into_model
    dwarf_db = DWARFDB(elf_file, debug_root)
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/io/dwarf_import.py", line 263, in __init__
    self._pri = DWARFData(elf_file, debug_root)
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/io/dwarf_import.py", line 182, in __init__
    self._index()
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/io/dwarf_import.py", line 185, in _index
    for cu in self._dwarf_info.iter_CUs():
  File "/home/vector35/.binaryninja/python38/site-packages/elftools/dwarf/dwarfinfo.py", line 353, in _parse_CUs_iter
    cu = self._cached_CU_at_offset(offset)
  File "/home/vector35/.binaryninja/python38/site-packages/elftools/dwarf/dwarfinfo.py", line 383, in _cached_CU_at_offset
    cu = self._parse_CU_at_offset(offset)
  File "/home/vector35/.binaryninja/python38/site-packages/elftools/dwarf/dwarfinfo.py", line 417, in _parse_CU_at_offset
    cu_structs = DWARFStructs(
  File "/home/vector35/.binaryninja/python38/site-packages/elftools/dwarf/structs.py", line 92, in __init__
    assert address_size == 8 or address_size == 4
AssertionError
Traceback (most recent call last):
  File "/home/vector35/binaryninja/plugins/../python/binaryninja/plugin.py", line 229, in _default_is_valid
    return is_valid(view_obj)
  File "/home/vector35/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/__init__.py", line 92, in is_valid
    reader = BinaryReader(bv.file.raw)
  File "/home/vector35/binaryninja/plugins/../python/binaryninja/binaryview.py", line 8196, in __init__
    _handle = core.BNCreateBinaryReader(view.handle)
AttributeError: 'NoneType' object has no attribute 'handle'
Exception ignored in: <function BinaryReader.__del__ at 0x7fa4fb5a55e0>
Traceback (most recent call last):
  File "/home/vector35/binaryninja/plugins/../python/binaryninja/binaryview.py", line 8209, in __del__
    core.BNFreeBinaryReader(self._handle)
AttributeError: 'BinaryReader' object has no attribute '_handle'

Feature Request: Load DWARF at specific function

A common pain point in my workflow is not being able to use DWARF for very large files (>100MB). Often I only need it for a specific function, as I am ignorant of the DWARF spec, would it be possible to add a feature to only enable loading dwarf at a specific function? Thanks!

Canabalizes your BV if you run it twice

We should probably make sure that it hasn't been run before running and re-applying info, or fix the more fundamental issue.

Broken on pyelftools 0.27

Error on import:

Failed to import python plugin: official/Vector35_dwarf_import: cannot import name 'GenericExprVisitor' from 'elftools.dwarf.dwarf_expr' (/usr/lib/python3.8/site-packages/elftools/dwarf/dwarf_expr.py)

Downgrading to 0.26 fixes this.

TypeError: BinaryReader.seek() takes 2 positional arguments but 3 were given

Traceback (most recent call last):
  File "/opt/binaryninja/plugins/../python/binaryninja/plugin.py", line 229, in _default_is_valid
    return is_valid(view_obj)
  File "/home/user/.binaryninja/repositories/official/plugins/Vector35_dwarf_import/__init__.py", line 86, in is_valid
    return raw and elf and ELFFile(bn.binaryview.BinaryReader(bv.file.raw)).has_dwarf_info()
  File "/home/user/.binaryninja/python310/site-packages/elftools/elf/elffile.py", line 81, in __init__
    self.stream.seek(0, io.SEEK_END)
TypeError: BinaryReader.seek() takes 2 positional arguments but 3 were given

Binary Ninja Version: 3.1.3469 Personal, 93650e26
Platform: Arch Linux

Unable to run on 2.4.3105-dev: Attribute Error

Executing the dwarf_import plugin on version 2.4.3105-dev leads to the following error `AttributeError: module 'binaryninja' has no attribute 'NamedTypeReference'. The full traceback is bellow.

Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\lagle\AppData\Local\Programs\Python\Python39\Lib\threading.py", line 954, in _bootstrap_inner
    self.run()
  File "C:\Users\lagle\AppData\Local\Vector35\BinaryNinja\plugins\..\python\binaryninja\plugin.py", line 808, in run
    self.task.run()
  File "C:\Users\lagle\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\__init__.py", line 42, in run
    bridge.import_debug_info()
  File "C:\Users\lagle\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\bridge.py", line 316, in import_debug_info
    self.apply_all_types()
  File "C:\Users\lagle\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\bridge.py", line 328, in apply_all_types
    self._translate_type(ty)
  File "C:\Users\lagle\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\bridge.py", line 566, in _translate_type
    binja_type = self._construct_binja_type(ty)
  File "C:\Users\lagle\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\bridge.py", line 717, in _construct_binja_type
    binja_type = bn.Type.named_type(bn.NamedTypeReference(name=ty.name, type_id=self._generate_typeid(ty.name), type_class=ntrc))
AttributeError: module 'binaryninja' has no attribute 'NamedTypeReference'

`AttributeError: 'BinaryReader' object has no attribute 'tell'`

Traceback (most recent call last):
  File "/nix/store/j2g1lns927px777irnraxk63fh91kppy-binary-ninja-personal/opt/plugins/../python/binaryninja/plugin.py", line 227, in _default_is_valid
    return is_valid(view_obj)
  File ".../.binaryninja/repositories/official/plugins/Vector35_dwarf_import/__init__.py", line 60, in is_valid
    return raw and elf and ELFFile(bn.binaryview.BinaryReader(bv.file.raw)).has_dwarf_info()
  File ".../.binaryninja/python39/site-packages/elftools/elf/elffile.py", line 182, in has_dwarf_info
    return bool(self.get_section_by_name('.debug_info') or
  File ".../.binaryninja/python39/site-packages/elftools/elf/elffile.py", line 124, in get_section_by_name
    for i, sec in enumerate(self.iter_sections()):
  File ".../.binaryninja/python39/site-packages/elftools/elf/elffile.py", line 133, in iter_sections
    yield self.get_section(i)
  File ".../.binaryninja/python39/site-packages/elftools/elf/elffile.py", line 113, in get_section
    return self._make_section(section_header)
  File ".../.binaryninja/python39/site-packages/elftools/elf/elffile.py", line 569, in _make_section
    return ARMAttributesSection(section_header, name, self)
  File ".../.binaryninja/python39/site-packages/elftools/elf/sections.py", line 474, in __init__
    self.subsec_start = self.stream.tell()
AttributeError: 'BinaryReader' object has no attribute 'tell'

The error only occurs when an Arm ELF file is opened. It doesn't occur with x86.

Systen information

NixOS Unstable (The problem can be reproduced on macOS, too)
Binary Ninja Version 2.4.2846 Personal
DWARF Import v1.0.4 (installed through "Manage Plugins")
pyelftools 0.27

AttributeError: 'BinjaBridge' object has no attribute '_create_global_component'

Windows 10 Personal 2480-dev

Tools -> DWARF Import gives the following error

Exception in thread Thread-2:
Traceback (most recent call last):
  File "C:\Python39\Lib\threading.py", line 950, in _bootstrap_inner
    self.run()
  File "C:\Users\x\AppData\Local\Vector35\BinaryNinja\plugins\..\python\binaryninja\plugin.py", line 731, in run
    self.task.run()
  File "C:\Users\x\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\__init__.py", line 42, in run
    bridge.import_debug_info()
  File "C:\Users\x\AppData\Roaming\Binary Ninja\repositories\official\plugins\Vector35_dwarf_import\bridge.py", line 304, in import_debug_info
    self._module.add_component(self._create_global_component())
AttributeError: 'BinjaBridge' object has no attribute '_create_global_component'

Support for pyelftools 0.29 (parsing DWARFv5)

In the latest version of pyelftools they added support for DWARFv5 (see eliben/pyelftools#391) but unfortunately this plugin only uses pyelftools 0.27
Can you please add support for pyelftools 0.29? There are some issues with that which must be solved (see #18, #24, #16)

Error on processing some linux files for is_valid

Unsure what source of this error is, not sure whether this is just outdated or if API in elftools has changed

Traceback (most recent call last):
  File "...\binaryninja\plugin.py", line 229, in _default_is_valid
    return is_valid(view_obj)
  File "...plugins\analysis_dwarf\__init__.py", line 106, in is_valid
    return raw and elf and ELFFile(reader).has_dwarf_info()
  File "...site-packages\elftools\elf\elffile.py", line 81, in __init__
    self.stream.seek(0, io.SEEK_END)
TypeError: BinaryReader.seek() takes 2 positional arguments but 3 were given

analysis_dwarf is renamed directory of dwarf_import, and this error triggers on loading elf-Linux-x64-bash

elf-Linux-x64-bash.zip

pyelftools==0.28

Better error message needed for stripped binaries

Generates the following error: AttributeError: 'BinjaBridge' object has no attribute '_create_global_component'

Change print statement

dwarf_import/location_index.py

Line 178 in 272360a

print(f'NO SUCH REG {expr} - {arch.name}')

Should this be sent to the logger instead of printed to stdout? It is kind of annoying headless.

debugging information in separate files

Is there a way to use the ubuntu-style debugging information (located in separate files) using this plugin or BinaryNinja itself?

I have many executables that ship with separate -dbg*.deb that I would like to load and did not find a way in BinaryNinja itself.

support for PE files

`requirements.txt` contains incorrect name for the package `pyelftools`

While the importable library is called elftools, the package is called pyelftools.

Installing the dependencies of this package via pip install -r requirements.txt is therefore broken.
This also affects installing dependencies automatically via the BinaryNinja plugin manager.

Support for mangled names

Hi Kyle,

I have come across a small hiccup when using this plugin. The DWARF info for the mangled name does not make it to the symbol.

My current workaround is to just get the address and then use objdump to get the mangled name at that address. Not a huge thing, but the mangled name is specifically useful for things like ignorelists.

Thanks!

Stack Variable Resolution

Sometimes

dwarf_import/bridge.py

Line 118 in 43e4f67

offset = loc.expr[1] - self._arch.address_size

needs to be + instead of -. We should try to detect and resolve that.