Giter Club home page Giter Club logo

x64asm's Introduction

x64asm

x64asm is a c++11 library for working with x86_64 assembly. It provides a parser, in-memory assembler and linker, and primitives for building data flow analyses. x64asm was built with the following design goals in mind:

  • Simplicity: x64 asm does NOT include a register allocator, instruction scheduler, control flow graph builder, or any of the features you would expect of a full compiler. It is a low-level library for building YOUR optimizing compiler.

  • Completeness: x64asm supports the entire ring 3 application level subset of the x86_64 instruction set, including the most recent AVX2/BMI1/BMI2/FMA extensions.

  • Correctness: The majority of the source in the x64asm repository is auto-generated using a declarative specification. This means that bugs can be fixed quickly without major modifications to its codebase.

Supported Platforms:

  • Ubuntu 12.04 LTS
  • Ubuntu 13.10
  • Ubuntu 14.04 LTS

Getting started:

Dependencies are available through apt-get.

$ sudo apt-get install bison ccache doxygen flex g++ g++-multilib ghc libghc-regex-tdfa-dev libghc-regex-compat-dev libghc-split-dev

To build x64asm, type:

$ make (release|profile|debug)

For examples of how to use the library, browse the examples/ folder:

  • abi.cc How the assembler API interacts with the linux ABI.
  • constants.cc Shows off the set of built-in assembler constants.
  • context.cc How to assemble functions that modify program state.
  • dataflow.cc How to use the dataflow API.
  • functions.cc How to assemble functions that call other functions.
  • hello.cc How to use the assembler API; look here first.
  • linker.cc How to assemble functions with external linkage.

And to use x64asm as an assembler from the command line, type:

$ cat test.s | <path/to/here>/bin/asm 

To use x64asm as part of a larger project, include the header:

#include "<path/to/here>/include/x64asm.h"

int main() {
  // ...
  return 0;
}

and link against the library:

$ g++ code.cc -I<path/to/here> <path/to/here>/lib/x64.a

Undefined Assembler Behavior

Jumps to undefined labels are handled by emitting a 32-bit relative displacement of 0x00000000.

Assembler Simplifications

Deciding between the 8- and 32-bit relative displacement forms of jump instructions is known as the (NP hard) branch displacement problem. The primary consequence of this decision is code size. Most compilers solve this problem using an iterative algorithm which initially assumes 8-bit displacements and then adjusts as necessary. We emit all jumps to labels using the 32-bit form.

Memory Types

In many cases, the only thing distinguishing two otherwise identical instructions is operand type. Furthermore, certain operand types (ie. M16) are required for infering prefix bytes. We account for this by introducing a distinct memory type for each operand type appearing in the Intel manual. Barring these requirements, a single memory type would simplify our implementation.

Ambiguity

The x86_64 instruction set contains many instructions that are indistinguishable up to mnemonic and operand. These ambiguities represent a distinction without a difference. They are alternate hardware methods for performing the same operation, which as far as we know, have no noticable performance tradeoffs. We remove this redudancy by choosing the encoding preferred by g++.

Intel Eratta

Most of the source code in this project is automatically generated using the the x64.csv spreadsheet. Unless otherwise noted below, the contents of the spreadsheet are transcribed directly from the Intel manuals. If you discover an error, or an edit which has not been documented below, please submit an error report.

String Instructions

Each instruction in this class takes an optional memory argument(s) which is (are) ignored except for the purpose of distinguishing operand width, ie: CMPS M8 = CMPSB, and for specifying a memory address prefix (67), which affects the implicit memory operand. Note that for the short form of these instructions, we add a PREF.66+ annotation, as there is no operand to induce this value.

Although in principle, the short form versions of these instructions can be modified by the addition of a memory address prefix (67), we ignore this possibility and require the user to use the long form.

The Intel manual omits any mention of the memory address prefix (67) for the LODS class of instructions. This is likely a typo, and we support the behavior per the other instructions in this class.

CWD/CDQ/CQO/POPF/PUSHF

Each of the instructions in this class require a word prefix (66) to distinguish the word version of the instruction from the otherwise identical long version. Because there are no operands to infer this prefix, we add it explicitly.

POPA/PUSHA

Each of the instructions in this class would have a similar problem to the above. However, as these are unsupported in 64-bit mode, we ignore these.

IRET/IRETD/IRETQ

Each of the instructions in this class APPEAR as though they would have a similar problem to the above, but don't. We've left them unmodified.

LEAVE/MOV/POP/RET/SYSEXT/SYSRET

Each of the instructions in this class suffer from ambiguity due to lack of mnemonic variation. Furthermore, some of the instructions in this class, even if they could be disambiguated by mnemonic, require prefix bytes which cannot be infered due to lack of operand. We add both an explicit annotation and operand to distinguish these cases. The Intel manual is unclear on what adding the REX.w+ prefix to MOV [mem8] AL accomplishes, but we support this behavior nonetheless.

XLATB

The rex.w prefix is a meaningless distinction in 64-bit mode. As a result, we do not support a distinction between the latter two variants of this instruction.

REP/REPE/REPZ/REPNE/REPNZ

The REX.w prefix has no control over count register for these instructions. It functions simply to disambiguate the 64-bit operand version of some versions of the instruction. Count register width is controled by the address width prefix which follows from the long form version of each instruction. Note that in contrast to the non-repeated versions of these instructions, no short form is given.

In either case, where the REX.w prefix isn't used to disambiguate a 64 bit operand form, it's use is meaningless. For these rows, any arbitrary choice of encoding will suffice.

Note that we have also added underscores to these mnemonics to maintain the one-word-per-mnemonic invariant.

Dataflow Values

Some dataflow information is missing and/or incomplete. If you discover an error, please submit a bug report.

Further documenation

See SPREADSHEET.md for the tiny bit of documentation on the internals that we have, which is focused on the x86.csv file.

x64asm's People

Contributors

bchurchill avatar coruus avatar eschkufz avatar jrkoenig avatar movingtomars avatar njoubert avatar sdasgup3 avatar stefanheule avatar uplinkcoder avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

x64asm's Issues

Assembler Bug

xorl %edx, %edx (segfaults when executed; extra 0x02)

Rename Rb to R8

Low priority --

Now that Rh and Rl are subclasses of Rb, it would be nice to rename Rb to reflect the symmetry with the rest of the general purpose register classes.

If we do this, we should leave a typedef from R8 to Rb just to preserve backwards compatibility for dependent code.

Disagreement with g++ for vmov variants

Can't fix with the code in its current state.
g++ selects variants based on the rex prefix of its arguments which the type system is unable to represent.

These instructions are correct, just not as short as possible.

Disagreement with g++ hex

Complete listing of opcode with this issue is below --

  • bswap
  • call with relative offset
  • jump with relative offset
  • loop
  • push imm16
  • vcvtdq2pd ymm ymm
  • vmov variants
  • xchg with rax operand

Cleanup regset >>

At some point I overloaded RegSet's stream operator before I knew that RegSetWriter was the best place to put that stuff. I'll clean this up sometime soon.

Transition away from spreadsheet

Spreadsheet has non-ascii characters in it, and manually exporting to .csv every time we make a bug fix is just awful.

Let's pick a more readable/editable tab formatting and edit the .csv directly from now on.

linux_caller_save() doesn't match ABI

I don't want to break anything, so I won't change this function quite yet. The name of these function doesn't quite match up to what they are though: linux_caller_save() currently returns the list of all registers used as parameters; however, the caller must also save rax, r9, r10 and r11, which aren't parameters.

I also have a really hard time with the names callee/caller save, but that's probably just me, so my current solution is to define four new additional functions:
linux_call_parameters()
linux_call_scratch()
linux_call_preserved()
linux_call_return() (including rax, rdx, xmm0, xmm1)

that return the appropriate sets of registers.

Memory operand printing

From jason koenig --

Stoke generates instructions like:
leaw -0x1(%rax,8), %r11w
which are ill-formed according to the "att" parser because they lack a base register. GNU as gives an warning for this code. It should be:
leaw -0x1(,%rax,8), %r11w
This can be fixed by changing line 91 in x64asm/src/m.cc:
if (contains_base() && contains_index()) {
to
if (contains_index()) {
i.e. the comma needs to be printed if there is an index register/multiplier regardless of whether there is a base.

Build problems on Ubuntu 13.10

The presence of Unicode non-ASCII codepoints caused building to fail on Ubuntu 13.10 with default package versions. (That is, system built according to instructions in repo.) See pull request #24.

Add functions to get the set of read/written registers to Code class

For my work I needed to get the set of possibly read and possibly written registers from a Code class. For the moment, it's implemented in my stoke branch, but it's somewhat out of place. Do you think it would make sense to add such functions here?

They're truly short functions, just 5/6 lines each.

/bin/stoke has trouble counting number of binaries

I'm having trouble with /bin/stoke because it always counts the number of binaries, which is different depending on whether or not the test suite has been built. It also differs between branches. In this branch, I remove this sanity check.

Also, make clean needs to delete all binaries besides /bin/stoke in the /bin folder, so that binaries from one branch aren't present when working in the other.

adding is_call method to Instruction class

(Berkeley is taking ownership of this). I'm adding functionality to see if a given instruction is a call (e.g. call or syscall). This means having codegen make a new table and having the x64asm library slurp it in. Creating a new branch based on this ticket number.

It's finally time for a linker

STOKE needs a linker now that we're ready to start sandboxing functions that use the call instruction.

I think this is actually really easy. All we need to do is store a symbol table in functions rather in the assembler. (And for our purposes, all the symbol table is is a mapping from labels to code offsets and jump locations). If a function has that, it can figure out the global offset for labels because it knows the offset that its buffer begins at.

From there, all a linker will have to do redo the offsets.

Assuming you're only working with a single function, the assembler should continue to work exactly as it used to.

Hex disagreement for arithmetic ALU ops

g++ prefers register immediate rather than specialized for rax.
See output of make check.

This isn't technically a bug.
It would just be nice to match the output of g++ exactly.

Invalid instruction variants allowed

The x64asm type system is insufficient for preventing users from requesting instruction such as:

crc32 %r12, %ah

(This shouldn't be allowed due to %r12 requiring a rex prefix.)

This occurs for other instructions in similar situations, such as when a memory operand requires an rex prefix. Fixing this issue will require a major re-architecting. I'd prefer to stay away from this is possible.

Known opcodes with this issue --

  • crc32
  • movsx
  • movzx

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.