Giter Club home page Giter Club logo

magma's Introduction

Magma

Documentation Status Linux Test codecov Join the chat at https://gitter.im/Magma-HDL/community

CHEAT SHEET

Magma is a hardware design language embedded in python.

The central abstraction in Magma is a circuit. A circuit is a set of functional units that are wired together. Magma circuits are analagous to verilog modules. Thus, all Magma programs are guaranteed to be synthesizable. Although wiring modules together may seem low-level, it encourages hardware designers to build reusable components, similar to how programmers build libraries.

Python is used to to create Magma circuits. This approach to hardware design using scripting languages is referred to as generators in the hardware community. Example hardware generators include arithmetic units, linear feedback shift registers, wallace trees, and sorting networks. Software engineers refer to this technique as metaprogramming. The scripting language is a metaprogram in the sense that it is a program that creates a hardware program.

In contrast to verilog, Python has powerful metaprogramming capabilities, such as decorators and metaclasses. This makes it possible to create higher-level domain-specific languages (DSLs). Examples include languages for finite state machines, memory controllers, image and signal processing, and even processors.

The best way to learn Magma is through examples. The magma_register_file_tutorial provides an example of defining and verifying a register file generator. The magmathon repository contains a set of Jupyter Notebooks that introduce the system. There's also magma_examples and magma_tutorial which contain a set of basic circuits and tests, providing an example templates for a magma projects. Finally there is magma_riscv_mini which provides an example of a simple RISCV processor. Please also refer to the documentation.

The design of Magma was heavily influenced by Chisel, so Magma should be easy to learn if you know Chisel. Some examples from the Chisel tutorial have been ported to Magma and can be found in here. Magma also has a FIRRTL backend, and we hope to demonstrate interoperability with Chisel via FIRRTL soon.

Magma is designed to work with Mantle which contains an a collection of useful circuits; and with Loam which is used to represent parts and boards, and to build applications for standalone FPGA boards.

Setting up Python

magma requires using Python 3.7+. This section walks through a few common methods for getting set up with Python. The first (recommended) method is to use miniconda which supports MacOS and Linux (and Windows, but magma's support for windows has not been tested). We also provide methods using the standard package managers for MacOS and Ubuntu Linux.

Universal

miniconda - A free minimal installer for conda. Miniconda is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on and a small number of other useful packages, including pip, zlib and a few others. Use the conda install command to install 720+ additional conda packages from the Continuum repository.

Download the installer for your operating system from https://conda.io/miniconda.html.

The following instructions uses the latest MacOSX release, replace the link in the wget command for you operating system.

$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
$ bash Miniconda3-latest-MacOSX-x86_64.sh
# Installing dependencies like matplotlib is typically more reliable with conda than pip
$ conda install matplotlib

MacOS/Homebrew

$ brew install python3

Ubuntu

$ sudo apt-get install python3 python3-pip

Arch

$ pacman -S python python-pip

Installing Dependencies

Magma depends on some third-party libraries that should be installed using your operating system's package manager.

MacOS/Homebrew

$ brew install verilator gmp mpfr libmpc

Ubuntu

$ sudo apt-get install make gcc g++ verilator libgmp-dev libmpfr-dev libmpc-dev cmake

Arch

$ pacman -S verilator gmp mpfr libmpc tk

User Setup

Magma is available as a pip package, install the lastest release with:

pip install magma-lang

Development Setup

Clone the magma repository

$ git clone https://github.com/phanrahan/magma
$ cd magma

Install magma as a symbolic package

$ pip install -e .

Install testing infrastructure and run tests to validate the setup

$ pip install pytest fault  # you may need to add $HOME/.local/bin to your $PATH
$ pytest tests

You should see something like

============================= test session starts ==============================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: ..../repos/magmacore, inifile:
collected 70 items

tests/test_circuit/test_anon.py .
tests/test_circuit/test_declare.py .
tests/test_circuit/test_define.py .
tests/test_higher/test_braid.py .
tests/test_higher/test_curry.py .
tests/test_higher/test_currylut.py .
tests/test_higher/test_curryrom.py .
tests/test_higher/test_flat.py .
tests/test_higher/test_fork.py .
tests/test_higher/test_higher_compose.py .
tests/test_higher/test_join.py .
tests/test_interface/test_interface.py ....
tests/test_io/test_inout1.py .
tests/test_io/test_inout2.py .
tests/test_io/test_out1.py .
tests/test_io/test_out2.py .
tests/test_ir/test_declaretest.py .
tests/test_ir/test_ir.py .
tests/test_meta/test_class.py .
tests/test_meta/test_creg.py .
tests/test_simulator/test_counter.py .
tests/test_simulator/test_ff.py .
tests/test_simulator/test_logic.py .
tests/test_type/test_anon_type.py .
tests/test_type/test_array.py .
tests/test_type/test_array2d.py .
tests/test_type/test_arrayconstruct.py .
tests/test_type/test_arrayflip.py .
tests/test_type/test_arrayval.py .
tests/test_type/test_awire1.py .
tests/test_type/test_bit.py .
tests/test_type/test_bitflip.py .
tests/test_type/test_bitval.py .
tests/test_type/test_tuple.py .
tests/test_type/test_tupleconstruct.py .
tests/test_type/test_tupleflip.py .
tests/test_type/test_tupleval.py .
tests/test_type/test_twire1.py .
tests/test_type/test_type_errors.py ...
tests/test_type/test_vcc.py .
tests/test_type/test_whole.py .
tests/test_type/test_wire1.py .
tests/test_type/test_wire2.py .
tests/test_type/test_wire3.py .
tests/test_type/test_wire4.py .
tests/test_type/test_wire5.py .
tests/test_verilog/test_verilog.py .
tests/test_wire/test_arg1.py .
tests/test_wire/test_arg2.py .
tests/test_wire/test_array1.py .
tests/test_wire/test_array2.py .
tests/test_wire/test_array3.py .
tests/test_wire/test_call1.py .
tests/test_wire/test_call2.py .
tests/test_wire/test_compose.py .
tests/test_wire/test_const0.py .
tests/test_wire/test_const1.py .
tests/test_wire/test_errors.py ..
tests/test_wire/test_flip.py .
tests/test_wire/test_named1.py .
tests/test_wire/test_named2a.py .
tests/test_wire/test_named2b.py .
tests/test_wire/test_named2c.py .
tests/test_wire/test_pos.py .

========================== 70 passed in 1.45 seconds ===========================

magma's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

magma's Issues

Add a cast operator

  • array(val) and type.as_array()
  • bits(val) and type.as_bits()
  • uint(val) and type.as_uint()
  • sint(val) and type.as_sint()

Clean up naming conventions and interfaces for defining, instancing, and wiring up circuits

Begin with two abstract inputs:

I0 = In(Array(2, Bit))
I1 = In(Array(2, Bit))

Pattern 1: DefineName(args) defines a circuit, which can be instanced and wired up normally

Add2 = DefineAdd(width=2)
add2 = Add2(**instance_args)
add2_O = add2(I0, I1)

Pattern 2: Name(args) defines or retrieves a cached definition of the circuit and instances it

add2 = Add(2, **instance_args) 
add2_O = add2(I0, I1)

Pattern 3: name(args) infers the widths of the inputs, defines or retrieves a cached definition of the circuit, instances it, wires up inputs, and returns output

add2_O = add(I0, I1, **instance_args)

Testing interface for primitives

Proposal: test(magma.operator.and_, operator.and_, [Bit, Bits, UInt, SInt])

Where test(magma_primitive_op, python_op, types) will test magma_primitive_op in the Python simulator and Verilator using python_op as a gold function. type is a list of magma types to test the operations on.

Define `__format__` specification for printing circuit components with qualified names

From Python docs:

object.format(self, format_spec)
Called by the format() built-in function (and by extension, the format() method of class str) to produce a “formatted” string representation of an object. The format_spec argument is a string that contains a description of the formatting options desired. The interpretation of the format_spec argument is up to the type implementing format(), however most classes will either delegate formatting to one of the built-in types, or use a similar formatting option syntax.

See Format Specification Mini-Language for a description of the standard formatting syntax.

This should be useful for simulation, but the specification might need to be blocked until we begin integrating the magma simulation code and have a better understanding of how we might use this.

Bit vs Bool

We should have two Bit types.

One type, say Bit, should only be wireable.

The other type, say Bool, should implement logical operations such as And, Or, Xor, Not.

We should be able to convert between the two.

Clock would be a subtype of Bit. You can only wire clock signals. To perform logical operations on Clock would require it to be converted to a Bool.

In terms of naming, the above suggestions is not ideal. We currently allow logical operations on Bits(n), which by analogy means we should allow logical operations on Bit. Not sure what to name the type which is just wireable.

Type conversions

I suggest all type conversions be done in a way similar to standard python type conversions. For example int(x) converts the string s to an int. There are other options like int(s, base).

In particular, we would implement bits(), uint(), and sint().

Here are some variants for bits. Similar versions would exist for uint and sint.

bits(i,n) - convert the python integer i to a magma Bits(n)
bits(b0, b1, ..., bn) - convert n Bit values to a Bits(n) value
bits(uint) - converts a UInt(n) to a Bits(n)
bits(sint) - converts a Sint(n) to Bits(n)

This issue is related to #37

Change Register(ce=False) to Register(has_ce=False)

Make it obvious that this is generating a register with a clock enable, not that we are wiring up a clock enable.

This should be done to all primitives with clock signals (including counters, shift registers, etc.)

Explore better syntax for subclassing Circuit

Can we remove the classmethod definition requirement?

In chisel you can write:

class Mux2 extends Module {
  val io = IO(new Bundle{
    val sel = Input(UInt(1.W))
    val in0 = Input(UInt(1.W))
    val in1 = Input(UInt(1.W))
    val out = Output(UInt(1.W))
  })
  io.out := (io.sel & io.in1) | (~io.sel & io.in0)
}

hypothetical python version

class Mux2(Circuit):
  io = IO({
    "sel" : In(Bit),
    "in0" : In(Bit),
    "in1" : In(Bit),
    "out" : Out(Bit)
  })
  io.out = (io.sel & io.in1) | (~io.sel & io.in0)

In(Bits(n))

Print out In(Bits(n)) yields "Bits(n)". Should probably be "In(Bits(n))". Same for Out(Bits(n)).

Similarly, In(UInt(n)) and In(SInt(n)) should print the direction.

Document magma higher order functions

These features are very powerful but a bit hard to understand, would be great to have:

  • Documentation on each supported function
    • braid
    • row
    • col
    • join
    • fork
    • flat
    • compose
    • curry
    • uncurry
    • map
  • Tutorials/Examples with demonstrative use cases composing these functions

The magma programming model has an existing section discussing this, but I think it needs to be expanded. There may also be some useful examples in the 488H lecture slides.

Can't set simulator inputs which aren't wired up

from magma import *
from loam.boards.icestick import IceStick

from magma.python_simulator import PythonSimulator
from magma.scope import *

icestick = IceStick()
icestick.Clock.on()
for i in range(4): icestick.PMOD0[i].input().on()

main = icestick.main()
EndCircuit()

sim = PythonSimulator(main)

scope = Scope()

sim.set_value(main.PMOD0[0], scope, True)

error on last line:

import lattice ice40
import lattice mantle40
Traceback (most recent call last):
  File "/Users/osnr/dev/magmacore/magma/transforms.py", line 35, in get_new_bit
    return self.orig_to_new[QualifiedBit(bit=orig_bit, scope=scope)]
KeyError: QualifiedBit(bit=main.PMOD0[0], scope=<magma.scope.Scope object at 0x102b44d68>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tests/test_simulator.py", line 18, in <module>
    sim.set_value(main.PMOD0[0], scope, True)
  File "/Users/osnr/dev/magmacore/magma/python_simulator.py", line 223, in set_value
    newbit = self.txfm.get_new_bit(bit, scope)
  File "/Users/osnr/dev/magmacore/magma/transforms.py", line 37, in get_new_bit
    raise MagmaTransformException("Could not find bit in transform mapping. bit={}, scope={}".format(orig_bit, scope))
magma.transforms.MagmaTransformException: Could not find bit in transform mapping. bit=main.PMOD0[0], scope=<magma.scope.Scope object at 0x102b44d68>

but works if you icestick.D1.on() and wire(main.D1, main.PMOD0[0]), so I guess it's the fact that PMOD0[0] wasn't wired in the example?

I'd find setting unwired inputs useful if I'm commenting out parts of the circuit or doing automatic circuit generation. I guess I could just ignore that exception, since the input doesn't matter anyway.

Move ice40 simulator to mantle/lattice/ice40

I think it would make more sense to have the ice40 simulator primitives be part of mantle/lattice/ice40.

We can also test the mantle40 code using the yosys implementations of the ice40 primitives. This would allow is to test the mantle40 implementations using verilator.

Create Bool, SInt, and UInt types that support operator overloading

See Chisel types

Proposed type hierarchy

Bit (Bool = Bit)

  • logical operations &, |, ^, !

Bits(n) = Array(n,Bit)

  • logical operations &, |, ^, ~, >>, <<, ...

UInt(n) \subclass Bits(n)

  • unsigned arithmetic operators +, -, *, /
  • unsigned relational operarors <, <=, >, >=

SInt(n) \subclass Bits(n)

  • signed arithmetic operators +, -, *, /
  • signed relational operarors <, <=, >, >=

Make naming circuits more convenient

Unique names for circuit definitions are needed for implementing caching.

Two possibilites:

  • Provide convenience function for generating the name based on parameters
  • Cache circuits based on something other than names

complete coreir primitives

These primitives need DefineOp, Op and op versions

  • mux
  • negate
  • min, max
  • abs
  • memory

Pseudoprimitives (meta-programmed in python)

  • concat
  • repeat
  • rol, ror

Improve test coverage

Past projects I've worked on have used 90% coverage as a target which seems pretty reasonable.

Report from today (5/24/17) obtained via pytest --cov=magma

Name                                      Stmts   Miss  Cover
-------------------------------------------------------------
magma/__init__.py                            22      1    95%
magma/array.py                              193     48    75%
magma/backend/__init__.py                     0      0   100%
magma/backend/blif.py                       104     86    17%
magma/backend/verilog.py                    134     38    72%
magma/bit.py                                101     20    80%
magma/bits.py                                49     37    24%
magma/bitwise.py                             37     30    19%
magma/board.py                                8      4    50%
magma/circuit.py                            285     81    72%
magma/compatibility.py                        7      2    71%
magma/compile.py                             54     29    46%
magma/compile_abstract.py                    78     78     0%
magma/config.py                               6      0   100%
magma/debug.py                               14      1    93%
magma/error.py                               23      4    83%
magma/fpga.py                                43     34    21%
magma/higher.py                             216     50    77%
magma/interface.py                          206     43    79%
magma/ir.py                                  13      0   100%
magma/part.py                                10      6    40%
magma/peripheral.py                           9      5    44%
magma/port.py                               105     25    76%
magma/python_simulator.py                   160    127    21%
magma/ref.py                                 60      8    87%
magma/scope.py                               26     17    35%
magma/simulator.py                           19      8    58%
magma/simulator_interactive_frontend.py     252    222    12%
magma/t.py                                   58      6    90%
magma/tests.py                               11      0   100%
magma/transforms.py                         147    128    13%
magma/tuple.py                              174     42    76%
magma/wire.py                                96     49    49%
-------------------------------------------------------------
TOTAL                                      2720   1229    55%

Int binop simulation doesn't work for 1-bit output

Example:

from magma import *
from mantle.lattice.mantle40 import Counter

hello = DefineCircuit('hello', 'O', Out(Bit), 'CLK', In(Bit))

counter = Counter(8)
wire(hello.O, counter.O < constarray(3, 8)) # binop with 1-bit output

from magma.python_simulator import PythonSimulator
from magma.scope import Scope

sim = PythonSimulator(hello)
scope = Scope()
sim.evaluate()
print(sim.get_value(hello.O, scope))

yields AssertionError in ValueStore.set_value because "Can only set boolean values".

I think the problem is that the binop simulate implementation in primitives.py always calls as_bool_list() to generate newval, even if newval needs to be just one bool instead of a list of bools.

Would it be better to do this in set_value:

        elif isinstance(newval, list) and len(newval) == 1:
            newval = newval[0]

or to dispatch on the out_type in simulate?

Clock, Enable, Reset

We should define single Bit types for FFs and change the wireclock logic in compile to test for the type rather than use string names (which is fragile since different vendors use different names for these signals).

  • Clock
  • Enable
  • Reset

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.