Giter Club home page Giter Club logo

svut's Introduction

SystemVerilog Unit Test (SVUT)

GitHub license Github Actions Github Actions GitHub issues GitHub stars GitHub forks Twitter

Introduction

SVUT is a very simple flow to create a Verilog/SystemVerilog unit test. It is widely inspired by SVUnit, but it's written in python and run with Icarus Verilog. SVUT follows KISS principle: Keep It Simple, Stupid.

Hope it can help you!

How to Install

Pypi

SVUT is availble on Pypi and can be installed as following:

pip3 install svut

Git

Git clone the repository in a path. Set up the SVUT environment variable and add SVUT to $PATH:

export SVUT=$HOME/.svut
git clone https://github.com/dpretet/svut.git $SVUT
export PATH=$SVUT:$PATH

SVUT relies on Icarus Verilog as simulation back-end. Please install it with your favourite package manager and be sure to use a version greater or equal to v10.2. SVUT is tested with v10.2 and cannot work with with lower version (<= v9.x).

SVUT can also use Verilator but the support is more limited for the moment. A future release will fix that. An example to understand how to use it along Icarus can be found here

How to use it

To create a unit test of a verilog module, call the command:

svutCreate your_file.v

No argument is required. SVUT will create "your_file_testbench.sv" which contains your module instanciated and a place to write your testcase(s). Some codes are also commented to describe the different macros and how to create a clock or dump a VCD for GTKWave. To run a test, call the command:

svutRun -test your_file_testbench.sv

or simply svutRun to execute all testbenchs in the current folder.

svutRun

SVUT will scan your current folder, search for the files with "_testbench.sv" suffix and run all tests available. Multiple suffix patterns are possible.

svutRun proposes several arguments, most optional:

  • -test: specify the testsuite file path or a folder containing tests
  • -f: pass the fileset description, default is files.f
  • -sim: specify the simulator, icarus or verilator
  • -main: specify the main.cpp file when using verilator, default is sim_main.cpp
  • -define: pass verilog defines to the tool, like -define "DEF1=2;DEF2;DEF3=3"
  • -vpi: specify a compiled VPI, for instance -vpi "-M. -mMyVPI"
  • -dry-run: print the commands but don't execute them
  • -include: to pass include path, several can be passed like -include folder1 folder2
  • -no-splash: don't print SVUT splash banner, printed by default
  • -compile-only: just compile the testbench, don't execute it
  • -run-only: just execute the testbench, if no executable found, also build it

Tutorial

Copy/paste this basic FFD model in a file named ffd.v into a new folder:

`timescale 1 ns / 1 ps

module ffd
    (
    input  wire aclk,
    input  wire arstn,
    input  wire d,
    output reg  q
    );

    always @ (posedge aclk or negedge arstn) begin
        if (arstn == 1'b0) q <= 1'b0;
        else q <= d;
    end

endmodule

Then run:

svutCreate ffd.v

ffd_testbench.v has been dropped in the folder from you called svutCreate. It contains all you need to start populating your testcases. In the header, you can include directly your DUT file (uncomment):

`include "ffd.v"

or you can store the path to your file into a files.f file, automatically recognized by SVUT. Populate it with the files describing your IP. You can also specify include folder in this way:

+incdir+$HOME/path/to/include/

Right after the module instance, you can use the example to generate a clock (uncomment):

initial aclk = 0;
always #2 aclk <= ~aclk;

Next line explains how to dump your signals values into a VCD file to open a waveform in GTKWave (uncomment):

initial $dumpvars(0, ffd_unit_test);
initial $dumpfile("ffd_testbench.vcd");

Two functions follow, setup() and teardown(). Use them to configure the environment of the testcases:

  • setup() is called before each testcase execution
  • teandown() after each testcase execution

A testcase is enclosed between to specific defines:

`UNIT_TEST("TESTNAME")
    ...
`UNIT_TEST_END

TESTNAME is a string (optional), which will be displayed when test execution will start. Then you can use the macros provided to display information, warning, error and check some signals status and values. Each error found with macros increments an error counter which determine a testcase status. If the error counter is bigger than 0, the test is considered as failed.

A testsuite, comprising several UNIT_TEST is declared with another define:

`TEST_SUITE("SUITENAME")
    ...
`TEST_SUITE_END

To test the FFD, add the next line into setup() to drive the reset and init the FFD input:

arstn = 1'b0;
d = 1'b0;
#100;
arstn = 1'b1;

and into the testcase:

`FAIL_IF(q);

Here is a basic unit test checking if the FFD output is 0 after reset. Once called svutRun in your shell, you should see something similar:

INFO: Start testsuite << FFD Testsuite >> (@ 0)

INFO: Starting << Test 0: Check reset is applied >> (@ 0)
I will test if Q output is 0 after reset (@ 100000)
SUCCESS: Test 0 pass (@ 110000)

INFO: Starting << Test 1: Drive the FFD >> (@ 110000)
I will test if Q output is 1 after D assertion (@ 210000)
SUCCESS: Test 1 pass (@ 236000)

INFO: Stop testsuite 'FFD Testsuite' (@ 236000)
  - Warning number:  0
  - Critical number: 0
  - Error number:    0
  - STATUS: 2/2 test(s) passed

SVUT relies (optionally) on files.f to declare fileset and define. The user can also choose to pass define in the command line:

svutRun -test my_testbench.sv -define "DEF1=1;DEF2;DEF3=3"

SVUT doesn't check possible collision between define passed in command line and the others defined in files.f. Double check that point if unexpected behavior occurs during testbench.

Finally, SVUT supports VPI for Icarus. Follow an example to compile and set up the flow of an hypothetic UART, compiled with iverilog and using a define "PORT":

iverilog-vpi uart.c
svutRun -vpi "-M. -muart" -define "PORT=3333" -t ./my_testbench.sv &

Now you know the basics of SVUT. The generated testbench provides prototypes of available macros. Try them and play around to test SVUT. You can find these files into the example folder.

Enjoy!

License

Copyright 2021 The SVUT Authors

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. imitations under the License.

svut's People

Contributors

damofthemoon avatar dpretet avatar evanmays avatar olofk avatar sarahec avatar sschaetz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

svut's Issues

Enable alternate color palettes and output formats

Building on #15, we'd be able to include implement svut_h.sv as a group of header files.

Why? I've prototyped a very nice color palette system (that uses a header file to define colors, another to define the palette (e.g. dark mode, light mode), and then the SVUT test macros.) While dark screens are all the rage, we also need to think about people who still work on white backgrounds, people who cant work with color (e.g. because the contrast is too low or due to colorblindness), etc.

This would also allow plugging in alternate output formats (Markdown, HTML, etc.) trivially by redefining one macro and including it in a file before loading svut_h.sv.

Using #15, a multi-file svut_h.sv would simply have the main header file include its subsidiary header files (with a ifndef wrapper) and they could be overridden by loading something different in front of svut_h.sv.

It sounds complicated, but it really isn't. Most people will include the standard header file. Some will also load a "light mode" or "monochrome" file in front of this to redefine the color palette. That's it. Fully backwards-compatible with easy room for growth.

Test name incorrectly parsed

Consider the following module, register.v:

module register(q, d, clk, enable, reset);
    // ...
endmodule

Running svutCreate register.v generates the file register(q,_unit_test.sv.

Build common framework for launching verilog tools

Every change we make to the iverilog launcher code has to be reflected in the other simulator(s), yet there are major commonalities between each. Also, some of the associated work (adding/removing/displaying files, opening a waveform viewer) may vary from platform to platform (e.g. the command to launch gtkwave on Linux fails on Mac OS X.)

Most of the logic for launching simulators, viewers, etc. can be factored out into a common class (including platform-specific logic.) This would make each tool's launcher into a very small subclass that provides any overrides for a specific tool.

Add an AUTHORS file

As a Google employee, if I add a new file I either have to copyright it to Google (just as you add copyright notices for your own work) under the MIT or Apache 2 licenses. The alternative is to add an AUTHORS file as described in https://opensource.google.com/docs/releasing/authors/ and edit the copyright to read "the SVUT project authors" (or similar).

Some of the upcoming work is likely to need new files: sample code, unit tests, etc. Making this change now will make it easier to maintain consistency across the project and appropriately highlight each of our contributions.

Enable svutRun to autorun tests on file save

Proposal:

  1. Add a --watch option to svutRun. It should run the appropriate test(s) when a test or source file changes.
  2. Respect the --tests flag (or absence thereof) to determine which tests to run.

Proposed enhancements:
3. Detect when a compilation error will prevent a test from running and raise a warning.
4. Capture the output of all tests and print a summary at the end. (Minimizes scrolling.) This would live behind a feature flag.

(Please assign to me. I will file subsidiary bugs for the enhancements.)

Detect when a compilation error will prevent a test from running

Recommended enhancement to #8 -- split the compilation and running stages into two parts (allowing the compiler's stderr and stdout to appear normally), and check the returned result. If compilation failed with an error, show a warning that we're running the previous version of the test.

This allows the watch command to run without erroring out and keeps running the un-afflicted tests.

(Kindly assign to me. Thanks)

Feature: Add verilator support

While writing a SystemVerilog based IP, Verilator became a very important feature to implement.
feature-verilator branch start to put in place this simulator support.

Suggestion: pip-installable Pyton package

Hi,

I'm a grateful user of svut (it is excellent work ๐Ÿ‘) and am wondering if there is any desire to make this package pip installable and turn it into a standard Python package. If you are interested I'd be happy to take that on. Indeed, I took a (very opinionated) stab at it here: master...sschaetz:svut:python-package where I am doing the following:

  • create a setup.py and pyproject.toml file which allows for this package to be pip-installable (and I think publishable to pypi but I have never done that)
  • move all sources into a sub-directory svut (this is somewhat of a convention for python packages)
  • rename the 2 command line tools to more of a Python convention snake_case format
  • expose the two command line tools via setup.py CLI tools -- this way once you install the python package you have the tools automatically in your path and don't need to mess with $PATH

The caveat is that I think I probably broke some of the shell scripts but I'm happy to fix them.

Cheers,
Seb

Regression in -test option

sec@dervish$ svutRun -test neopixel_driver_eol_tb.sv
Traceback (most recent call last):
File "/home/sec/.svut/svutRun", line 126, in
if "all" in ARGS.test.lower():
AttributeError: 'list' object has no attribute 'lower'

Set up imports properly at runtime

svutRun enables svut_h.sv inclusion by copying it into the source directory and then deleting it after. We could make tests run a bit faster (and pave the way for new test features) by passing this information to the simulator instead.

At the same time, tests that use a split src/test directory structure have to use weird tricks like prefixing all includes with ../src/. This is the kind of information that should be passed as an include option and is one that the tool can detect (i.e. not requiring the user to pass a bunch of arguments on the command line.)

So: Enable the tool to detect and specify a default set of imports (which can then be modified by the command line flags.)

Summarize results at end of run

This is a recommended extension to #8.

When running large tests and/or multiple tests, spotting errors often requires a great deal of scrolling. If we run tests by forking a process and getting a copy of stdout (see #9), we can collect markers in the test output and generate a summary.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.