datawire / quark Goto Github PK

Quark is a specialized language for defining and implementing communication protocols in multiple languages.

Home Page: http://datawire.github.io/quark/

License: Apache License 2.0

Python 33.37% Java 19.76% JavaScript 26.02% Shell 1.11% Ruby 19.21% CSS 0.09% HTML 0.43%

quark's Introduction

Quark

Quark is a specialized language for defining and implementing communication protocols in multiple languages. In particular, protocols that have semantics that require sophisticated client behavior are well suited to Quark.

Motivation

As distributed systems have proliferated (e.g., microservices), the need for more sophisticated protocols that incorporate backpressure, failover, retries, timeouts, rate limiting, and other behavior has increased. Unfortunately, this sophistication introduces more complexity in implementation. In turn, implementation complexity reduces adoption because supporting a protocol implementation in multiple languages requires manual porting.

Quark addresses this problem by enabling protocol authors to write a single canonical implementation in the Quark language. The Quark compiler then generates idiomatic implementations of the protocol in multiple languages (currently Java, JavaScript, and Python, with Ruby support in progress).

Use cases

The use cases for Quark are highly varied. Some examples:

the Slack API includes rate limiting behavior over a WebSockets and REST API; see slack.q for an example implementation
the AMQP 1.0 protocol requires sophisticated client behavior for flow control and reconnect

Getting Started

A popular use case for Quark is to add resilience semantics for microservice architectures. If you're interested in doing this, check out Datawire Connect which embeds Quark.

If you're interested in using Quark directly to develop a protocol, just install Quark:

    curl -sL https://raw.githubusercontent.com/datawire/quark/master/install.sh | sh

Examples

Check out the examples in the Quark Examples repository:

    git clone [email protected]:datawire/quark.git
    cd quark/examples

Next Steps

The Quark Quick Start provides an overview of the basic installation requirements and process, basic compile commands, and the Quark language. If you have successfully run any of the examples linked above, you can probably skip this book.

Please read the Quark Installation and Configuration Guide for more detailed information about installation, configuration, compilation, packaging, and documentation generation.

The Quark Language Reference provides information about how to code in Quark and specifics about individual language elements including keywords, operators, functions, data types, and annotations.

The Quark Issues list provides a list of known issues - please feel free to add new issues or feature requests as you start to use Quark. You can also join our public Slack support channel.

If you want to contribute to Quark see the contributing documentation and testing documentation.

quark's People

Contributors

Stargazers

Watchers

quark's Issues

Spurious empty directories created in base quark directory

When I compile quark files with packages I am getting an empty directory for each compiled package in my base quark directory.

I am not sure if this happens with a specific language or with all of them; I will try to figure it out and edit or leave a comment.

Compiler should catch name collision between package and file name

The package name and the quark file name must be different. The compiler currently allows them to be the same - it should check and throw an error in this case.

Referencing invalid package names causes a compiler crash

I changed the name of a package and missed a reference to the old name in the code. When I tried to compile the file, I got the following compiler crash:

$ quark --python ~/quark-examples/python examples/list.q
Traceback (most recent call last):
  File "/Users/janicekarin/virtualenv/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/janicekarin/quark/quark/compiler.py", line 566, in main
    exit(_main(docopt(main.__doc__)))
  File "/Users/janicekarin/quark/quark/compiler.py", line 542, in _main
    c.compile()
  File "/Users/janicekarin/quark/quark/compiler.py", line 497, in compile
    self.root.traverse(use)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 94, in traverse
    leave(self, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/compiler.py", line 233, in leave_Type
    if n.text in type.env:
AttributeError: 'NoneType' object has no attribute 'env'

>=, =, <= comparison operations not defined for numbers

Quark does not currently support using greater than or equal to, less than or equal to, or equal to comparison operators with numerical types. Statements like the following will not compile:

if (foo >= 20) {
    print("this is the output when the if condition is true");
    }

Cryptic error message when you use else without a corresponding if statement

When the Quark compiler encounters an else statement that doesn't follow an if statement it throws the following cryptic error:

examples/foo.q:85:10: Rule 'SEMI' didn't match at '{
    print("this is' (line 85, column 10).

Names of options to set compiler output directories inconsistent

It would be easier to remember compiler options with consistent naming. However, the option for a single output directory is --output while the names of the options to set output directories for java, js, python are --language-out. We should use either output or out for both.

Doc generated for Javascript only includes the package version #, @doc directives for classes, and method signatures

The README.md generated for Javascript only appears to include the following:

The package name with version (the version # is not included in Javadoc)
class names (with no indication that they are classes)
method signatures (with no indication of what they are)

None of the properties are included by name, their @doc content is absent, and so is the package description.

Quark should support some mechanism for error handling

There should be a way to throw an exception or return an error within Quark.

quark runtime module dependencies should be immune to quark code

example:

quark_runtime.py imports json

if one creates json.q it gets emitted as json.py and shadows the stdlib json module

Quark interfaces have no code restrictions resulting in the generation of invalid Java interfaces

I can define an interface with any valid Quark code inside of it. However, Java has strict rules about what's allowed inside an interface which our generated code will happily break. For example:

Quark source:

interface Foo {
   float e;
   float radius;
   int E;

   float diameter(float radius) {
      return radius*2;
   }
}

Generated Java:

public interface Foo {
    public Double e;
    public Double radius;
    public Integer E;
    public Double diameter(Double radius) {
        return (Double) ((radius) * (2));
    }
}

Quark doesn't support compound if conditionals because it doesn't support && and || for integers

Quark uses 1/0 for true/false. They are treated as integers. && and || are not supported for integers. As a consequence, statements like the following are not supported:

if (input.type=="length" && output.type=="length"){...}

If you try to use them, the compiler throws this error for all languages:

foo.q:36:16:int has no such attribute: and

You can work around this for && by nesting if statements as follows:

if (input.type=="length"){
    if (output.type=="length"){
         ...
    }
}

Note: Issue #3 requests support for booleans. If that issue is addressed, the lack of && and || support becomes less important (and possibly irrelevant) as it no longer impedes if statement construction.

Quark backends sometimes don't handle multiple instances of a package correctly

i.e. having

package foo { ... }

... other stuff ...

package foo { ... }

will collect multiple package AST nodes and not merge them in some cases, such as for handling annotations and for output of things like package.json and setup.py.

Need to turn Backend self.packages into an OrderedDict.
Need to add a test for this stuff working correctly.

Quark class implements a quark interface

In quark one can write

interface Itf { ... }
class X extends Itf {...}

but it does not produce correct java. Python happens to be OK. Javascript didn't check yet.

Doc generation for python should include a basic make file

The generated python doc should include a basic make file for Sphinx and some boilerplate instructions on how to use it so folks who aren't familiar with python can still generate the docs.

Inconsistent compiler error formats

Quark currently uses two different formats for compiler errors:

All errors start with :: . However, parse errors append (line , column ). after the message.

For example:

examples/convert.q:22:24: Rule 'expr' didn't match at '.3048;
float ' (line 22, column 24).

examples/convert.q:6:9: unresolved variable: string

Incompatible return types compile in Quark, generated code throws errors

Quark lets me return a value of the wrong type. However, the generated code complains in all three languages.

For example, I compiled this class:

    class ConversionFactors {
         float foo=1;
         Map<String,float> initializeConversion() {
             Map<String,float> lengthFactors = new Map<String,float>();  
             lengthFactors.__set__("meter",1);
             lengthFactors.__set__("centimeter",100);
             lengthFactors.__set__("foot",3.2808399); 
             lengthFactors.__set__("mile",0.000621371);
          lengthFactors.__set__("parsec",0.000000000000000032407557442396);
             lengthFactors.__set__("league",0.00020712330174427);
             return foo;
         }
    }

It compiled and generated code in all three languages. However, all three threw errors when I tried to use the code:

$ python wrongReturn.py
Traceback (most recent call last):
  File "wrongReturn.py", line 40, in <module>
    main()
  File "wrongReturn.py", line 11, in main
    (length2).value = (conversion).Convert(length1, length2)
  File "/Users/janicekarin/quark-examples/python/convert/__init__.py", line 47, in Convert
    (input).conversionFactor = (self.relativeLengths).get((input).label)
AttributeError: 'int' object has no attribute 'get'

$ node wrongReturn.js
/Users/janicekarin/quark-examples/javascript/node_modules/quark_runtime.js:31
        if (m.has(key)) {
              ^
TypeError: undefined is not a function
    at Object.map_get (/Users/janicekarin/quark-examples/javascript/node_modules/quark_runtime.js:31:15)
    at Conversion_Convert [as Convert] (/Users/janicekarin/quark-examples/javascript/convert/index.js:63:45)
    at main (/Users/janicekarin/quark-examples/javascript/wrongReturn.js:10:36)
    at Object.<anonymous> (/Users/janicekarin/quark-examples/javascript/wrongReturn.js:39:1)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)

$ javac Functions.java
Functions.java:4: error: incompatible types: int cannot be converted to Double
        convert.Unit length2 = new convert.Unit("length", "foot", 0);
                                                                  ^
./convert/Unit.java:8: error: incompatible types: int cannot be converted to Double
    public Double value = (Double) (0);
                                   ^
./convert/Unit.java:9: error: incompatible types: int cannot be converted to Double
    public Double conversionFactor = (Double) (0);
                                              ^
./convert/Conversion.java:15: error: incompatible types: int cannot be converted to Double
                return (Double) (-(123456789));
                                ^
./convert/Conversion.java:18: error: incompatible types: int cannot be converted to Double
            return (Double) (-(987654321));
                            ^
./convert/ConversionFactors.java:4: error: incompatible types: int cannot be converted to Double
    public Double foo = (Double) (1);
                                 ^
./convert/ConversionFactors.java:7: error: no suitable method found for put(String,int)
        (lengthFactors).put(("meter"), (1));
                       ^
    method Map.put(String,Double) is not applicable
      (argument mismatch; int cannot be converted to Double)
    method AbstractMap.put(String,Double) is not applicable
      (argument mismatch; int cannot be converted to Double)
    method HashMap.put(String,Double) is not applicable
      (argument mismatch; int cannot be converted to Double)
./convert/ConversionFactors.java:8: error: no suitable method found for put(String,int)
        (lengthFactors).put(("centimeter"), (100));
                       ^
    method Map.put(String,Double) is not applicable
      (argument mismatch; int cannot be converted to Double)
    method AbstractMap.put(String,Double) is not applicable
      (argument mismatch; int cannot be converted to Double)
    method HashMap.put(String,Double) is not applicable
      (argument mismatch; int cannot be converted to Double)
./convert/ConversionFactors.java:13: error: incompatible types: Double cannot be converted to HashMap<String,Double>
        return (java.util.HashMap<String,Double>) (foo);
                                                  ^
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
9 errors

Compiler crash on method call in the emit stage

This code crashes the compiler in the emit stage:

class Foo {
    void next() {
    }

    void test() {
        next();
    }
}

Workaround: call self.next() instead of next().

$ quark --java=java attrerror.q 
Traceback (most recent call last):
  File "/Users/ark3/.virtualenvs/datawire/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 528, in main
    exit(_main(docopt(main.__doc__)))
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 504, in _main
    c.compile()
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 477, in compile
    self.emit(backend)
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 481, in emit
    self.root.traverse(backend)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 89, in traverse
    visit(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 57, in visit_Class
    self.files["%s.java" % c.name.text] = c.match(self.dfnr)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 146, in match_Class
    body = "\n".join([d.match(self) for d in c.definitions])
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 166, in match_Function
    body = " %s" % m.body.match(self.stmtr) if m.body else ";"
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 259, in match_Block
    return "{%s}" % indent("\n".join([s.match(self) for s in b.statements]))
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 236, in match_ExprStmt
    return "%s;" % stmt.expr.match(self.exprr)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 285, in match_Call
    return self.invoke(type, c.expr, [a.match(self) for a in c.args])
  File "/Users/ark3/Dropbox/datawire/quark/quark/dispatch.py", line 52, in __call__
    return method(self.object, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 346, in invoke
    return "(%s).%s(%s)" % (expr.expr.match(self),
AttributeError: 'Var' object has no attribute 'expr'

Support division for numerical types

Numerical types currently have add, sub (subtract), and mul (multiply) functions but do not currently support div (divide). We should add it.

Quark compiles with package-internal references to package names but the resulting code doesn't work

If I reference another class in the current package using package.class notation, Quark accepts it and spits out generated code. However, that code will not compile and/or run in any of the languages.

See attached file for an example.
convertUnits2.txt

The code generated by that file throws the following Python errors:

$ python convertUnits2.py
Traceback (most recent call last):
  File "convertUnits2.py", line 16, in <module>
    main()
  File "convertUnits2.py", line 10, in main
    conversion = convert2.Conversion();
  File "/Users/janicekarin/quark-examples/python/convert2/__init__.py", line 47, in __init__
    def __init__(self): self._init()
  File "/Users/janicekarin/quark-examples/python/convert2/__init__.py", line 44, in _init
    self.conversionFactors = convert2.ConversionFactors()
NameError: global name 'convert2' is not defined

Either the Quark compiler should complain about these references or it should generate code that works in the supported languages when they are present.

cannot return from void function

$ cat void_return.q
void foo() { return; }
$ quark --python=/tmp void_return.q
void_return.q:1:14: unresolved variable: return

No way to remove elements from a map

There's no way to remove an element from a map. We should support remove(key) or similar.

Unhelpful error if you use "Class" instead of "class"

When I accidentally used "Class" instead of "class" in one of my Quark files I got the following error giving me no indication of what the problem actually was:

examples/null.q:1:11: Rule 'LPR' didn't match at '{
   float a;
   int' (line 1, column 11).

@version behavior inconsistent

I am not sure what the intended behavior is for @Version. It doesn't seem to do anything unless it's at the top level of a file. If I place a single @Version annotation at the top of a file just above the first package the following happens:

Java - nothing. No entries in pom.xml
Javascript - package.json for all packages in the file are set to the specified version number
Python - setup.py has an entry as follows:

setup(name="package1",
      version="version",
      packages=['package1', 'package2',...,'packageK'])

where package1 is the first package in the file, defined just under @Version, and the other packages are all defined lower in the file.

If I add a second @Version with a different value just above package2, the following happens:

Java - nothing. No entries in pom.xml
Javascript - package.json for package1 is set to version1 and package.json for package2 is set to version2. package.json for later packages is set to 0.0 which I believe is the default
Python - setup.py is the same as the previous case

If I add a second file with further definition of package1 and I add @Version before it with a different version that version is ignored in all cases.

macros can be defined with a subset of the supported languages; the resulting code may throw errors or cause unexpected behavior

I defined two macros in Python and Java but not Javascript:

macro void print3(String msg) $java{System.out.println($msg)}
                             $py{_println($msg)};

macro float degreesToDecimal(int degrees, int minutes, int seconds) $java{($degrees) + ($minutes/60.0) + ($seconds/3600.0)} $py{($degrees) + ($minutes/60.0) + ($seconds/3600.0)};

When I use print3 in Quark code, the code compiles and generates code in all three languages. The generated code prints to stdout in Python (I did not check Java because of other known issues) and results in a no-op in Javascript.

When I use degreesToDecimal in Quark code as follows:

float latitude=degreesToDecimal(38,53,55);
float longitude=degreesToDecimal(-77,2,16);

the code compiles and generates code in all three languages. It calculates the expected values in Python and results in the following error in Javascript:

$ node testingMacros.js
/Users/janicekarin/quark-examples/javascript/testingMacros.js:4
    var latitude = ;
                   ^
SyntaxError: Unexpected token ;
    at exports.runInThisContext (vm.js:73:16)
    at Module._compile (module.js:443:25)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3

The Quark compiler should throw an error indicating the macro is not defined rather than having it evaluate to null which may not be a valid value (as illustrated above)

Keywords are not reserved

I was able to replace "i" with "if" in countHello.q and it still compiled:

    int if=0;
    while (if<3) {
        if=if+1;
        print(if.toString() + ": Hello World!");
    }
}

I should get an error telling me keywords are reserved or similar.

Quark needs a story around casting and coercion

Due to the way autocasting works, it's not possible to turn floats into ints in Python and JavaScript, nor is it possible to turn ints into floats in Java.

float computedValue = 3.75;
int index = computedValue;
return someList[index];

In Python and JS, casting (and thus coercion) doesn't happen at all, so the interpreter tries to index someList with 3.75 and fails. In Java, the assignment fails ("double cannot be converted to Integer") at javac time.

autocasting from int to long does not work for java

Trying to compile a quark file without the files for referenced packages crashes the compiler

We should catch this and throw a reasonable error like "package foo must also be compiled" or similar

Instead, currently we get the following crash:

Traceback (most recent call last):
  File "/Users/janicekarin/virtualenv/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/janicekarin/quark/quark/compiler.py", line 566, in main
    exit(_main(docopt(main.__doc__)))
  File "/Users/janicekarin/quark/quark/compiler.py", line 542, in _main
    c.compile()
  File "/Users/janicekarin/quark/quark/compiler.py", line 497, in compile
    self.root.traverse(use)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 93, in traverse
    c.traverse(visitor, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/ast.py", line 94, in traverse
    leave(self, *args, **kwargs)
  File "/Users/janicekarin/quark/quark/compiler.py", line 233, in leave_Type
    if n.text in type.env:
AttributeError: 'NoneType' object has no attribute 'env'

Errors sometimes report the line after the issue

I omitted a ; at the end of a line and the error reported the issue was on the subsequent line.

Here's the Quark code:

package foobar{

}

void main() {
    int i=0;
    String j="foo";
    while (j == "foo") {
        i=i+1;
        j="bar"
        print(i.toString() + ": Hello World!");
    }
}

Note that the issue is on line 10, j="bar"

Here's the error when I try to compile:

examples/comparison.q:11:9: Rule 'SEMI' didn't match at 'print(i.toString() +' (line 11, column 9).

Quark should only generate code that's actively used

Some building block code is getting generated whether or not it's actively used. For example, the python generator currently includes the following in every generated .py file whether or not lists and maps are used:

_Map = dict
class _List(list):
    def __repr__(self):
        return "[%s]" % ", ".join([str(e) for e in self])

quark macros should not evaluate their arguments more than once

it's easy to write a macro that evaluates an expression more than once -- dangerous when used with methods that have side-effects

Support for floats

Currently Quark only supports integer types. It should support non-integer numbers too.

It would be nice to support i++ style incrementing

It would be nice to support i++ to increment integer i.

It would be nice to support exponential notation for numbers

It would be nice to support writing large/small numbers (or any numbers) using exponential notation - ex: 3.2x10^-14 should be supported as 3.2E-14 or 3.2e-14

Compiler crash calling List<>.set with too few arguments

package foo {}

void main() {
    List<String> asdf = new List<String>();
    asdf.__set__("aadsf");
}

yields

$ quark compile hello.q 
Traceback (most recent call last):
  File "/Users/ark3/.virtualenvs/datawire/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/ark3/Dropbox/datawire/quark/quark/command.py", line 150, in call_main
    exit(main(docopt(__doc__)))
  File "/Users/ark3/Dropbox/datawire/quark/quark/command.py", line 116, in main
    res = compiler._main(compiler_args)
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 542, in _main
    c.compile()
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 515, in compile
    self.emit(backend)
  File "/Users/ark3/Dropbox/datawire/quark/quark/compiler.py", line 519, in emit
    self.root.traverse(backend)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 94, in traverse
    leave(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 94, in leave_Root
    functions.append(f.match(self.dfnr))
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 238, in match_Function
    body = " %s" % m.body.match(self.stmtr) if m.body else ";"
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 331, in match_Block
    return "{%s}" % indent("\n".join([s.match(self) for s in b.statements]))
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 308, in match_ExprStmt
    return "%s;" % stmt.expr.match(self.exprr)
  File "/Users/ark3/Dropbox/datawire/quark/quark/ast.py", line 97, in match
    return self.lookup(transform, "match")(self, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 357, in match_Call
    return self.invoke(type, c.expr, [a.match(self) for a in c.args])
  File "/Users/ark3/Dropbox/datawire/quark/quark/dispatch.py", line 52, in __call__
    return method(self.object, *args, **kwargs)
  File "/Users/ark3/Dropbox/datawire/quark/quark/java.py", line 449, in invoke
    env[p.name.text] = args[idx]
IndexError: list index out of range

Quark doesn't check usage against declared type

Quark compiles the following code into all three languages:

void main() {
    int i=0;
    while (i<2.5) {
        i=i+0.5;
        print(i.toString() + ": Hello World!");
    }
}

The generated Python and Javascript work, treating i as a float and printing 0.5, 1, 1.5, etc.

The generated Java code throws errors because it wants i to be an integer:

$ javac Functions.java
Functions.java:5: error: incompatible types: double cannot be converted to Integer
            i = (Integer) ((i) + (0.5));
                          ^
1 error

Declaring variables before they are used results in errors in the generated Python code

When I declare variables before assigning them values within main(), the Quark compiler generates code in all three languages and the Javascript and Java code works as expected. Python, however, gives the following error:

$ python usingMacros.py
Traceback (most recent call last):
  File "usingMacros.py", line 13, in <module>
    main()
  File "usingMacros.py", line 5, in main
    latitude;
UnboundLocalError: local variable 'latitude' referenced before assignment

If I change the code so that the variables are declared at the same time they are assigned, all three languages are happy and the code works.

usingMacros.q declares variables up front, usingMacros2.q declares them at assignment. Both use a macro defined in macros.q.

macros.txt
usingMacros.txt
usingMacros2.txt

Note: I have successfully declared before assignment inside class definitions.

Cryptic error message when you put initialization code outside of a constructor

The following error message was thrown when I had initialization code defined outside of a constructor as in the attached file. It should be improved to better indicate the actual problem.

examples/convertUnits2.q:16:31: Rule 'name' didn't match at '("meter",1);
' (line 16, column 31).
convertUnits2.txt

Note: you cannot directly attach a .q file - just change the extension.

@doc directives above methods are not translated into the Javadoc

@doc directives for packages, classes, and properties appear to be working but @doc directives for methods are currently ignored.

The package description is also missing. For info on how to set it, see the following:

http://docs.oracle.com/javase/specs/jls/se7/html/jls-7.html#jls-7.4.1

Quark compiler should clean up package directories before repopulating them

We shouldn't touch the top-level output directory as other legitimate files might be there, but Quark should clean out any pre-existing files in a package directory before repopulating it. In particular, Javadoc picks up any pre-existing files and assumes that they are part of the package being documented when javadoc is generated.

Add support for booleans

Most languages include a boolean type. Quark should as well.

Compiler should catch missing function name

I have the following class defined:

    class Conversion {
       float answer;
       float Convert ( Unit input, Unit output){
          float answer = input.value.__mul__((input.baseValue.__mul__(output.baseValue)));
          return answer;
       } 
    }
}

I accidentally used foo=conversion(meters,feet) instead of foo=conversion.Convert(meters,feet) when referencing it. The compiler should catch this and complain.

Quark files without packages do not compile for Java or Python

Quark files that don't include a package declaration work without any issues in Javascript but throw errors when you try to generate either Java or Python code:

Here's the Java error:

$ quark --java ~/quark-examples/java examples/while.q
quark (compiler): wrote /Users/janicekarin/quark-examples/java/src/main/java/io/datawire/quark_runtime.java
quark (compiler): wrote /Users/janicekarin/quark-examples/java/src/main/java/Functions.java
Traceback (most recent call last):
  File "/Users/janicekarin/virtualenv/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/janicekarin/quark/quark/compiler.py", line 566, in main
    exit(_main(docopt(main.__doc__)))
  File "/Users/janicekarin/quark/quark/compiler.py", line 542, in _main
    c.compile()
  File "/Users/janicekarin/quark/quark/compiler.py", line 516, in compile
    backend.write(target)
  File "/Users/janicekarin/quark/quark/java.py", line 67, in write
    firstPackageName, firstPackageList = self.packages.items()[0]
IndexError: list index out of range

and the similar results when trying to generate Python code:

$ quark --python ~/quark-examples/python examples/while.q
Traceback (most recent call last):
  File "/Users/janicekarin/virtualenv/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/janicekarin/quark/quark/compiler.py", line 566, in main
    exit(_main(docopt(main.__doc__)))
  File "/Users/janicekarin/quark/quark/compiler.py", line 542, in _main
    c.compile()
  File "/Users/janicekarin/quark/quark/compiler.py", line 516, in compile
    backend.write(target)
  File "/Users/janicekarin/quark/quark/python.py", line 284, in write
    firstPackageName, firstPackageList = self.packages.items()[0]
IndexError: list index out of range

The workaround is to define an empty package inside files that do not need a real package.

~a partially supported; use causes compiler errors

~ is listed as a valid symbol and has a defined rule in parser.py but it isn't fully implemented and trying to use a=~b in Quark code results in similar errors in all three languages. Here's the Javascript version of the error:

$ quark --javascript ~/quark-examples/javascript examples/tilde.q
Traceback (most recent call last):
  File "/Users/janicekarin/virtualenv/bin/quark", line 9, in <module>
    load_entry_point('datawire-quark==0.1.0', 'console_scripts', 'quark')()
  File "/Users/janicekarin/quark/quark/compiler.py", line 566, in main
    exit(_main(docopt(main.__doc__)))
  File "/Users/janicekarin/quark/quark/compiler.py", line 541, in _main
    c.parse(src, fd.read())
  File "/Users/janicekarin/quark/quark/compiler.py", line 449, in parse
    self.parse_r(name, text)
  File "/Users/janicekarin/quark/quark/compiler.py", line 456, in parse_r
    file = self.parser.parse(text)
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 244, in parse
    return self._parse_or_match(text, pos, 'parse')
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 278, in _parse_or_match
    return self.visit(getattr(self.grammar, method_name)(text, pos=pos))
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/virtualenv/lib/python2.7/site-packages/parsimonious/nodes.py", line 208, in visit
    return method(node, [self.visit(n) for n in node])
  File "/Users/janicekarin/quark/quark/grammar.py", line 27, in decorator
    result = action(self, node, children)
  File "/Users/janicekarin/quark/quark/parser.py", line 357, in visit_uop
    return Name(self.unary_aliases[op])
parsimonious.exceptions.VisitationError: KeyError: '~'

Parse tree:
<Node called "uop" matching "~">  <-- *** We were here. ***
    <Node called "TWIDDLE" matching "~">
        <Node called "_" matching "">
        <Node matching "~">
        <Node called "_" matching "">

Quark allows you to reference an undefined list; generated code throws errors in all languages

Quark compiles code that adds elements to a list that has not been initialized but the generated code throws errors in all languages.

Quark code:

package likes{

    class Color {
        List<String> favorites;
    } 
}

void main() {
    int i=0;
    likes.Color myColors=new likes.Color();
    myColors.favorites.add("red");
    myColors.favorites.add("green");
    myColors.favorites.add("blue");
    myColors.favorites.add("purple");
    ...
}

Error in Javascript:

/Users/janicekarin/quark-examples/javascript/list.js:9
    ((myColors).favorites).push("red");
                          ^
TypeError: Cannot read property 'push' of null
    at main (/Users/janicekarin/quark-examples/javascript/list.js:9:27)
    at Object.<anonymous> (/Users/janicekarin/quark-examples/javascript/list.js:25:1)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3

Error in Python:

  File "list.py", line 26, in <module>
    main()
  File "list.py", line 10, in main
    ((myColors).favorites).append("red");
AttributeError: 'NoneType' object has no attribute 'append'

in Java:

Exception in thread "main" java.lang.NullPointerException
    at Functions.main(Functions.java:5)
    at Functions.main(Functions.java:20)

Java cannot find symbol, illegal start of type errors from Quark file that compiles and runs in Python and Javascript

convertUnits2.txt

I am getting the following error when I try to run the Java generated by the attached Quark file:

(virtualenv)Janices-MacBook-Pro:convert2 janicekarin$ javac Functions.java
Functions.java:5: error: cannot find symbol
        convert2.Unit length1 = new convert2.Unit("length", "meter", 20.0);
                ^
  symbol:   class Unit
  location: package convert2
Functions.java:5: error: cannot find symbol
        convert2.Unit length1 = new convert2.Unit("length", "meter", 20.0);
                                            ^
  symbol:   class Unit
  location: package convert2
Functions.java:6: error: cannot find symbol
        convert2.Unit length2 = new convert2.Unit("length", "foot", 0);
                ^
  symbol:   class Unit
  location: package convert2
Functions.java:6: error: cannot find symbol
        convert2.Unit length2 = new convert2.Unit("length", "foot", 0);
                                            ^
  symbol:   class Unit
  location: package convert2
Functions.java:7: error: cannot find symbol
        convert2.Conversion conversion = new convert2.Conversion();
                ^
  symbol:   class Conversion
  location: package convert2
Functions.java:7: error: cannot find symbol
        convert2.Conversion conversion = new convert2.Conversion();
                                                     ^
  symbol:   class Conversion
  location: package convert2
Functions.java:9: error: illegal start of type
        System.out.println(((((Float.toString((length1).value)) + ((length1).label)) + (" is ")) + (Float.toString((length2).value))) + (" feet."));
                                                                  ^
7 errors

The same file works in both Python and Javascript.

Default values are not checked against type declarations

I was able to compile the following without complaint:

    class Foot extends Unit {
       String type=1;
       String system="English";
       float baseValue="3.2808399";
    }

I expected complaints that 1 is not a string and "3.2808399" is not a float. Instead, I get complaints from the Java compiler:

javac Functions.java
./convert/Foot.java:4: error: incompatible types: int cannot be converted to String
    public String type = (String) (1);
                                  ^
./convert/Foot.java:6: error: incompatible types: String cannot be converted to float
    public float baseValue = (float) ("3.2808399");
                                     ^
2 errors

The Python and Javascript work as is and return the same results as if I had supplied a float for Foot.baseValue.