beeware / batavia Goto Github PK

View Code? Open in Web Editor NEW

1.4K 1.4K 425.0 22.76 MB

A JavaScript implementation of the Python virtual machine.

Home Page: http://pybee.org/batavia

License: Other

JavaScript 57.14% HTML 0.62% Python 42.15% CSS 0.09%

batavia's Introduction

BeeWare

BeeWare is a collection of tools and libraries for building and distributing native applications in Python.

For an introduction to the full BeeWare suite, we recommend running the BeeWare Tutorial.

Community

You can talk to the BeeWare community through:

We foster a welcoming and respectful community as described in our BeeWare Community Code of Conduct.

Contributing

If you experience problems with BeeWare, log them on GitHub. If you want to contribute code, please fork the code and submit a pull request.

Translations

We manage translations using Weblate.

If you'd like to contribute to the translation effort, join the #translations channel on Discord and introduce yourself!

batavia's People

Contributors

Stargazers

Watchers

Forkers

bussiere chubbymaggie xsongx yourmoonlight candeira kashyap32 szborows janusnic glasnt pombredanne timmyhadwen mcanthony moaxey ith4cker ojii thefon postsql mcgallaspy misterrios gasman phildini splab andrzejkrawczyk markush asser funkybob henrikossipoff eichin pietertolsma flaviabastos winza83 dawnpaladin adamchainz tfors briankaywalker mauritt schuyler1d kwokster10 rileyrustad srink netoxico camdroid miki725 mbrowngraham darrylhall lrhewus annanymouse bellisk phoikoi the-mean-green-data-machine jschwarzwalder jrabbit aedunn6 olga0911 li3n3 phoebebauer tjsoftworks raehart lawlheart michellejl hwesta xpostudio4 madeleinenyhagen-zz jlebzelter freakboy3742 jubjubjubjub willingc 314developer asher-scott greeve joslarson kennethlove timb07 kccarr kgoodwill sierrabrown jcronyn havocbane deltadada dcstlouis slick666 dori-amara vector919 flipperpa kojoidrissa asauber wtheisen danlindeman jstoebel nanaanim27 msbluphire jjooeh danielchennessy krantzinator ewdurbin james-prior ewjoachim pausz swenson jasonthomasdata

batavia's Issues

Javascript <-> Python bridge

We should be able to pass all kinds of values back and forth between Python and Javascript, like in the testserver, and Pythonize and de-Pythonize.

Possible bug/possible me? Errors running tests

Hi @freakboy3742 @glasnt @phildini:

I ran the tests last night after doing all the things in the Getting Started, and 21 tests failed, mostly in tests.modules.test_math.MathTests. I scrolled through the tests that failed and it seems the tests are failing with these errors:

'AttributeError'> : 'module' object has no attribute 'isclose'
'AttributeError'> : 'module' object has no attribute 'gcd'
<class 'TypeError'> : 'float' object cannot be interpreted as an integer
AttributeError: 'module' object has no attribute 'inf'

Sample screenshot:

It seems unlikely that tests are failing only for me :) Any ideas as to what I don't have installed or need to do differently? I checked and I don't have uncommitted changes from DjangoCon Europe, and my fork is in sync with the main branch.

Complete implementation of operations on standard types

Javascript has a specific set of allowed operator and operations. Python has a set of operators and operations as well. However, the two don't match exactly. For example, Javascript behavior for adding/multiplying with Strings leads to behaviour that is quite distinct to the behavior provided by Python.

In order to replicate Python behavior in Javascript, Python's logic for all the basic operations needs to be implemented in the Batavia support library.

The test suite contains around 10000 tests in the that are currently marked as "expected failures". These reflect an attempt to do "every operation between every base data type". The task: pick a data type, and write the implementation of one of the math operation or comparison operators.

The tests will pass (and become unexpected successes) when the output of Python code doing that operation is the same when run through CPython and Batavia - and I mean byte-identical, down to the text and punctuation of the error message if appropriate.

If you want to see an example of what is involved, check out the implementation of add for integers. If you have an int, Python will allows you to add an int or a float to it; all other types raise a TypeError. The list of operations on int that still need to be implemented can be found here; if you add a new operation, delete the line that corresponds to that test, and the test will report as a pass, rather than an expected fail.

len function error always returns "0 given"

If 2 or more arguments are passed to the len function it errors and always states "0 given"

Remove non-Py3 builtins

For example:

NotImplementedError: Builtin Batavia function 'reduce' not implemented

whereas:

>>> reduce
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'reduce' is not defined

Maybe this is intentional? If not should they be removed to match Python behavior?

so far found these:

reduce
unichr

Syntax highlighting on Batavia's initial demo page

There are JavaScript plugins that give text boxes syntax highlighting.
Great for code readability, highlighting keywords, functions and different types.

CPython 3.6+ is switching to a 16-bit wordcode format

Via @ncoghlan, reporting beeware/voc#132:

This is an early heads up that CPython will be switching to a 16-bit wordcode format as of 3.6a2: http://bugs.python.org/issue26647

The change itself is in https://hg.python.org/cpython/rev/3a57eafd8401

In theory this should be a transparent change for any project that interacts with compiled code solely through the dis module, but since VoC transpiles CPython bytecode I figured you may want to take a closer look at the implications.

Makefile does not work with Windows

The current Makefile does not support windows. The job 'batavia.js' fails on windows because windows does not support to the 'cat' command.

EDIT: The 'clean' job will also fail and should be fixed in a similar manner.

Travis CI issues with 3.4.2

The master branch is currently broken since 6ea3cff

The travis log looks something like this:

3.4.2 is not installed; attempting download
bzip2: (stdin) is not a bzip2 file.
tar: Child returned status 2
tar: Error is not recoverable: exiting now
The command "sudo tar xjf python-3.4.2.tar.bz2 --directory /" failed and exited with 2 during .

However, this "just works" on VOC

The reason is that they are using two separate images. Batavia's travis file specifies

sudo: required
dist: trusty
language: python
python:
  - "3.4.2"
  - "3.4.4"
...

But VOC specifies

language: python
python:
  - "3.4.2"
  - "3.4.4"
...

The difference in the dist values means that Batavia is running on a Ubuntu 14.04.3 LTS image while VOC is Ubuntu 12.04.5 LTS

I'm guessing that because we're playing with PhantomJS in batavia that 14.04 is required. However, if that's the case, we might have to do a bit more to help the installation of 3.4.2 in this environment.

I'm not sure if this is a known upstream issue, but I'll work out how to submit a ticket for Travis

Use isbataviainstance where check classes are needed.

Rename the function to isbataviainstance.

https://github.com/pybee/batavia/blob/master/batavia/utils.js#L48

Example of use: https://github.com/ramiroluz/batavia/blob/8708f51d7b183d632c5465a26481675d5d52c7a4/batavia/types/Bool.js#L301

I can do that, just let me know if you don't think it is a good idea.

InplaceStrOperationTests - TypeError message for string.

I am trying to implement some test cases but I have been facing with some assertion problems.
This a diff from a test case that I tried to implement 'test_subtract_int' for string. I raise an exception
message ' unsupported operand type(s) for -: 'str' and 'int' ' and the test for BinaryStrOperationTests passes with no issues but the test for InplaceStrOperationTests expects for a different exception
message.This is the message that InplaceStrOperationTests is expecting ' unsupported operand type(s) for -=: 'str' and 'int' ' . The equal sign is added from previous message.

I do not see how I can change the message dynamically depending on the class being test. Am I missing something or a change in the expected message is needed?

Thank you.

diff --git a/batavia/utils.js b/batavia/utils.js
index 73e000e..26f19e7 100644
--- a/batavia/utils.js
+++ b/batavia/utils.js
@@ -468,7 +468,11 @@ batavia.operators = {
         return result;
     },
     SUBTRACT: function(a, b) {
-        return a - b;
+        if (typeof a === 'string') {
+            throw new batavia.builtins.TypeError("unsupported operand type(s) for -: 'str' and 'int'");
+        } else {
+            return a - b;
+        }
     },
     SUBSCR: function(a, b) {
         if (b instanceof Object) {
diff --git a/tests/datatypes/test_str.py b/tests/datatypes/test_str.py
index 270902f..8c90d68 100644
--- a/tests/datatypes/test_str.py
+++ b/tests/datatypes/test_str.py
@@ -259,7 +259,6 @@ class BinaryStrOperationTests(BinaryOperationTestCase, TranspileTestCase):
         'test_subtract_dict',
         'test_subtract_float',
         'test_subtract_frozenset',
-        'test_subtract_int',
         'test_subtract_list',
         'test_subtract_none',
         'test_subtract_set',
@@ -435,7 +434,6 @@ class InplaceStrOperationTests(InplaceOperationTestCase, TranspileTestCase):
         'test_subtract_dict',
         'test_subtract_float',
         'test_subtract_frozenset',
-        'test_subtract_int',
         'test_subtract_list',
         'test_subtract_none',
         'test_subtract_set',

8 errors when running tests on Fedora23

I am on Fedora 23:
Linux r2d2 4.4.6-301.fc23.x86_64 #1 SMP Wed Mar 30 16:43:58 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Used anaconda to install a python 3.5 environment.
I downloaded phantomjs for linux 64bit version 2.1.1

Followed the instructions at here and successfully ran the test server. When I run the tests python setup.py test I get 8 errors:


======================================================================
FAIL: test_subscr_dict (tests.datatypes.test_list.BinaryListOperationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 637, in func
    self.assertBinaryOperation(x=value, y=example, operation=operation, format=self.format)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 658, in assertBinaryOperation
    """ % kwargs, "Error running %(operation)s with x=%(x)s and y=%(y)s" % kwargs)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '### [30 chars] indices must be integers, not dict\n    test.py:3\n' != '### [30 chars] indices must be integers or slices, not dict\n    test.py:3\n'
  ### EXCEPTION ###
- TypeError: list indices must be integers, not dict
+ TypeError: list indices must be integers or slices, not dict
?                                         ++++++++++
      test.py:3
 : Global context: Error running x[y] with x=[] and y={}

======================================================================
FAIL: test_subscr_float (tests.datatypes.test_list.BinaryListOperationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 637, in func
    self.assertBinaryOperation(x=value, y=example, operation=operation, format=self.format)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 658, in assertBinaryOperation
    """ % kwargs, "Error running %(operation)s with x=%(x)s and y=%(y)s" % kwargs)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '### [31 chars]indices must be integers, not float\n    test.py:3\n' != '### [31 chars]indices must be integers or slices, not float\n    test.py:3\n'
  ### EXCEPTION ###
- TypeError: list indices must be integers, not float
+ TypeError: list indices must be integers or slices, not float
?                                         ++++++++++
      test.py:3
 : Global context: Error running x[y] with x=[] and y=2.3456

======================================================================
FAIL: test_subscr_list (tests.datatypes.test_list.BinaryListOperationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 637, in func
    self.assertBinaryOperation(x=value, y=example, operation=operation, format=self.format)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 658, in assertBinaryOperation
    """ % kwargs, "Error running %(operation)s with x=%(x)s and y=%(y)s" % kwargs)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '### [30 chars] indices must be integers, not list\n    test.py:3\n' != '### [30 chars] indices must be integers or slices, not list\n    test.py:3\n'
  ### EXCEPTION ###
- TypeError: list indices must be integers, not list
+ TypeError: list indices must be integers or slices, not list
?                                         ++++++++++
      test.py:3
 : Global context: Error running x[y] with x=[] and y=[]

======================================================================
FAIL: test_subscr_none (tests.datatypes.test_list.BinaryListOperationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 637, in func
    self.assertBinaryOperation(x=value, y=example, operation=operation, format=self.format)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 658, in assertBinaryOperation
    """ % kwargs, "Error running %(operation)s with x=%(x)s and y=%(y)s" % kwargs)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '### [34 chars]ices must be integers, not NoneType\n    test.py:3\n' != '### [34 chars]ices must be integers or slices, not NoneType\n    test.py:3\n'
  ### EXCEPTION ###
- TypeError: list indices must be integers, not NoneType
+ TypeError: list indices must be integers or slices, not NoneType
?                                         ++++++++++
      test.py:3
 : Global context: Error running x[y] with x=[] and y=None

======================================================================
FAIL: test_subscr_str (tests.datatypes.test_list.BinaryListOperationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 637, in func
    self.assertBinaryOperation(x=value, y=example, operation=operation, format=self.format)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 658, in assertBinaryOperation
    """ % kwargs, "Error running %(operation)s with x=%(x)s and y=%(y)s" % kwargs)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '### [29 chars]t indices must be integers, not str\n    test.py:3\n' != '### [29 chars]t indices must be integers or slices, not str\n    test.py:3\n'
  ### EXCEPTION ###
- TypeError: list indices must be integers, not str
+ TypeError: list indices must be integers or slices, not str
?                                         ++++++++++
      test.py:3
 : Global context: Error running x[y] with x=[] and y=""

======================================================================
FAIL: test_subscr_tuple (tests.datatypes.test_list.BinaryListOperationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 637, in func
    self.assertBinaryOperation(x=value, y=example, operation=operation, format=self.format)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 658, in assertBinaryOperation
    """ % kwargs, "Error running %(operation)s with x=%(x)s and y=%(y)s" % kwargs)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '### [31 chars]indices must be integers, not tuple\n    test.py:3\n' != '### [31 chars]indices must be integers or slices, not tuple\n    test.py:3\n'
  ### EXCEPTION ###
- TypeError: list indices must be integers, not tuple
+ TypeError: list indices must be integers or slices, not tuple
?                                         ++++++++++
      test.py:3
 : Global context: Error running x[y] with x=[] and y=(1, 2.3456, 'another')

======================================================================
FAIL: test_creation (tests.datatypes.test_dict.DictTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/datatypes/test_dict.py", line 33, in test_creation
    """)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '{}\n' != "{'a': 1}\n"
- {}
+ {'a': 1}
 : Global context

======================================================================
FAIL: test_getitem (tests.datatypes.test_dict.DictTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/swacker/pycon/beeware/batavia/tests/datatypes/test_dict.py", line 43, in test_getitem
    """)
  File "/home/swacker/pycon/beeware/batavia/tests/utils.py", line 448, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: "False\nTrue\n### EXCEPTION ###\nKeyError: 'a'\n    test.py:5\n" != 'True\nFalse\n1\n'
+ True
  False
+ 1
- True
- ### EXCEPTION ###
- KeyError: 'a'
-     test.py:5
 : Global context

----------------------------------------------------------------------
Ran 4146 tests in 548.626s

FAILED (failures=8, expected failures=3262)

Why not add option to transpile python to js instead of a VM

compile python to js
use google closure compile to minimise it
implement python3.5 features including async/await
I think that will be the betterway for perf & reducing the complexity, we can also add plugins for the system, been looking for a python solution in web, checked many but all have problems; believe me am also worked on web frontend technologies, if we are going to focus more on VM implementation of python like brython, it will not be effective to the pythonistas who are trying to do python in browser

Register Batavia with Bower

Batavia is a Javascript tool; we need to add a bower configuration file so that it can be easily added to the asset configuration for a new project.

compile_stdlib.py improvements

The compile_stdlib.py is a program that converts standard library .py files into .js files by compiling them to bytecode (pyc), and then encoding that in JS.

This is completely a Python program, and most of the these tasks should require little or no JS knowledge.

There are a number of improvements that we would like to make to this tool.

Accept arguments to specify what libraries to include
Modify it to produce the batavia.js file itself, patching it together, rather than doing it in the Makefile
Have arguments for common library collections (e.g., --basic-stdlib for a lighter standard library, once we have more of it implemented)
Your other cool ideas go here

Bytecode changes for dict in python 3.6 causes batavia.core.Dict to fail

This is a continuation of #26 that was fixed in 3.4 and 3.5.. Python 3.6 fails for this same input but for different reasons.

With python 3.5 the bytecode for storing data to the dict has changed to:

2 0 LOAD_CONST 1 ('a')
3 LOAD_CONST 2 (1)
6 LOAD_CONST 3 ('b')
9 LOAD_CONST 4 (2)
12 BUILD_MAP 2
15 STORE_FAST 0 (lonk)
causing dicts to be empty.

To reproduce use (in the testbed):

foo = {'a': 1, 'b': 2}
print('bar', foo)
Expected result: bar 1
Actual result: bar

Create better styles for Batavia demo page

We can add some styles in the same style as the Batavia docs page, here: http://pybee.org/contributing/what/

We can add some font styles and other pretty changes to give a better first impression.

Sample.py breaks on second run

When running the testserver and running sample.py via its button, it works fine the first time, but throws an exception on second and subsequent button pushes:

Caught exception during execution: TypeError: Cannot read property 'hasOwnProperty' of undefinedCaught exception during execution: TypeError: this.manage_block_stack is not a functionCaught exception during execution: null

This was on Chrome 44.0.2403.125 (64-bit) on Ubuntu 14.04.

Test "test_set_of_ints_and_strs (tests.datatypes.test_set.SetTests)" is failing

Traceback (most recent call last):
  File "/Users/johnny/Sandbox/Batavia/Tutorial/batavia/tests/datatypes/test_set.py", line 10, in test_set_of_ints_and_strs
    """, substitutions={"{1, '1'}": ["{'1': 1}"]})
  File "/Users/johnny/Sandbox/Batavia/Tutorial/batavia/tests/utils.py", line 550, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: "{1, '1'}\n" != "{'1', 1}\n"
- {1, '1'}
+ {'1', 1}
 : Function context

`set` iterator yields floats as strings, or possible ordering issue

My bytes implementation was initialising objects from set fixtures perfectly, and then I rebased on master and these errors happened:

  >>> f = bytes
  >>> x = {1, 2.3456, "another"}
  >>> f(x)
- <class 'TypeError'> : 'float' object cannot be interpreted as an integer
?                        ^^^^
+ <class 'TypeError'> : 'str' object cannot be interpreted as an integer
?                        ^ +

Since the bytes() initialisation works fine when iterating over dicts, lists and tuples, I suspect the reworking of set objects on top of the new dicts here: #288, or maybe some ordering/substitutions issue.

@swenson: if you don't mind, I'd like to dib this one, and learn more about the dict/set implementation. I looked at the commit yesterday, and was intrigued.

Run sample link going to wrong place

For some reason I can't figure out, the Batavia test page opens a link to the Batavia page when it is clicked on. Very strange!

Bytecode changes for dict in python 3.5 causes batavia.core.Dict to fail

With python 3.5 the bytecode for storing data to the dict has changed to:

  2           0 LOAD_CONST               1 ('a')
              3 LOAD_CONST               2 (1)
              6 LOAD_CONST               3 ('b')
              9 LOAD_CONST               4 (2)
             12 BUILD_MAP                2
             15 STORE_FAST               0 (lonk)

causing dicts to be empty.

To reproduce use (in the testbed):

foo = {'a': 1, 'b': 2}
print('bar', foo)

Expected result: bar 1
Actual result: bar

`python setup.py test` results in 725 failures [Windows]

Your contributor guide says "If you see any lines that end FAIL, ERROR, or unexpected success, [...] please get in touch, because you may have found a problem." So that's what I'm doing. Running python setup.py test results in "failures = 725, expected failures = 3403".

That page only lists instructions for doing this on OSX, Ubuntu, and Rasbperry Pi, and I'm doing this on Windows, so maybe that's the problem? I'm running Windows 10 build 14342 (just released onto the Slow ring), Python 3.5.1, and PhantomJS 2.1.1.

Am I doing something wrong? I want to contribute. How can I proceed?

Add support for importing a module by HTTP

At present, if you want to import a module, it must either be shipped as part of Batavia, or included as a payload in a <script> tag with id=batavia-module.name.here.

It would also be desirable to be able to import over HTTP. When the Batavia VM is initialised, you would provide a URL endpoint; if a module can't be found locally, the __import__ mechanism would hit that URL, request the module, then inject it into the sys.modules namespace and resume execution.

Use pypy or brython implementation for builtins and standard lib

Hi,

I was wondering if it makes sense to use an implementation for the python builtins that's actually implemented in python.

For example, pypy has the following https://bitbucket.org/pypy/pypy/src/35c27789e35368ac3ebefedd83af0829d2cf89d4/pypy/module/__builtin__/

Brython similarly has pure python implementations for large parts of the standards library (not sure about builtins, though).

Maybe it would be good to create a distinguished repo for this kind of reimplementation of python in pure python.

Cheers,

Wolf

Class inheritance errors out

When importing the datetime module, which went something like this:

class date:
  ...
class datetime(date):
  ...

the second line errors out with something like date not found.

Attempting to get a small reproducer gives a different error:

class date:
    pass

class d2(date):
    pass

print(d2)

gives the error TypeError: Cannot read property 'prototype' of undefined. Different, but also needs to be addressed.

Implement regular expression library (re)

Most of re is written in C in CPython, meaning we have to find an alternative.

One possibility is PyPy's implementation, which appears to be mostly written in RPython:

https://bitbucket.org/pypy/pypy/src/b0e70c0c2ac0d1b8a066b64eb94bd3b858a9be8f/rpython/rlib/rsre/?at=default

We could possibly, with some modifications, load this, similarly to how we load in ouroboros modules.

Inconsistent failures during test run on win for module 'test_filter'

I wanted to get involved with the project so thought I would start by running the test suite; however, I'm seeing some strange behavior.

running setup.py test caused the test to get stuck in test_filter.test_str_range
I ran just that module and got several Fails, everyone in the FilterTests class but also several tests that weren't in the not_implemented list, like test_str_range, test_tuple_dict, etc.
The errors associated with the FilterTest class all gave the foolowing type of error:

======================================================================
FAIL: test_int (tests.builtins.test_filter.FilterTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\onedrive\web\projects\tutorial\batavia\tests\utils.py", line 485, in assertCodeExecution
    args=args
  File "d:\onedrive\web\projects\tutorial\batavia\tests\utils.py", line 288, in runAsJavaScript
    on_fail="Unable to create webpage."
ValueError: write to closed file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\onedrive\web\projects\tutorial\batavia\tests\builtins\test_filter.py", line 35, in test_int
    self.assertCodeExecution(self.base_code % ("[1, 2, 3]", "x * 2"), run_in_function=False)
  File "d:\onedrive\web\projects\tutorial\batavia\tests\utils.py", line 488, in assertCodeExecution
    self.fail(e)
AssertionError: write to closed file

But if I only run those tests (... tests.builtins.test.filter.FilterTests), they all pass.
What can be causing this?

Running python 3.4.4 on Win 7

str() takes at most 1 argument (2 given)

Essentially I just tried to do the tutorial and putting
print('Hello World')
into the text area and clicking run immediately resulted in a trace-back. Is this due to django running as python 2. Because the Shebang says python not python3.

By the way this is an awesome concept and so small and light weight! I am going to start testing if I can do AJAX using it; has this already been tested; are there some examples?

hex() produces incorrect output

hex() produces incorrect output.

>>> print(hex(3735928559))
0x3735928559

instead of giving 0xdeadbeef

This is because, hex conversion is done by toString() method and toString() method is overridden here in Int.js

So, the implementation of hex() in builtins.js will always return

return this.val.toFixed(0);

which is of course, not the hexadecimal value of the integer.

Would like to hear your thoughts on this, before I can work on a PR 😄

test_add_none does not exist.

test_add_none is in the not_implemented list in file test_str.py.

if I run "python setup.py test -s tests.datatypes.test_str.BinaryStrOperationTests.test_add_none" to work on this test case I get 'type object 'BinaryStrOperationTests' has no attribute 'test_add_none''

Create functional tests

Issues like #154 means we should really be creating functional tests to ensure that python scripts that run through batavia are the right output.

I think we have the mechanism for this already, but having more would be helpful, as we completely flesh out more functionality.

Complete standard library

We should be able to load the complete standard library.

Here is a list of the potential base modules we should be able to run that are present in Ouroboros, sorted (roughly) in the order of difficulty.

These are blocked for various reasons:

time.mktime needs to handle tm_isdst

Seems like Batavia's mktime is assuming the wrong timezone - all expected values are exactly one hour away!

======================================================================
FAIL: test_mktime (tests.modules.test_time.TimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bruno/PycharmProjects/batavia/tests/modules/test_time.py", line 203, in test_mktime
    self.assertCodeExecution(test_str)
  File "/home/bruno/PycharmProjects/batavia/tests/utils.py", line 504, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '>>> [99 chars]0))\n7200.0\n>>> seq = time.struct_time((1970,[53 chars].0\n' != '>>> [99 chars]0))\n10800.0\n>>> seq = time.struct_time((1970[55 chars].0\n'
  >>> import time
  >>> seq = (1970, 1, 1, 0, 0, 0, 0, 0, 0)
  >>> time.mktime((1970, 1, 1, 0, 0, 0, 0, 0, 0))
- 7200.0
+ 10800.0
  >>> seq = time.struct_time((1970, 1, 1, 0, 0, 0, 0, 0, 0))
  >>> time.mktime(seq)
- 7200.0
+ 10800.0
 : Global context

======================================================================
FAIL: test_mktime_args (tests.modules.test_time.TimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bruno/PycharmProjects/batavia/tests/modules/test_time.py", line 226, in test_mktime_args
    self.assertCodeExecution(test_str)
  File "/home/bruno/PycharmProjects/batavia/tests/utils.py", line 504, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '>>> [124 chars]n10378800.0\n>>> time.mktime((1970, 1, 5, 0, 0[256 chars].0\n' != '>>> [124 chars]n10375200.0\n>>> time.mktime((1970, 1, 5, 0, 0[259 chars].0\n'
  >>> import time
  >>> time.mktime((1999, 1, 1, 0, 0, 0, 0, 0, -1))
  915156000.0
  >>> time.mktime((1970, 5, 1, 0, 0, 0, 0, 0, 1))
- 10378800.0
?     ^^
+ 10375200.0
?     ^^
  >>> time.mktime((1970, 1, 5, 0, 0, 0, 0, 0, -1))
- 352800.0
?   ^^
+ 356400.0
?   ^^
  >>> time.mktime((1970, 1, 1, 5, 0, 0, 0, 0, -1))
- 25200.0
?  ^^
+ 28800.0
?  ^^
  >>> time.mktime((1970, 1, 1, 0, 5, 0, 0, 0, -1))
- 7500.0
+ 11100.0
  >>> time.mktime((1970, 1, 1, 0, 0, 5, 0, 0, -1))
- 7205.0
+ 10805.0
  >>> time.mktime((1970, 1, 1, 0, 0, 0, 5, 0, -1))
- 7200.0
+ 10800.0
 : Global context

======================================================================
FAIL: test_mktime_dst (tests.modules.test_time.TimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bruno/PycharmProjects/batavia/tests/modules/test_time.py", line 251, in test_mktime_dst
    self.assertCodeExecution(test_str)
  File "/home/bruno/PycharmProjects/batavia/tests/utils.py", line 504, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '>>> [73 chars]0))\n7200.0\ntrying month 2\n>>> time.mktime(([813 chars].0\n' != '>>> [73 chars]0))\n10800.0\ntrying month 2\n>>> time.mktime([814 chars].0\n'
  >>> import time
  trying month 1
  >>> time.mktime((1970, 1, 1, 0, 0, 0, 0, 0, 0))
- 7200.0
+ 10800.0
  trying month 2
  >>> time.mktime((1970, 2, 1, 0, 0, 0, 0, 0, 0))
- 2685600.0
?    ^^
+ 2689200.0
?    ^^
  trying month 3
  >>> time.mktime((1970, 3, 1, 0, 0, 0, 0, 0, 0))
  5108400.0
  trying month 4
  >>> time.mktime((1970, 4, 1, 0, 0, 0, 0, 0, 1))
- 7786800.0
?    ^^
+ 7783200.0
?    ^^
  trying month 5
  >>> time.mktime((1970, 5, 1, 0, 0, 0, 0, 0, 1))
- 10378800.0
?     ^^
+ 10375200.0
?     ^^
  trying month 6
  >>> time.mktime((1970, 6, 1, 0, 0, 0, 0, 0, 1))
- 13057200.0
?     ^^
+ 13053600.0
?     ^^
  trying month 7
  >>> time.mktime((1970, 7, 1, 0, 0, 0, 0, 0, 1))
- 15649200.0
?     ^^
+ 15645600.0
?     ^^
  trying month 8
  >>> time.mktime((1970, 8, 1, 0, 0, 0, 0, 0, 1))
- 18327600.0
?     ^^
+ 18324000.0
?     ^^
  trying month 9
  >>> time.mktime((1970, 9, 1, 0, 0, 0, 0, 0, 1))
- 21006000.0
?     ^^
+ 21002400.0
?     ^^
  trying month 10
  >>> time.mktime((1970, 10, 1, 0, 0, 0, 0, 0, 1))
- 23598000.0
?     ^^
+ 23594400.0
?     ^^
  trying month 11
  >>> time.mktime((1970, 11, 1, 0, 0, 0, 0, 0, 1))
  26272800.0
  trying month 12
  >>> time.mktime((1970, 12, 1, 0, 0, 0, 0, 0, 0))
- 28864800.0
?      -
+ 28868400.0
?     +
 : Global context

======================================================================
FAIL: test_mktime_too_late (tests.modules.test_time.TimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bruno/PycharmProjects/batavia/tests/modules/test_time.py", line 397, in test_mktime_too_late
    """.format(hour=hour, second=second))
  File "/home/bruno/PycharmProjects/batavia/tests/utils.py", line 606, in assertJavaScriptExecution
    self.assertNotEqual(js_out, py_out, 'Global context')
AssertionError: '>>> import time\n>>> time.mktime((275760, 9, 12, 22, 0, 0, 0, 0, 1))\n### EXCEPTION ###\nOverflowError: signed integer is greater than maximum\n    test.py:4\n' == '>>> import time\n>>> time.mktime((275760, 9, 12, 22, 0, 0, 0, 0, 1))\n### EXCEPTION ###\nOverflowError: signed integer is greater than maximum\n    test.py:4\n' : Global context

----------------------------------------------------------------------
Ran 28 tests in 3.097s

FAILED (failures=4, expected failures=2)

Confused END_FINALLY: TypeError: undefined is not a valid argument for 'instanceof'

Working on String.__ge__ I encountered this error when running tests. Would it be appropriate to take a crack at this? Any words of wisdom/hints?

Traceback (most recent call last):

  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 58, in testPartExecutor
    yield

  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 577, in run
    testMethod()

  File "/Users/stoebelj/Dropbox/batavia/tests/utils.py", line 846, in func
    substitutions=SAMPLE_SUBSTITUTIONS

  File "/Users/stoebelj/Dropbox/batavia/tests/utils.py", line 893, in assertBinaryOperation
    substitutions=substitutions

  File "/Users/stoebelj/Dropbox/batavia/tests/utils.py", line 496, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)

  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 797, in assertEqual
    assertion_func(first, second, msg=msg)

  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 1170, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))

  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 642, in fail
    raise self.failureException(msg)

AssertionError: '>>> [50 chars]= y\nConfused END_FINALLY: TypeError: undefine[79 chars]')\n' != '>>> [50 chars]= y\n<class \'TypeError\'> : unorderable types[739 chars]\n\n'
  >>> x = ""
  >>> y = bytearray(b"hello world")
  >>> x >= y
- Confused END_FINALLY: TypeError: undefined is not a valid argument for 'instanceof' (evaluating 'obj instanceof type')
+ <class 'TypeError'> : unorderable types: str() >= bytearray()
+ 
+ >>> x = "3"
+ >>> y = bytearray(b"hello world")
+ >>> x >= y
+ <class 'TypeError'> : unorderable types: str() >= bytearray()
+ 
+ >>> x = "This is another string"
+ >>> y = bytearray(b"hello world")
+ >>> x >= y
+ <class 'TypeError'> : unorderable types: str() >= bytearray()
+ 
+ >>> x = "Mÿ hôvèrçràft îß fûłl öf éêlś"
+ >>> y = bytearray(b"hello world")
+ >>> x >= y
+ <class 'TypeError'> : unorderable types: str() >= bytearray()
+ 
+ >>> x = "One arg: %s"
+ >>> y = bytearray(b"hello world")
+ >>> x >= y
+ <class 'TypeError'> : unorderable types: str() >= bytearray()
+ 
+ >>> x = "Three args: %s | %s | %s"
+ >>> y = bytearray(b"hello world")
+ >>> x >= y
+ <class 'TypeError'> : unorderable types: str() >= bytearray()
+ 
 : Global context: Error running x >= y

unexpected results for list // complex

Implementing list // complex() Python throws TypeError: unsupported operand type(s) for //: 'list' and 'complex' but expectation from the unit test is <class 'TypeError'> : can't take floor of complex number.

Is this a bug or am I misunderstanding something? I'd love to work on this if its not to...complex rim shot

implementing time module

many of the functions for time will, I believe, be covered with Ouroboros, but some need to be implemented here. I'd like to work on them.

These four were suggested to me by @swenson, but are there others?

struct_time
mktime
localtime
gmtime

Dictionary, set, and frozenset implementations have duplicate key problems

Since JavaScript is pretty lax about equality, we get unexpected behavior when mixing types in dictionaries and sets:

d = {}
d['1'] = 123
d[1] = 456
print(d)

gives {'1': 456} for example, and

print(set([1, '1']))

gives {'1'}.

I think we may have to roll our own hash table implementation to fix this.

Running tests regularly takes > 10 minutes

I think we might be at the point where profiling is needed to see where the testing bottleneck is. Normally I would be ok leaving performance optimizations until the project was more complete, but test builds regularly time out on Travis. This slows down the contributor workflow, and generally makes the project move slower.

I have never done serious python profiling, open to suggestions and/or someone else taking this on.

Switch to Nose for testrunner

This has some advantages:

Auto-parallelization in CircleCI
Switch to jUnit XML output so that CircleCI will have better test result messages
Easier to collect names of tests, and other automation

Callables do not enforce required parameters

For example:

def foo(a):
    print(a)
foo()

results in (traceback from Chrome):

Traceback (most recent call last):
  File "/var/folders/jy/_hlv4sxd78nd0dlxq4p4zmxr0000gn/T/tmpbu2tr58w", line 3, in <module>
  File "/var/folders/jy/_hlv4sxd78nd0dlxq4p4zmxr0000gn/T/tmpbu2tr58w", line 2, in foo
NameError: local variable 'a' referenced before assignment

whereas correct Py behavior is:

>>> def foo(a):
...     print(a)
...
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() missing 1 required positional argument: 'a'

Issue with local on sample.py

I tried to run the sample.py via the testserver harness, and got some issues, which I think I resolved in #7 .

However, now I'm seeing the python error NameError: local variable '__locals__' referenced before assignmentnull in the testserver output.

It appears to be coming from the Point class, because if I remove the Point class code from sample.py code, it runs ok.

implementing String.mod

The modulus operator for Str seems straight forward enough. for example:

>>> 'spam' % b'123'
'spam'

However, someone very smart wrote the tests here:

>>> x = "One arg: %s"
>>> y = slice(5)
>>> x % y
One arg: slice(None, 5, None)

The mod operator is misunderstood as string formatting. Any guidance for how to implement this? I could use a regex to check if the left operand has a string formatting marker in it, but there might be a smarter method. I welcome input.

Float issues

Probably the same as #12. Detailed debugging report, from djangocon eu sprints:

Python 3.4.3+, Ubuntu 15.10

Running the testserver with "abs(3)" returns 3

Running the testserver with "abs(3333)" returns 3333

Running the testserver with "abs(3.333)" returns 0

Diving into VirtualMachine.js, batavia.VirtualMachine.prototype.run_code has a code.co_const that is showing badvalues instead of useful content.

Sending: abs(1); abs(2); abs(3); abs(4)

// batavia.VirtualMachine.prototype.run_code - code.co_consts
1,2,3,4,

Sending two calls: abs(3); abs(3.33333333)

// batavia.VirtualMachine.prototype.run_code - code.co_consts
3,2.590327e-318,

Investigations continuing, but I think that this is the unprocessed bytecode, as a breakpoint in abs is hit after this earlier marshelling.

Possibly related to 64 vs 32 bit things, according to @freakboy3742, but I would have to dive into bytecode things to work out if this is part of the problem.

Implement a function to check valid types for a type.

A valid function in utils would receive a type name parameter and return true if the type is compatible with the name.

Example:

other is a batavia.types.Bool: valid('Bool', other) -> true
other is a batavia.types.Int: valid('Bool', other) -> true
other is a batavia.types.Float: valid('Bool', other) -> true
other is a batavia.types.Dict: valid('Bool', other) -> false

It could be used in places like this one:
https://github.com/pybee/batavia/blob/master/batavia/types/Bool.js#L58

And refactor other parts:
https://github.com/pybee/batavia/blob/master/batavia/types/Bool.js#L124

If and floating point

if 2.1 > 3.123:
    print("a")
else:
    print("b")

--> b

but

if 2 > 3.123:
    print("a")
else:
    print("b")

--> a

Seems like an error in the comparison.

Longs not supported

Entering a number that is larger than a double can support seems to cause a type error.

e.g., 2**64 gives:

TypeError: undefined is not a constructor (evaluating 'batavia.modules.marshal.r_PyLong(vm,p)')

Rough TODOs:

Marshal PyLongs
Auto-promote arithmetic (2**1024)
Auto-demote float arithmetic (2**1024/3)
Make sure all arithmetic operations and builtins are ported over correctly

Compile Python into bytecode in batavia directly

In order to be self-hosted (and run without CPython also being present), we need to be able to compile Python into bytecode ourselves.

There are four big pieces to making this happen:

Lexical analyzer (convert source to series of simple tokens).
Write code generator for grammar to autogenerate a lot of JS for us.
Implement the parser to convert tokens into an AST (with the help of the grammar).
Convert the AST into Python byte code (generate).

In CPython, all of this is done in C.

Running Batavia in Vagrant Box - Broken Pipe

Hello!

When running the test suite for batavia, python setup.py test, I get 836 failures in addition to the 3262 expected failures. It appears that all of the failures are "Broken Pipe" errors.

My environment:

Windows 7 laptop, running a Vagrant box
Vagrant version: 1.8.1
Virtualbox Version: 5.0.2
Linux Version: Ubuntu Trusty 14.04
Python Version: 3.4.3 (in a virtualenv)

Error example:

======================================================================
FAIL: test_js_code (tests.test_utils.JavaScriptBootstrapTests)
You can supply JavaScript code and use it from within Python
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant/trusty_backup/batavia/tutorial/batavia-dev/batavia/tests/utils.py", line 529, in assertJavaScriptExecution
    args=args
  File "/vagrant/trusty_backup/batavia/tutorial/batavia-dev/batavia/tests/utils.py", line 275, in runAsJavaScript
    on_fail="Unable to create webpage."
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/vagrant/trusty_backup/batavia/tutorial/batavia-dev/batavia/tests/test_utils.py", line 234, in test_js_code
    """,
  File "/vagrant/trusty_backup/batavia/tutorial/batavia-dev/batavia/tests/utils.py", line 532, in assertJavaScriptExecution
    self.fail(e)
AssertionError: [Errno 32] Broken pipe

Thank you for your help!

Complete implementations of Python builtins

batavia/core/builtins.js is an implementation of all the Python builtins - methods like print(), input(), list(), abs(), min() and so on.

An implementation of these builtins is required.