hippyvm / hippyvm Goto Github PK

View Code? Open in Web Editor NEW

879.0 50.0 55.0 8.36 MB

HippyVM - an implementation of the PHP language in RPython

Home Page: http://hippyvm.baroquesoftware.com/

License: MIT License

PHP 72.24% Python 12.03% HTML 0.01% JavaScript 2.18% Shell 0.07% Makefile 0.02% C 12.51% C++ 0.94% Assembly 0.01%

hippyvm's Introduction

HippyVM

HippyVM is an implementation of the PHP language using RPython/PyPy technology.

HippyVM right now works only on 64bit linux on x86 platform (this limitation is temporary though, the RPython toolchain supports 32 and 64 bit x86, ARMv6 and ARMv7 on windows, os x and linux).

Building

The build process was tested for Ubuntu 14.04. Please create an issue/submit pull request if things are not working as expected.

1. Clone this repo ;)

git clone https://github.com/hippyvm/hippyvm

2. Get a full source checkout of RPython:

There are two alternative ways to achieve this, both equally functional:

a snapshot of PyPy is the easiest way to get a large repository.

wget https://bitbucket.org/pypy/pypy/get/default.tar.gz mkdir pypy tar xfv default.tar.gz -C pypy --strip-components 1
a full checkout of the RPython repository translation toolchain (currently inside the PyPy repository). It will download the whole commit history of PyPy.

hg clone http://bitbucket.org/pypy/pypy

3. Get the build dependencies:

pip install -r requirements.txt
sudo apt-get install libmysqlclient-dev libpcre3-dev librhash-dev libbz2-dev php5-cli libffi-dev

4. The building process goes like this:

cd hippyvm
<path to pypy>/rpython/bin/rpython -Ojit targethippy.py

This will create a hippy-c binary that works mostly like a php-cli without readline support.

Running it

You can run it with ./hippy-c <file.php>. Example of benchmarks are in bench/ sub-directory.

Contribution

Like many open-source projects, HippyVM is looking for contributors.

In contrast with most language implementations that use C or C++, HippyVM has a low barrier of entry since it uses RPython, a subset of the Python language. It's really easy to write and read. Check out our implementation of strstr

@wrap(['space', str, W_Root, Optional(bool)], aliases=['strchr'])
def strstr(space, haystack, w_needle, before_needle=False):
    """Find the first occurrence of a string."""
    try:
        needle = unwrap_needle(space, w_needle)
    except ValidationError as exc:
        space.ec.warn("strstr(): " + exc.msg)
        return space.w_False
    if len(needle) == 0:
        space.ec.warn("strstr(): Empty delimiter")
        return space.w_False
    pos = haystack.find(needle)
    if pos < 0:
        return space.w_False
    if before_needle:
        return space.newstr(haystack[:pos])
    else:
        return space.newstr(haystack[pos:])

Doesn't look that scary, right?

The reasons why HippyVM uses RPython go beyond this README. If you are interested, you can read more here

HippyVM's tests

If the project is up and running, which means the building section from this README went well, you can try to run HippyVM's tests.

PYTHONPATH=$PYTHONPATH:/path-to-pypy py.test testing

This will execute all the tests that were explicitly written for HippyVM, these tests are written in Python as well. The example test for the strstr is here.

PHP's tests

After having HippyVM tests up and running you can try running PHP's tests against the HippyVM implementation.

PYTHONPATH=$PYTHONPATH:/path-to-pypy py.test test_phpt/

Those tests are exact copies from the reference PHP implementation

What now?

If you find something missing, broken, or poorly implemented:

please create an issue, or better,
create a pull request and update the AUTHORS file.

Also please visit us on the irc channel #[email protected]

hippyvm's People

Contributors

Stargazers

Watchers

hippyvm's Issues

CI/Travis builds

While the project is interesting, it would be very useful to see builds for it on either travis-ci or a publicly exposed CI environment.

Without builds, we can't really know how much can be expected to work with this VM

Compilation failed: ffi.h not found

I guess the readme should include libffi-dev in the thing you should install with apt-get.

`getmygid` has the implementation of `getmypid`

The function getmygid is actually returning what getmypid should return. The latter is currently not implemented.

Missing PHP functions needed to run DokuWiki

Tried to run DokuWiki with hippy. Installation says the following PHP functions are missing:

ignore_user_abort
parse_ini_file
usleep
stream_select
pack

Hex2bin is 5-7 times slower than PHP's implementation

With the following script one can see that hex2bin in Hippy is much slower than in PHP.

<?php

$data = str_repeat("a", $argv[1]);

for($i=0;$i<10000;++$i){
    hex2bin($data);
}

In PHP:

time php speed_hex2bin.php 1000

real    0m0.195s
user    0m0.128s
sys 0m0.028s

time php speed_hex2bin.php 10000

real    0m0.916s
user    0m0.696s
sys 0m0.024s

And in Hippy

time ~/hippy/hippyvm/hippy-c speed_hex2bin.php 1000

real    0m1.007s
user    0m0.704s
sys 0m0.032s

time ~/hippy/hippyvm/hippy-c speed_hex2bin.php 10000

real    0m6.684s
user    0m6.548s
sys 0m0.052s

Nowdoc is not supported

Nowdocs ('single quoted' heredocs) are not supported.

Simple test case:

<?php
$str = <<<'EOD'
Example of string
spanning multiple lines
using nowdoc syntax.
EOD;
echo $str;

Expected output:

Example of string
spanning multiple lines
using nowdoc syntax

Hippy's output (note that there is no reference to the error):

Error opening /tmp/error.php

Hi, Windows Support?

Hi, any chance I can try this in Windows? Is there any Docs on that?

Dead readme link

Why RPython link died.

hex2bin does not return false with invalid input

With invalid input Hippy crashes instead of returning false.

get_html_translation_table not implemented

Needed to run squirrelmail under hippy.

http://php.net/manual/en/function.get-html-translation-table.php

Pool configuration (different user:group)

This feature is important for shared servers which handles more than one domain. It should be possible to define an own config file with a custom user:group (also different php settings) for each domain.

Besides is it already possible to run hippyvm as fastcgi and also define php.ini files?

PS: A wiki wouldn't be bad :)

Thanks,
Ninos

Missing $_SERVER variables

When running hippy from command line (not tested on server) many $_SERVER variables are missing.

Below are the missing variables when compared to PHP 5.4.4

[PHP_SELF] => tests/server.php
[SCRIPT_NAME] => tests/server.php
[SCRIPT_FILENAME] => tests/server.php
[PATH_TRANSLATED] => tests/server.php
[DOCUMENT_ROOT] => 
[REQUEST_TIME_FLOAT] => 1406537711.5441
[REQUEST_TIME] => 1406537711
[argv] => Array
    (
        [0] => tests/server.php
    )
[argc] => 1

Many projects, including PhpUnit and samples from "The Computer Language Benchmarks Game", rely at least on $_SERVER['argv'].

Random number generator always has the same seed

It seems that the seed of the PRNG is always the same.

This is a potential security vulnerability.

Run .phpt tests on Travis

We should add a build that runs py.test test_phpt/.

dirname() and FILE broken.

Consider the following scenario:

$ pwd
/tmp
$ find a
a
a/z.php
$ cat a/z.php 
<?php

$p = dirname(__FILE__);
echo "p = '$p'\n";

?>

Now let's run this from /tmp in both php and hippyvm (e.g. php a/z.php):

PHP gives:

p = '/tmp/a'

Hippy gives:

p = '.'

dirname() is supposed to give an absolute path.
The wrong path is reported regardless of absolute/relative paths.

HHVM deltablue benchmark crashes hippy

Hi,

We have taken this deltablue benchmark:
https://github.com/facebook/hhvm/blob/master/hphp/benchmarks/php-octane/deltablue.php

and hacked it, removing the dependency upon facebook's benchmark runner, and adding support for our own. Our runner would usually import this benchmark and call run_iter() repeatedly. We removed some extraneous comments also.

The result is here: https://gist.github.com/vext01/3cc74eac5d416abf05d4

For the purpose of this bug, i have manually called run_iter(2000) so that the file is standalone.

This works under Zend PHP:

$ time php mono.php 

real    0m1.021s
user    0m1.012s
sys     0m0.004s

But under hippy:

vext01@bencher3:~/research/pyhyp_experiments/benchmarks/deltablue$ ~/research/hippyvm.hippyvm/hippy-c  mono.php             
In function <main>, file mono.php, line 219
  class UnaryConstraint extends Constraint {
Strict Standards: Declaration of UnaryConstraint::__construct() should be compatible with Constraint::__construct($strength)
 in mono.php on line 219
In function <main>, file mono.php, line 317
  class BinaryConstraint extends Constraint {
Strict Standards: Declaration of BinaryConstraint::__construct() should be compatible with Constraint::__construct($strength
) in mono.php on line 317
In function <main>, file mono.php, line 414
  class ScaleConstraint extends BinaryConstraint {
Strict Standards: Declaration of ScaleConstraint::__construct() should be compatible with BinaryConstraint::__construct($var
1, $var2, $strength) in mono.php on line 414
RPython traceback:
  File "hippy_main.c", line 458, in entry_point
  File "hippy_main.c", line 4635, in main
  File "hippy_interpreter.c", line 7081, in Interpreter_run_main
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret                                                 [42/1809]
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_klass.c", line 3934, in ClassBase_call_args
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 35375, in portal_1
  File "hippy_interpreter_1.c", line 27624, in Interpreter_CALL
  File "hippy_interpreter.c", line 13681, in Interpreter_interpret
  File "rpython_jit_metainterp_warmspot.c", line 1149, in ll_portal_runner__Signed_hippy_bytecode_ByteCode
  File "hippy_interpreter.c", line 30465, in portal_1
  File "hippy_interpreter.c", line 61654, in Interpreter_BINARY_NE
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare                                    [0/1809]
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  File "hippy_objspace.c", line 13481, in _compare_array
  File "hippy_objects_instanceobject.c", line 20941, in W_InstanceObject_compare
  ...
Fatal RPython error: StackOverflow
Aborted

Perhaps Hippy is stuck in infinite recursion?

Site is down

http://hippyvm.com doesn't resolve for me :(

Missing spl_autoload_register support

Fatal error: Call to undefined function spl_autoload_register() in /usr/local/bin/phpunit.phar on line 13

This is the most used function :/
Cannot make any test without this.

bin/hip.py broken

wilfred:hippyvm> ./bin/hip.py
...
-=- Hippy -=-

<? $a = 1;
TypeError: parse() takes exactly 4 arguments (3 given)
<? lfkjsdlkfjsd
TypeError: parse() takes exactly 4 arguments (3 given)
<?

compiling on windows using mingw

Hey!

I'm trying to compile HippyVM on Windows using "i686-w64-mingw32-gcc".

At first, i tried to use Python3.4, but this won't work with RPython.

File "../pypy-src/rpython/bin/rpython", line 17
print __doc__

So i switched to Python2.7.

Now i'm running into CompilationErrors:
1. cannot import name nl_langinfo

Traceback (most recent call last):
  File "../pypy-src/rpython/bin/rpython", line 20, in <module>
    main()
  File "D:\pypy\pypy-src\rpython\translator\goal\translate.py", line 217, in main
    targetspec_dic, translateconfig, config, args = parse_options_and_load_target()
  File "D:\pypy\pypy-src\rpython\translator\goal\translate.py", line 156, in parse_options_and_load_target
    targetspec_dic = load_target(targetspec)
  File "D:\pypy\pypy-src\rpython\translator\goal\translate.py", line 98, in load_target
    mod = __import__(specname)
  File "targethippy.py", line 2, in <module>
    from hippy.main import entry_point
  File "hippy\main.py", line 17, in <module>
    from hippy.interpreter import Interpreter
  File "hippy\interpreter.py", line 32, in <module>
    from hippy.module.spl import spl
  File "hippy\module\spl\spl.py", line 10, in <module>
    from hippy.module.standard.file.funcs import (_is_dir, _is_file, _is_link,
  File "hippy\module\standard\file\funcs.py", line 10, in <module>
    from hippy.sort import _sort
  File "hippy\sort.py", line 4, in <module>
    from hippy.module.standard.strings.funcs import _strnatcmp
  File "hippy\module\standard\strings\funcs.py", line 27, in <module>
    from hippy import locale
  File "hippy\locale.py", line 4, in <module>
    from rpython.rlib.rlocale import (setlocale as rsetlocale, LocaleError,
ImportError: cannot import name nl_langinfo

https://github.com/hippyvm/hippyvm/blob/master/hippy/locale.py#L5
I managed to bypass that error by removing the import.

2. 'platcheck_t' has no member named 'tm_zone'

Traceback (most recent call last):
  File "../pypy-src/rpython/bin/rpython", line 20, in <module>
    main()
  File "D:\pypy\pypy-src\rpython\translator\goal\translate.py", line 217, in main
    targetspec_dic, translateconfig, config, args = parse_options_and_load_target()
  File "D:\pypy\pypy-src\rpython\translator\goal\translate.py", line 156, in parse_options_and_load_target
    targetspec_dic = load_target(targetspec)
  File "D:\pypy\pypy-src\rpython\translator\goal\translate.py", line 98, in load_target
    mod = __import__(specname)
  File "targethippy.py", line 2, in <module>
    from hippy.main import entry_point
  File "hippy\main.py", line 17, in <module>
    from hippy.interpreter import Interpreter
  File "hippy\interpreter.py", line 48, in <module>
    import hippy.module.standard.misc.funcs
  File "hippy\module\standard\misc\funcs.py", line 7, in <module>
    from hippy.module.date import timelib
  File "hippy\module\date\__init__.py", line 3, in <module>
    from hippy.module.date import timelib
  File "hippy\module\date\timelib.py", line 67, in <module>
    conf = platform.configure(CConfig)
  File "D:\pypy\pypy-src\rpython\rtyper\tool\rffi_platform.py", line 224, in configure
    ignore_errors=ignore_errors))
  File "D:\pypy\pypy-src\rpython\rtyper\tool\rffi_platform.py", line 731, in run_example_code
    output = build_executable_cache(files, eci, ignore_errors=ignore_errors)
  File "D:\pypy\pypy-src\rpython\tool\gcc_cache.py", line 28, in build_executable_cache
    result = platform.execute(platform.compile(c_files, eci))
  File "D:\pypy\pypy-src\rpython\translator\platform\__init__.py", line 53, in compile
    ofiles = self._compile_o_files(cfiles, eci, standalone)
  File "D:\pypy\pypy-src\rpython\translator\platform\__init__.py", line 75, in _compile_o_files
    ofiles.append(self._compile_c_file(self.cc, cfile, compile_args))
  File "D:\pypy\pypy-src\rpython\translator\platform\posix.py", line 37, in _compile_c_file
    cwd=str(cfile.dirpath()))
  File "D:\pypy\pypy-src\rpython\translator\platform\__init__.py", line 139, in _execute_c_compiler
    self._handle_error(returncode, stdout, stderr, outname)
  File "D:\pypy\pypy-src\rpython\translator\platform\windows.py", line 495, in _handle_error
    returncode, '', stderr + stdout, outname)
  File "D:\pypy\pypy-src\rpython\translator\platform\__init__.py", line 151, in _handle_error
    raise CompilationError(stdout, stderr)
rpython.translator.platform.CompilationError: CompilationError(err="""
    In file included from d:\qt\qt5.2.0\tools\mingw48_32\lib\gcc\i686-w64-mingw32\4.8.0\include\stddef.h:1:0,
                     from d:\qt\qt5.2.0\tools\mingw48_32\i686-w64-mingw32\include\inttypes.h:14,
                     from D:\pypy\hippyvm\hippy\module\date\lib/timelib_structs.h:26,
                     from D:\pypy\hippyvm\hippy\module\date\lib/timelib.h:24,
                     from c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:31:
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c: In function 'dump_section_tm':
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:109:27: error: 'platcheck_t' has no member named 'tm_gmtoff'
      dump("fldofs tm_gmtoff", offsetof(platcheck_t, tm_gmtoff));
                               ^
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:110:38: error: 'platcheck_t' has no member named 'tm_gmtoff'
      dump("fldsize tm_gmtoff",   sizeof(s.tm_gmtoff));
                                          ^
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:111:3: error: 'platcheck_t' has no member named 'tm_gmtoff'
      s.tm_gmtoff = 0; s.tm_gmtoff = ~s.tm_gmtoff;
       ^
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:111:20: error: 'platcheck_t' has no member named 'tm_gmtoff'
      s.tm_gmtoff = 0; s.tm_gmtoff = ~s.tm_gmtoff;
                        ^
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:111:35: error: 'platcheck_t' has no member named 'tm_gmtoff'
      s.tm_gmtoff = 0; s.tm_gmtoff = ~s.tm_gmtoff;
                                       ^
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:112:33: error: 'platcheck_t' has no member named 'tm_gmtoff'
      dump("fldunsigned tm_gmtoff", s.tm_gmtoff > 0);
                                     ^
    In file included from d:\qt\qt5.2.0\tools\mingw48_32\lib\gcc\i686-w64-mingw32\4.8.0\include\stddef.h:1:0,
                     from d:\qt\qt5.2.0\tools\mingw48_32\i686-w64-mingw32\include\inttypes.h:14,
                     from D:\pypy\hippyvm\hippy\module\date\lib/timelib_structs.h:26,
                     from D:\pypy\hippyvm\hippy\module\date\lib/timelib.h:24,
                     from c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:31:
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:113:25: error: 'platcheck_t' has no member named 'tm_zone'
      dump("fldofs tm_zone", offsetof(platcheck_t, tm_zone));
                             ^
    c:\users\koch\appdata\local\temp\usession-default-34\platcheck_8.c:114:36: error: 'platcheck_t' has no member named 'tm_zone'
      dump("fldsize tm_zone",   sizeof(s.tm_zone));
                                        ^
    """)

I believe the tm_zone attribute is available only on Python3.

Regards, Jens

Performance question

The website states "HippyVM on average is 7.3x faster than stock PHP and 2x faster than Facebook's HHVM. Please check our benchmarks."

While the latest blog post says "In some areas we are even a bit ahead of both HHVM and PHP. Still there's lots of room for improvements - and we are on it!"

I think you should clarify this point and specify what version of both HHVM and PHP you compare to (I would also be worth comparing to PHP.ng)

Note: this is not a proper "issue" but there is no comment I can add to your blog post

Hippy does not distinguish a parse error from being unable to open the file.

Hippy reports that the file cannot be opened if:

A parse error occurs.
The file does not exist.

In the former case, we should report that a parse error occurred, the line number, and (possibly?) the expected token.

Examples:

$ ./hippy-c /some/nonexisting/file.php
Error opening /some/nonexisting/file.php

$ echo "<?php ;;;parse_errors_all_round$$$<<< ?>" > ~/file.php
$ ./hippy-c ~/file.php 
Error opening /home/vext01/file.php

Another example is shown in #46.

New version of rply now depends on appdir

In the pypy-bridge branch, I think requirements.txt needs to be updated to also include https://pypi.python.org/pypi/appdirs/1.4.0#downloads, which rply now seems to need.

Heredocs fails with $ enclosed with double quotes

Having a "$" in a heredoc string makes the parser to choke.

Below is a simple script that won't be parsed:

<?php
$a = <<<EOD
"$"
EOD;

print $a;

Hippy's output is just (note that there is no reference to the source of the problem):

Error opening /tmp/error.php

Whereas PHP's is:

"$"

Missing composer support

https://getcomposer.org/

Missing PHPUnit support

http://phpunit.de/index.html

make fastcgi interface, not webserver

Just a suggestion and encouragement for the project:

Maybe don't try to make a webserver like the early versions of HHVM, just go for a fastcgi interface which seems like it would be far easier to accomplish. Don't waste energy reinventing the wheel where nginx/apache, etc. do a feature-rich, high performance job already.

Really looking forward to seeing what can be done with this as you make progress, 2x faster than HHVM would be stunning. Best wishes for great inspirations.

Many functions are just mocks

In funcs.py many functions are just mocks, and no warning is raised to let the user know about it.

Some of these functions are: set_time_limit, memory_get_usage, memory_get_peak_usage, getrusage.

Seg fault when calling mysql_connect() with no args.

vext01@bencher2:~/research/hippyvm$ cat mysql.php 
<?php

mysql_connect();

?>
vext01@bencher2:~/research/hippyvm$ ./hippy-c mysql.php 
Segmentation fault

Running untranslated gives a clue:

vext01@bencher2:~/research/hippyvm$ python2.7 hippy/main.py mysql.php
...
Traceback (most recent call last):
  File "hippy/main.py", line 192, in <module>
    sys.exit(entry_point(sys.argv))
  File "hippy/main.py", line 106, in entry_point
    bench_mode, bench_no)
  File "hippy/main.py", line 154, in main
    interp.run_main(space, bc, top_main=True)
  File "/home/vext01/research/hippyvm/hippy/interpreter.py", line 772, in run_main
    w_result = self.interpret(frame)
  File "/home/vext01/research/hippyvm/hippy/interpreter.py", line 837, in interpret
    return self._interpret(frame, frame.bytecode)
  File "/home/vext01/research/hippyvm/hippy/interpreter.py", line 892, in _interpret
    pc = bc_impl(bytecode, frame, space, arg, pc)
  File "/home/vext01/research/hippyvm/hippy/interpreter.py", line 1424, in CALL
    w_res = self.call(func, args_w)
  File "/home/vext01/research/hippyvm/hippy/interpreter.py", line 1414, in call
    return callable.call_args(self, args_w)
  File "/home/vext01/research/hippyvm/hippy/builtin.py", line 859, in call_args
    return self.runner(interp, args_w, w_this, thisclass)
  File "/home/vext01/research/hippyvm/hippy/builtin.py", line 899, in call_args
    res = ll_func(*ll_args)
  File "/home/vext01/research/hippyvm/ext_module/mysql/funcs.py", line 143, in mysql_connect
    return _mysql_connect(interp, host, username, password)
  File "/home/vext01/research/hippyvm/ext_module/mysql/funcs.py", line 104, in _mysql_connect
    'mysql.default_user'))
  File "/home/vext01/research/hippyvm/hippy/objspace.py", line 185, in str_w
    res = w_v.deref().str(self, quiet=quiet)
AttributeError: 'NoneType' object has no attribute 'deref'

require_once() semantics incorrect

Consider the following files:

src/login.php:

<?php
require_once("../functions/global.php");
foo();
?>

functions/global.php:

<?php
require_once("../functions/string.php");
function check_php_version($a, $b){
    echo $a;
}
?>

functions/strings.php:

<?php
require_once("../functions/global.php");
function foo(){
    echo "foo";
}
?>

Running login.php under hippy returns the following error:

In function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/src/login.php, line 2
  require_once("../functions/global.php");
In function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php, line 2
  require_once("../functions/string.php");
In function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/functions/string.php, line 2
  require_once("../functions/global.php");
In function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php, line 3
  function check_php_version($a, $b){
Fatal error: Cannot redeclare check_php_version() (previously declared in /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php:3) in /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php on line 3
function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/src/login.php:2
require_once("../functions/global.php");
function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php:2
require_once("../functions/string.php");
function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/functions/string.php:2
require_once("../functions/global.php");
function <main>, file /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php:3
function check_php_version($a, $b){
Fatal interpreter error Cannot redeclare check_php_version() (previously declared in /home/lukas/research/hippybridge/hippyvm/temp/functions/global.php:3)

Running under PHP returns the expected result:

foo

Interestingly, if I use just the basename of the require_once call (e.g. change "../functions/strings.php" to "strings.php" and "../functions/globals.php" to "globals.php") the output under hippy is consistent with PHP and prints "foo".

PHP extensions?

I know its on everybody's mind so I'll go ahead and ask it - what are your thoughts on PHP's extensions - do you expect to implement them soon, or are you expecting other people to contribute them?

Needless to say any non trivial use of PHP requires one extension or another (pdo, mysqli, zlib, mcrypt ...), also HipHopVM are close to implementing a lot of them.

That said this is simply incredible effort from you guys - I really really hope this can take off - having so readable php code and possibility for interop with different languages (written in RPython themselves) is simply mind blowing :) Best of 🍀

HippyVM cannot read from stdin.

Hippy should be able to read a program from stdin, like PHP:

$ echo '<?php echo "hi php\n"; ?>' | php
hi php
$ echo '<?php echo "hi php\n"; ?>' | ~/research/hippyvm.hippyvm/hippy-c
 Hippy VM. Execute by typing

hippy [--gcdump dumpfile] [--cgi] [--server port] <file.php> [php program options]

and enjoy

$

I think this is a requirement for running hippy under Apache/SimpleCGI.

testing/test_cgi.py is unsafe

The test_session test in testing/test_cgi.py creates a file with a fixed name (/tmp/xyz-abcd) which it then reads in. This isn't great for security and also prevents 2 people running tests (the file isn't cleaned up, so user #2 gets a weird error, which ultimately means that they don't have permissions to write/read the file).

This should really use some sort of unique filename, but I don't know the API well enough to see how to do this.

Wrong(?) output for simple CGI

My lighttpd server doesn't accept the normal cgi output from hippy. I'm not completely sure about the CGI standards but it seems that there is a newline at the beginning of the output that shouldn't be there.

The responsible method is:

    def send_headers(self):
        self.any_output = True
        if self.cgi:
            if self.cgi != constants.CGI_FASTCGI:
                self._writestr("\r\n")
            if self.http_status_code != -1:
                self._writestr('Status: %d\r\n' % self.http_status_code)
            for k in self.headers:
                self._writestr(k + "\r\n")
            for elem in self.extra_headers:
                self._writestr(elem + "\r\n")
            self._writestr("\r\n")
        self.headers = None
        self.extra_headers = None

In this function a leading newline is appended to the headers if the mode is anything but "fastcgi", but normal cgi should't have this newline either.

A fix would be:

if self.cgi not in [constants.CGI_FASTCGI, constants.CGI_SIMPLE]:

Error running rpython -Ojit targethippy.py

Hi,

i followed the steps from documentions 2 times on OSX and Ubuntu Precise, in both cases the output is:

mcuadros@sd-56536:~/hippyvm$ ../pypy-pypy-fe72a15b0882/rpython/bin/rpython -Ojit targethippy.py
[translation:info] 2.7.2 (1.8+dfsg-2, Feb 19 2012, 19:18:08)
[translation:info] [PyPy 1.8.0 with GCC 4.6.2]
[platform:msg] Set platform with 'host' cc=None, using cc='gcc'
[translation:info] Translating target as defined by targethippy
Traceback (most recent call last):
  File "app_main.py", line 51, in run_toplevel
  File "../pypy-pypy-fe72a15b0882/rpython/bin/rpython", line 20, in <module>
    main()
  File "/home/mcuadros/pypy-pypy-fe72a15b0882/rpython/translator/goal/translate.py", line 217, in main
    targetspec_dic, translateconfig, config, args = parse_options_and_load_target()
  File "/home/mcuadros/pypy-pypy-fe72a15b0882/rpython/translator/goal/translate.py", line 156, in parse_options_and_load_target
    targetspec_dic = load_target(targetspec)
  File "/home/mcuadros/pypy-pypy-fe72a15b0882/rpython/translator/goal/translate.py", line 98, in load_target
    mod = __import__(specname)
  File "targethippy.py", line 2, in <module>
    from hippy.main import entry_point
  File "hippy/main.py", line 16, in <module>
    from hippy.phpcompiler import compile_php
  File "hippy/phpcompiler.py", line 1, in <module>
    from hippy.lexer import Token, Lexer
  File "hippy/lexer.py", line 3, in <module>
    from rply.token import BaseBox, Token
ImportError: No module named rply

Need integration documentation

It would be nice if the wiki had the following articles:

Basic Installation
Integrating with Nginx
Integrating with Apache
Hosting an example PHP application
Known issues and limitations

Hash module broken on OpenBSD

Hi,

The hash tests are failing on OpenBSD. I have actually known about this for a while, but I pushed it aside since hash used to be in the closed repo, which I rarely use.

I have not had time to investigate yet, but here is the log from the latest hippy master and pypy default:

============================= test session starts ==============================
platform openbsd5 -- Python 2.7.8 -- py-1.4.20 -- pytest-2.5.2
collected 88 items

testing/test_hash.py ...F.F........................................F.F.......................................

=================================== FAILURES ===================================
______________________________ TestHash.test_sha1 ______________________________

self = <testing.test_hash.TestHash object at 0x1db451250250>

    def test_sha1(self):
        output = self.run('''
            echo hash("sha1", "php");
            ''')
        space = self.space
>       assert space.str_w(output[0]) == "47425e4490d1548713efea3b8a6f5d778e4b1766"
E       assert '67452301efcd...25476c3d2e1f0' == '47425e4490d15...f5d778e4b1766'
E         - 67452301efcdab8998badcfe10325476c3d2e1f0
E         + 47425e4490d1548713efea3b8a6f5d778e4b1766

testing/test_hash.py:33: AssertionError
------------------------------- Captured stdout --------------------------------
   0  LOAD_NAME 0
   2  GETFUNC
   3  LOAD_NAME 1
   5  ARG_BY_VALUE 0
   7  LOAD_NAME 2
   9  ARG_BY_VALUE 1
  11  CALL 2
  13  ECHO
_____________________________ TestHash.test_sha256 _____________________________

self = <testing.test_hash.TestHash object at 0x1db3a9d6a9d0>

    def test_sha256(self):
        output = self.run('''
            echo hash("sha256", "php");
            ''')
        space = self.space
>       assert space.str_w(output[0]) == "12a5d18ee896e59954bdce0f4acc7212eebe03dae1834ef4ce160ac5afa5c4a8"
E       assert 'aca00f710da9...911d7ceb52804' == '12a5d18ee896e...60ac5afa5c4a8'
E         - aca00f710da98fc6f850641d1f07d869ce7d315ed1c7b3d8f58911d7ceb52804
E         + 12a5d18ee896e59954bdce0f4acc7212eebe03dae1834ef4ce160ac5afa5c4a8

testing/test_hash.py:47: AssertionError
------------------------------- Captured stdout --------------------------------
   0  LOAD_NAME 0
   2  GETFUNC
   3  LOAD_NAME 1
   5  ARG_BY_VALUE 0
   7  LOAD_NAME 2
   9  ARG_BY_VALUE 1
  11  CALL 2
  13  ECHO
____________________________ TestHash.test_raw_sha1 ____________________________

self = <testing.test_hash.TestHash object at 0x1db3f96baf10>

    def test_raw_sha1(self):
        import md5
        output = self.run('''
            echo hash("sha1", "php", 1);
            ''')
        space = self.space
        s = space.str_w(output[0])
>       assert md5.new(s).hexdigest() == "c409c0762e20b61fca8a5661415495b8"
E       assert 'e4102ad9c480...042b8fde8ec37' == 'c409c0762e20b...a5661415495b8'
E         - e4102ad9c4801e959c6042b8fde8ec37
E         + c409c0762e20b61fca8a5661415495b8

testing/test_hash.py:342: AssertionError
------------------------------- Captured stdout --------------------------------
   0  LOAD_NAME 0
   2  GETFUNC
   3  LOAD_NAME 1
   5  ARG_BY_VALUE 0
   7  LOAD_NAME 2
   9  ARG_BY_VALUE 1
  11  LOAD_CONST 0
  13  ARG_BY_VALUE 2
  15  CALL 3
  17  ECHO
___________________________ TestHash.test_raw_sha256 ___________________________

self = <testing.test_hash.TestHash object at 0x1db45cc10450>

    def test_raw_sha256(self):
        import md5
        output = self.run('''
            echo hash("sha256", "php", 1);
            ''')
        space = self.space
        s = space.str_w(output[0])
>       assert md5.new(s).hexdigest() == "1e8fa266b5f865c7af37cdb282ddbe3e"
E       assert 'a5b9017d7b54...c900087ef9944' == '1e8fa266b5f86...7cdb282ddbe3e'
E         - a5b9017d7b549e65fcac900087ef9944
E         + 1e8fa266b5f865c7af37cdb282ddbe3e

testing/test_hash.py:360: AssertionError
------------------------------- Captured stdout --------------------------------
   0  LOAD_NAME 0
   2  GETFUNC
   3  LOAD_NAME 1
   5  ARG_BY_VALUE 0
   7  LOAD_NAME 2
   9  ARG_BY_VALUE 1
  11  LOAD_CONST 0
  13  ARG_BY_VALUE 2
  15  CALL 3
  17  ECHO
===================== 4 failed, 84 passed in 11.83 seconds =====================

Warnings slow down code quite a lot

When the code generates warnings then the overall performance of the script gets hurt a lot.

Such warnings include:

Hippy warning: Creating array from empty value
Warning: date(): It is not safe to rely on the system's timezone settings.

For the date case I'm seeing a slowdown of about 9x, whereas in stock PHP there's a slowdown of only 2x.

For the first case, there's a slowdown of about 28x in Hippy, whereas in PHP there's almost no slowdown.

Below is the sample code:

<?php

function foo($init_arrray) {
    if ($init_arrray) {
        $a = array();
    }
    $a[0] = 'Foo';
    $a[1] = 'Bar';
    return sprintf('%s%s', $a[0], $a[1]);
}

for ($i = 0; $i < 1000000; ++$i) {
    $init_arrray = $argv[1] === 'init';
    foo($init_arrray);
}

And below are the times:

time ../hippyvm/hippy-c test_uninitialized.php no_init 2>/dev/null

real    0m6.215s
user    0m4.860s
sys 0m1.336s

time ../hippyvm/hippy-c test_uninitialized.php init 2>/dev/null

real    0m0.220s
user    0m0.212s
sys 0m0.008s

time php test_uninitialized.php no_init 2>/dev/null

real    0m0.843s
user    0m0.832s
sys 0m0.004s

time php test_uninitialized.php init 2>/dev/null

real    0m0.811s
user    0m0.804s
sys 0m0.004s

Tests fail if closed extension source code is missing.

It occurs to me that when the optional modules are missing testing will fail
like this:

wilfred:hippyvm-git> py.test -vx testing/              
============================= test session starts ==============================
platform openbsd5 -- Python 2.7.7 -- pytest-2.2.4.dev2 -- /usr/local/bin/python2.7
collecting 0 items / 1 errors
==================================== ERRORS ====================================
____________________ ERROR collecting testing/test_array.py ____________________
../pypy/py/_path/local.py:531: in pyimport
>           mod = __import__(modname, None, None, ['__doc__'])
../pypy/_pytest/assertion/rewrite.py:156: in load_module
>           py.builtin.exec_(co, mod.__dict__)
../hippyvm/testing/test_array.py:8: in <module>
>   from testing.test_interpreter import BaseTestInterpreter, hippy_fail
../pypy/_pytest/assertion/rewrite.py:156: in load_module
>           py.builtin.exec_(co, mod.__dict__)
testing/test_interpreter.py:16: in <module>
>   from testing.runner import MockEngine, MockInterpreter, preparse
testing/runner.py:6: in <module>
>   from hippy.interpreter import Interpreter
hippy/interpreter.py:20: in <module>
>   from hippy.objects.closureobject import W_ClosureObject, new_closure
hippy/objects/closureobject.py:55: in <module>
>                name='Closure::__invoke')
hippy/builtin.py:945: in inner
>                            error_handler, check_num_args)
hippy/builtin.py:893: in make_runner
>       parse_args = builder.build(error_handler, check_num_args)
hippy/builtin.py:672: in build
>           from hippy.module.mysql.link_resource import W_MysqlLinkResource
E           ImportError: No module named mysql.link_resource

This is because is_optional_extension_enabled() is unaware of what code is actually available. For tests we are assuming all extensions are enabled.

As far as I know we cannot simply look onto the disk for source code, since some of the is_optional_extension_enabled() stuff could happen at runtime(?).

We could try to catch the ImportError if we are not translated and either NOP or turn off the missing extension? A bit filthy.

We need to find a solution for this, since hippy will be untestable as an opensource project.

CCing @xando since he was working in this area the other day. Maybe this is what you were trying to address? Also CC @rlamy since he reviewed my initial work on optional extensions.

Thanks

CGI session test not multi-user friendly.

The test 'TestCGI.test_session' is failing for me:

E           Failed: got an unexpected warning:
E           Warning: session_start(): open(/tmp/xyz-abcd, O_RDWR) failed: Permission denied (13)

This is because another user has run the tests and now owns the file.

Looking at the test, I don't know a non-racy way of doing this properly.

Hippy cannot detect cycles in comparisons.

If you were to run this code on hippy, it would not terminate:

class A {
    function __construct($x=NULL) {
        $this->x = $x;
    }
}

$a = new A();
$b = new A($a);
$a->x = $b; // completes the loop

echo $a == $b;

There is a test to this effect in #85.

Zend would say this:

PHP Fatal error:  Nesting level too deep - recursive dependency? in /tmp/cycle.php on line 9

Notice how it is not sure. This is because -- short of storing all pairs of things that you have previously compared (obviously infeasable) -- they use counters to guess when a cycle has been encountered.

Relevant code in PHP:
https://github.com/php/php-src/blob/master/Zend/zend_object_handlers.c#L1338

Specifically this macro:
https://github.com/php/php-src/blob/master/Zend/zend_object_handlers.c#L42

It's not yet clear to me if zend could reject programs that don't have cycles. The recursion counter threshold 3 at first glance is fishy (I have not spent much time thinking about this).

Also relevant:

http://www.phpinternalsbook.com/classes_objects/internal_structures_and_implementation.html

Specifically:

The apply_count member serves the same role as the nApplyCount member of
HashTable: It protects against infinite recursion. It is used via the macros
Z_OBJ_UNPROTECT_RECURSION(zval_ptr) (leave recursion) and
Z_OBJ_PROTECT_RECURSION(zval_ptr) (enter recursion). The latter will throw an error
if the nesting level for an object is 3 or larger. Currently this protection mechanism
is only used in the object comparison handler.

Random number generated numbers greater then MAX_RAND

The PRNG generates numbers greater then RANDMAX. This makes the output of mt_rand to be incorrect.

The attached code will detect this case.

<?php
$maxrandom = getrandmax();
for($i=0;$i<1000;++$i){
    $r = mt_rand();
    if ($r > $maxrandom) {
        echo $r;
        exit(1);
    }
}

php adapters assert instead of raising exception

PyHyp users can encounter into run-time asserts with things like this:

def to_php(self, php_interp):
    # Classes are not first class in PHP, so this would make no sense.
    assert False

Should they not be exceptions?

HippyVM and Nginx

Is it possible to interface HippyVM and Nginx ? I would like use HippyVM by a socket.