pschanely / crosshair Goto Github PK
View Code? Open in Web Editor NEWAn analysis tool for Python that blurs the line between testing and type systems.
License: Other
An analysis tool for Python that blurs the line between testing and type systems.
License: Other
Maybe this has been already done, but I can not find it -- do you use a code formatter for the project? I only see that you check flake8 on Travis, but can not find any script that would automatically format the code.
Would you like me to set up a pre-commit script, akin to this one so that the code can be automatically formatted, test automatically run, docstrings checked etc.?
Expected vs actual behavior
Attempting to verify this function (crosshair-web link):
def make_bigger(s: str) -> str:
'''
post: __return__ != 'FOO'
'''
return s.upper()
produces this error:
/tmp/main.py:5:error:TypeError: __init__() missing 2 required positional arguments: 'typ' and 'smtvar' when calling make_bigger(s = '')
This case would ideally produce a counterexample of "foo"
But at the very least it shouldn't produce this internal CrossHair error.
You know what would be amazing? Symbolically executing tests written for Hypothesis! For example:
@given(st.integers(min_value=0))
def test(x):
assert x ** 2 >= x
# could be equivalent to
def test(x: int):
"""pre: x >= 0"""
assert x ** 2 >= x
this is particularly interesting to me because (a) I'm a core dev of Hypothesis π, (b) Hypothesis tests have to precisely describe the domain of valid inputs, and (c) they have a high density of assertions about error-prone behaviour!
I don't have time to write this myself (at least in the next few months), but would be delighted to help anyone who wants to have a go.
Originally posted by @Zac-HD in #21 (comment)
when trying to pip install crosshair-tool on windows,
conda 4.9.2 ,python 3.7.4
I get the error:
ERROR: Command errored out with exit status 1: 'C:\Python\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\...\AppData\Local\Temp\pip-install-o1trm1ie\z3-solver-crosshair\setup.py'"'"'; file='"'"'C:\Users\....\AppData\Local\Temp\pip-install-o1trm1ie\z3-solver-crosshair\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users...\AppData\Local\Temp\pip-record-yewf5w13\install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.
Per #2, deque cannot be analyzed symbolically. Tentative plan is to implement a "dumb" version on top of a generic list:
CrossHair/crosshair/simplestructs.py
Line 57 in 56ce14e
Let's consider the simple function f
, and imagine executing it with a symbolic value x=0
:
def f(x: int) -> Optional[bool]:
if x > 0:
if x % 2:
return True
return False
return None
Because we execute if x > 0:
, we discover both the False (taken) branch and it's negation but don't discover the un-executed inner branch if x % 2:
. Execution isn't the only way to discover what code does though - we could also read it, or write a program to analyse the abstract syntax tree of f
:
body=[
If(
test=Compare(left=Name(id='x', ctx=Load()), ops=[Gt()], comparators=[Num(n=0)]), # We already found this one
body=[
If(
test=BinOp(left=Name(id='x', ctx=Load()), op=Mod(), right=Num(n=2)), # but *didn't* see this branch
body=[Return(value=NameConstant(value=True))],
orelse=[]),
Return(value=NameConstant(value=False))
],
orelse=[]),
Return(value=NameConstant(value=None))
]
The Fuzzing Book has a chapter on symbolic fuzzing for Python, which is both useful background reading and a good source of starter code!
A trivial answer is that finding more branches per executed example is faster, and who doesn't like improved performance? More seriously, I think this would allow CrossHair to support more programs as well as improving how it tests those which are already supported.
The current "concolic" analysis is fantastic, and handles cases such as dict-based control flow which cannot be inferred from a simple syntactic analysis (but doesn't see inner branches). On the other hand, this AST-based approach can handle cases where it's very difficult to patch in symbolic objects at runtime (but doesn't handle complicated scoping or control-flow issues).
I think it's possible, and feasible, to combine these approaches at runtime using e.g. ast.parse(inspect.getsource(f))
and therefore to keep the strengths and work around the weaknesses of each π
The docstrings seem to be not compliant with PEP 257. For example, the simple present tense is used instead of imperative mood. Is there a particular reason or just habit?
I am not aware of a tool that can enforce present tense, but pydocstyle can enforce imperative mood. Since the project is still young, I would suggest to introduce PEP 257 compliant docstrings as long as this requires only little effort.
See the discussion #51 for more details.
Is your feature request related to a problem? Please describe.
When running into a NameError, in case of a missing import, the resulting stack trace shows the affected class but lacks detail information to pinpoint the location of the line number or function name of the class under inspection.
For example:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/main.py", line 118, in pool_worker_main
messages = analyze_any(module, options)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/core.py", line 536, in analyze_any
return analyze_module(cast(types.ModuleType, entity), options)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/core.py", line 546, in analyze_module
messages.extend(analyze_any(member, options))
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/core.py", line 525, in analyze_any
return analyze_class(cast(Type, entity), options)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/core.py", line 572, in analyze_class
class_conditions = get_class_conditions(cls)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/util.py", line 197, in memo_wrapper
saved[a] = f(a)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/condition_parser.py", line 359, in get_class_conditions
parsed_conditions = get_fn_conditions(method, self_type=cls)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/condition_parser.py", line 273, in get_fn_conditions
sig = resolve_signature(fn)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/condition_parser.py", line 193, in resolve_signature
type_hints = get_type_hints(fn, fn_globals(fn))
File "/usr/lib/python3.8/typing.py", line 1264, in get_type_hints
value = _eval_type(value, globalns, localns)
File "/usr/lib/python3.8/typing.py", line 270, in _eval_type
return t._evaluate(globalns, localns)
File "/usr/lib/python3.8/typing.py", line 518, in _evaluate
eval(self.__forward_code__, globalns, localns),
File "<string>", line 1, in <module>
NameError: name 'BarFoo' is not defined
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/crosshair_tool-0.0.7-py3.8.egg/crosshair/main.py", line 121, in pool_worker_main
raise CrosshairInternal(
crosshair.util.CrosshairInternal: Worker failed while analyzing ./foo/bar/fooBar.py
Describe the solution you'd like
I'd like to see a line number or function name, when available (and sensible of course)
Describe alternatives you've considered
It might be that my claim is not sensible: it could always be the init, but what would happen when there are more classes in a single file? Could be that any missing import in any python file is relatively easy to spot, but sometimes a specific typing for a single function may be faulty or missed in a refactoring.
In general, I like the idea of having one file in libimpl/ per standard library. But builtinslib.py contains the majority of CrossHair's meaningful logic. It's a little too big right now, and will likely get quite a bit bigger.
Suggestions?
When I'm getting started, I often have a module full of type-annotated code but no CrossHair compatible docstrings. As a quick way to get started, it would be great to have an option to treat all functions as if they had a docstring containing post: True
.
This would save some time initially, but also work well with an "assert all your postconditions" style - we'd then only use docstrings for preconditions and invariants.
(which raises a related issue: does CrossHair use information from assertions for interproceedural analysis? Doing so would make this style considerably more efficient...)
Is your feature request related to a problem? Please describe.
One of CrossHair's strengths is that it models the heap and can detect a variety of "aliasing" problems - bugs caused by having the same object accessible through different paths of references. In this example, we can detect that two sub-lists might be the same list, which breaks the expected postcondition.
But the example CrossHair returns is shown as a repr, and the identity characteristics of the inputs is lost - they look just like independent values, even if they are the same.
Describe the solution you'd like
In many cases, this could be expressed using eval-able Python with the "walrus" operator; e.g. (a:=[], a)
to represent a tuple with the same list in it twice.
Even this isn't perfect; there is no single expression to create the list that contains itself as the first item. (a:=[a]
is not a valid expression) Even though it can't be eval'd, though, that kind of an expression would be enough to communicate the intent.
Describe alternatives you've considered
We could re-think how examples are returned; for instance, we could expose id()
s for the objects or something. Probably lots of other options too.
In other words, by default, we detect PEP316, icontract, and assert-based contracts.
This crosshair-web example should be confirmed, but reports a confusing counterexample instead.
Expected vs actual behavior
Running crosshair watch
with Python file or directory raises exception.
Traceback (most recent call last):
File "envs/default/bin/crosshair", line 8, in <module>
sys.exit(main())
File "/home/lhayhurst/aq/code/testing/envs/default/lib/python3.7/site-packages/crosshair/main.py", line 475, in main
options = process_level_options(args)
File "/home/lhayhurst/aq/code/testing/envs/default/lib/python3.7/site-packages/crosshair/main.py", line 60, in process_level_options
arg_val = getattr(command_line_args, optname)
AttributeError: 'Namespace' object has no attribute 'report_all'
To Reproduce
$ envs/default/bin/pip freeze | grep -i crosshair
crosshair-tool==0.0.3
touch foo.py
envs/default/bin/crosshair watch foo.py
The above exception stack should be generated.
Looking through the source code, it looks like the report_all
flag should only be processed when run with crosshair check
, so it seems the wires are getting crossed.
Right now, CrossHair models dictionaries and sets in a underspecified manner when it comes to iteration.
The fix for (1) is straightforward - lock CrossHair's behavior down to the ordered behavior described in the spec.
The fix for (2) is less obvious, but likely we create a subclass of set with completely specified ordering semantics (the constructor takes some sort of ordering key or seed), and only use that when simulating a regular Python set. This has the unfortunate side-effect of making our counterexamples ugly when ordering isn't important. (though in theory that could be detected in a second pass and rewritten to a regular set)
Expected vs actual behavior
The repro case below incorrectly fails with a TypeError saying "sequence item 0: expected str instance, SmtStr found".
str.join is actually quite problematic for CrossHair right now, because (unlike integers, list, etc) there is no good way to make a string-like object that doesn't inherit from string.
To Reproduce
def f(items: List[str]) -> str:
''' post: True '''
return ', '.join(items)
Expected vs actual behavior
I'm not fully sure if it is sensible to expect that this will work. But here goes: In my code I have circular dependencies. I'm not in a position to remove them. I also have mypy type checking in place. MyPy can handle circular dependencies but in order for the code to function, the 'offending' imports are put behind a 'if TYPE_CHECKING:' check. Doing so, MyPy can still perform it's magic.
When I try to have crosshair look at this code, it cannot resolve the imports behind the TYPE_CHECKING flag.
To Reproduce
In a module 'foobar', have an __init__.py
and:
Bar.py:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from foobar.foo import Foo
class Bar():
def __init__(self, foo: "Foo") -> None:
pass
Foo.py:
from foobar.hidey import Hidey
class Foo():
pass
Hidey.py
from foobar.bar import Bar
class Hidey():
pass
Running crosshair ('python3 ../CrossHair/crosshair/main.py watch ./foobar') will result in something like:
NameError: name 'Foo' is not defined
Removing the TYPE_CHECKING flag, results in:
cannot import name 'Foo' from partially initialized module 'foobar.foo' (most likely due to a circular import)
Per #2, all the python date/time support is in C and so we cannot analyze it effectively. Need to figure out a strategy here.
Is your feature request related to a problem? Please describe.
CrossHair cannot do many useful things with regular expression code today, like in this crosshair-web example.
Describe the solution you'd like
Z3 supports a small set of regular expression capabilities natively. In theory, we could monkey-patch the regular expression match/find functions and detect whether the original regular expression string (available via the pattern attribute) is of a form that is expressible in Z3. If so, we could construct a Match-like return object with symbolic values.
This will be a lot of work to implement completely - we'll likely prioritize parts of it selectively.
Describe alternatives you've considered
Letting this use case fall back to a fast randomized fuzz tester may help in some cases.
I see that there are no contributing guidelines yet:
- Make a pull request. There aren't contributing guidelines yet - just check in on gitter to coordinate.
However, it would make sense to describe at least in very short lines how to run the tests, reformat the code etc. @pschanely could you give me at least a set of commands you execute before making a commit?
This issue is related to the issue #57.
Pytest seems like it might be better for us then unittest:
Right now, CrossHair can only effectively create symbolic class instances when types are present on the implementation. It would be great if CrossHair could understand .pyi
typing stubs and use those when appropriate.
Understanding @overload
would have added benefits: we could avoid reporting reporting counterexamples that don't fit one of the given combination of types.
Is your feature request related to a problem? Please describe.
There are multiple runtime DBC packages for Python (e.g., PyContracts, dpcontracts, icontract, β¦). It would be nice to be able to use the same contracts for runtime checking with one of these tools, and compile-time checking with CrossHair.
Describe the solution you'd like
Being able to tell when one of these packages is imported and reading their annotations would probably be simplest. I donβt know how decorated functions are currently handled, but it might also be simpler than trying to statically analyze the functions those decorators return.
Describe alternatives you've considered
CrossHair could either support runtime checking on its own, or it could have a decorator that reads the docstring and applied the relevant decorators from one of these other systems. Both of these sound more complicated than my proposed solution, and they are less compatible with existing runtime-checked contracts.
(This is a split from the issue #58 where we started discussing what happens with the side effects.)
@pschanely could you describe the current behavior? The symbolic values are passed into the function. What happens if a function has side effects in the body? Both the contracts and the body will be executed on the symbolic values, correct? (Maybe you can also describe the monkey patching and why it is necessary?)
(I know we discussed this already off-line, but maybe I missed a couple of important bits.)
I'd like to challenge my understanding, assemble all the puzzle pieces, and finally try to put it into the docs afterwards.
Is your feature request related to a problem? Please describe.
crosshair watch
can accept a directory or filename, but crosshair check
can only accept a filename.
Describe the solution you'd like
It would be nice if both commands supported both filenames and directory arguments.
Describe alternatives you've considered
At a minimum, the current behavior could be clearer in the command help / docs.
Expected vs actual behavior
Because the json module is implemented in C by default, CrossHair cannot analyze it symbolically. (instead, it falls back to testing with a few concrete values)
To Reproduce
This is an example where we'd like to find a counterexample, but cannot.
Ideas for fixing
Cpython actually contains both a pure python implementation and a C implementation. By dropping the appropriate C internal module, we can force use of the pure python version. We did this with the datetime module here, and presumably could do a similar thing for the json module. Note that I'm not completely confident this will be enough to demonstrate the example above, but it'll get us quite a bit closer.
Is your feature request related to a problem? Please describe.
The new PEP591 Final
type breaks CrossHair right now. crosshair-web example
Describe the solution you'd like
When Final is used on an attribute, CrossHair should understand that attribute cannot vary. (by default, CrossHair assumes that attributes can have any value matching their type, subject to CrossHair invariants on the class)
I just cloned the repo and tried to run the examples locally. I got a AttributeError: 'Namespace' object has no attribute 'report_all'
$ python --version
3.8.2
$ crosshair watch -v crosshair/examples/
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/envs/CrossHair/bin/crosshair", line 11, in <module>
load_entry_point('crosshair-tool', 'console_scripts', 'crosshair')()
File "/Users/saul/p/CrossHair/crosshair/main.py", line 475, in main
options = process_level_options(args)
File "/Users/saul/p/CrossHair/crosshair/main.py", line 60, in process_level_options
arg_val = getattr(command_line_args, optname)
AttributeError: 'Namespace' object has no attribute 'report_all'
The documentation is now a bit scattered -- what about moving it to readthedocs and using a proper Sphinx project?
I can set up a Sphinx project and create a pull request. (It makes sense to do this early in the project before it becomes unwieldy. Later migrations are quite painful at later stages of a project as the references need to be manually adapted.)
Per the python docs, int
and float
objects have some additional methods that aren't part of the abc interface. CrossHair doesn't implement these, and therefore explodes when evaluating code that uses these methods on symbolic values.
This is a tracking issue that we will likely never mark resolved.
CrossHair wants to run in the background (via crosshair watch
), but IDE integrations can still provide value by showing the same results inside the editor.
Generally, we expect IDE integrations to be externally owned/supported, but do want to provide easy tools for such integrations.
Is your feature request related to a problem? Please describe.
I can't test my functions that involve random numbers or time.time()
- the repro cases don't reproduce.
More generally, CrossHair cannot currently be used to test stateful behavior; this includes filesystem, networking, and pipe things, as well astime.time
and the globalrandom
instance.
Describe the solution you'd like
Ideally, a repro case could set up system patches to tee up the statefulness required to reproduce. As a simple example,
time.time = ClockYielding([1500000000, 1500000005, 1500000010]) # the values to be returned by time.time, in order
Obviously, emulating the filesystem and other operating system calls is quite a bit more involved.
Describe alternatives you've considered
One alternative is to do nothing. As it stands right now, testing this kind of behavior is simply out of scope for CrossHair.
Minimising crosshair
-specific configuration maximises ease of experimentation and adoption. To that end, I'd like to propose that leading assertions in a function could be treated as preconditions rather than invariants. An example:
def fib(n: int) -> int:
assert n >= 1 # treated like `pre: n >= 1`
... # calculate the result here
assert result >= 1 # Checked as usual
return result
Such leading assertions currently must hold - because no work is done before they are checked, they are logically equivalent to preconditions. Treating them as such would allow preconditions to be checked at runtime without duplicating them, and synergises well with #22 to support precise checks without any crosshair
-specific configuration.
Expected vs actual behavior
After keeping crosshair watch ./foobar
active for an extended period of time (> 60 min) I get the following messages:
malloc(): unsorted double linked list corrupted
malloc(): unsorted double linked list corrupted
corrupted double-linked listles.
corrupted double-linked listles.
malloc(): unsorted double linked list corrupted
corrupted double-linked listles.
malloc(): unsorted double linked list corrupted
corrupted double-linked listles.
corrupted double-linked listles.
corrupted double-linked listles.
malloc(): unsorted double linked list corrupted
malloc(): unsorted double linked list corrupted
corrupted double-linked listles.
To Reproduce
Leave crosshair watch
active for an while (> 60 min). What might also influence this, is the amount of files in the project; I've got about 486 files under watch.
Is your feature request related to a problem? Please describe.
Particularly when testing functions with Any
type, CrossHair is likely to attempt object()
as an input. The repr for an object looks like <object object at 0x7efc3089f600>
and it's not obvious that the appropriate way to replicate a failure is to pass in object()
.
Describe the solution you'd like
It might make sense to special case the message in this case to be something that's more clear.
Today, many string methods cause symbolic strings to be "materialized" (the symbolic string becomes a concrete string and then gets handed to the native string implementation).
For some of these operations, this is likely the best we can do, but others may be implementable with the SMT solver directly. This is a list of those methods, and we'll check them off when we've decided that we have the implementation we want. For each method, we'll decide to:
Note further that direct unicode support in Z3 isn't yet implemented. We might want to defer spending time on this until that's resolved or until we implement a workaround (as a sequence of bitvectors ... or a massive enum?)
Hey!
This package is really impressive and I am just starting to look through it to better understand how it works. Thanks for putting it together!
One of my first questions was "how does it analyze python control flow?" Other libraries like this I have seen take one of a number of options:
I tried looking a bit through the source code, but wasn't sure which approach you were using here. Just a pointer to a file would be helpful!
Python 3.9 makes some significant changes to type hinting, and CrossHair is likely unprepared. Needs investigation.
Is your feature request related to a problem? Please describe.
Test coverage of main.py is an unfortunate 30 percent presently. Testing this stuff is harder than some of the other bits, but it's important.
Describe the solution you'd like
Improve coverage; this may encourage/require refactoring some of how that code works.
Would you like me to translate the workflow from travis to GitHub?
Additionally, I can adapt a script such as this one for making releases to PyPI automatically whenever you release on GitHub.
It is probably easier to maintain the CI pipeline if everything runs on a single platform. Moreover, Travis seems to be in trouble (see, e.g., this thread on Hacker News).
This is a tracking issue that we will likely never mark resolved.
Much of the data types in the Python standard library is implemented in C; such implementations cannot use symbolic inputs. In these cases, symbolic values have to materialize into concrete values, exploding the search tree and generally turning CrossHair into a naive, very slow fuzz tester.
Expected vs actual behavior
When I run the below code, with crosshair check t.py
, it raises the assertion (i.e. passes in a SmtList
rather than a List
), even though the mypy type states that x
should be a List
or Dict
:
t.py:11:error:AssertionError: Shouldn't have gotten type: <class 'crosshair.core.SmtList'> when calling f(self = 1, x = [0, 0, 0])
To Reproduce
from typing import List, Union, Dict
def f(self, x: Union[List, Dict]):
'''
post: True
'''
if isinstance(x, list): # using isinstance is convention for mypy to know the type of x in the branch
pass
elif isinstance(x, dict):
pass
else:
assert False, f"Shouldn't have gotten type: {type(x)}"
EDIT: Updated code with a Union, since I'm assuming SmtList
is the proxy type.
If the type passed in is going to be different, what's the best way of working with Union types so that both CrossHair and MyPy understand the code?
EDIT 2: fix stupid bug (should be elif rather than if) in my test code
Right now, CrossHair doesn't behave well if you consume an iterator or generator inside a condition:
Possibly, we could use the heap checkpointing logic that we use to implement __old__
to resolve this.
The module localhost_comms
is broken on Windows (due to the way that named temporary files are handled in Windows, see https://bugs.python.org/issue14243).
I also don't see that the module is used anywhere -- can it be simply removed?
This is a very minor thing to consider, but helps with a clean commit history: https://github.com/mristin/opinionated-commit-message
Please let me know if I should add it to the GitHub workflows.
One of CrossHair's strengths is that it models the heap and can detect a variety of "aliasing" problems - bugs caused by having the same object accessible through different paths of references. In this example, we can detect that two sub-lists might be the same list, which breaks the expected postcondition.
But CrossHair only does this for container contents, not for top-level parameters. For instance, it does not understand that the two parameters of this function could be the same list.
Right now, CrossHair only checks functions with at least one postcondition (or an invariant).
But it would be reasonable to check functions with preconditions too, as the precondition could be seen as an assertion that the function shouldn't throw unexpected exceptions.
It's less clear what should happen for methods on classes with invariants but without any explicit preconditions or postconditions.
Is your feature request related to a problem? Please describe.
CrossHair dies with an exception when encountering a PEP589 TypedDict. See this crosshair-web example.
Describe the solution you'd like
Ideally, CrossHair would produce a symbolic value that conforms to the type specification. Note that this is complicated by the fact that TypeDict classes can inherit from each other, and the total
option. I don't think we need to care about the "alternative syntax," as CrossHair requires Python 3.7+ already.
Right now, CrossHair wants you to specify the attributes for your custom classes as class attributes, like so:
class Foo:
age: int
name: str
See also official guidance for typed class definitions here.
Notably, CrossHair doesn't infer types for members that are only assigned in __init__()
. It would be nice to investigate what we could do to support this. A likely option would be to construct proxies using init's (typed) signature. (this is sub-optimal though, because we cannot represent symbolic instances that have been later in modified in ways that are not directly constructible)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.