point72 / csp Goto Github PK
View Code? Open in Web Editor NEWcsp is a high performance reactive stream processing library, written in C++ and Python
Home Page: https://github.com/Point72/csp/wiki
License: Apache License 2.0
csp is a high performance reactive stream processing library, written in C++ and Python
Home Page: https://github.com/Point72/csp/wiki
License: Apache License 2.0
There are various formatting errors which occurred when we converted our docs to Markdown. For example, in https://github.com/Point72/csp/wiki/1.-Generic-Nodes-(csp.baselib) the csp.or_
node is formatted as csp.or\_
and in https://github.com/Point72/csp/wiki/2.-Statistics-Nodes-(csp.stats) the LaTeX formatting under the EMA section is terrible. I'm sure there are many similar issues in all the other documentation files as well.
It would be great if someone can go through all the docs and proof-read/fix these errors.
@ngoldbaum if you can create a branch that setups up a build env on windows, I can work on getting the windows build working
@robambalu and I were doing some back and forth on an older fedora-based linux with no ability to modify the system packages.
The install instructions should be straightforwardly runnable in such an environment.
Is your feature request related to a problem? Please describe.
At the moment, you can't define a csp node interactively in ipython (need to be part of an importable script)
Defining the following in IPython (inside of PyCharm)
@csp.node
def f(x: csp.ts[int]) -> csp.ts[int]:
return x
Throws
OSError: could not get source code
Describe the solution you'd like
Ideally we should be able to define them in ipython, but at the very least, the error message should be a bit more helpful (i.e. explaining that defining nodes interactively is not supported yet)
Currently the scripts in the examples
folder are not being run in the tests. We should define a test_examples.py
that disovers example scripts, executes them to check they run without error, and if possible, verifies the output of the script.
As the release processes are updated, we need to document the new workflow to release on PyPI and conda-forge for current and future maintainers.
(Originally noted by @ptomecek, @ngoldbaum, and @trallard)
origin
your fork, set upstream
to the real repo, add remotes for collaborators as neededgit
Additional items that need documenting:
__________________________ TestNumpyAdapter.test_int ___________________________
self = <csp.tests.adapters.test_numpy.TestNumpyAdapter testMethod=test_int>
def test_int(self):
raw_vals = [7, -13, 21]
res = csp.run(g, typ=int, values=np.array(raw_vals), dts=test_dts_ndarray, starttime=test_starttime)
self.assertEqual(res["out"], list(zip(test_dts, raw_vals)))
for dtype in ("b", "h", "i", "l", "object"):
res = csp.run(
g, typ=int, values=np.array(raw_vals, dtype=dtype), dts=test_dts_ndarray, starttime=test_starttime
)
> self.assertEqual(res["out"], list(zip(test_dts, raw_vals)))
E AssertionError: Lists differ: [(dat[61 chars], 1, 2, 1, 0), 243), (datetime.datetime(2000, 1, 2, 2, 0), 21)] != [(dat[61 chars], 1, 2, 1, 0), -13), (datetime.datetime(2000, 1, 2, 2, 0), 21)]
E
E First differing element 1:
E (datetime.datetime(2000, 1, 2, 1, 0), 243)
E (datetime.datetime(2000, 1, 2, 1, 0), -13)
E
E [(datetime.datetime(2000, 1, 1, 1, 0), 7),
E - (datetime.datetime(2000, 1, 2, 1, 0), 243),
E ? ^^
E
E + (datetime.datetime(2000, 1, 2, 1, 0), -13),
E ? ^^
E
E (datetime.datetime(2000, 1, 2, 2, 0), 21)]
csp/tests/adapters/test_numpy.py:34: AssertionError
Not sure exactly what change in numpy caused this. I'm a numpy maintainer and am happy to try to look more closely at this if you'd like me to.
A concise description and key features can be very helpful for new users (across several domains) looking into CSP.
We can discuss some ideas here, and update the wiki homepage and README with this information.
Some ideas for "key features" from discussions with @ptomecek:
We haven't yet decided whether we want to push for adapters to all be part of csp
proper, or if we want/expect adapters to be maintained independently. Regardless, we currently have C++-based parquet
and kafka
adapters, python-based db
, csv
, perspective
, websocket
, symphony
, and slack
adapters, with plans for C++-based redis
, andwebsocket
adapters, so it seems likely that we will have a reliance on lots of python libraries.
We will want to establish a (better) pattern for optional dependency sets, at the very least moving optional dependencies to their own extras blocks in pyproject.toml
, and having a more graceful handling at runtime when dependencies are missing.
Now that it is public, once we clear up the remaining externalization issues, we should do a public pypi release. Currently the 0.0.0
release is just a placeholder.
At the very least, we should tag a release so we don't block #10
Full build: https://github.com/Point72/csp/actions/runs/7821169540
e.g. python code in https://github.com/Point72/csp/wiki/0.-Introduction#graph-building-concepts
Is your feature request related to a problem? Please describe.
Docs navigation can be improved, especially keeping new users in mind. Currently, there is a topical-organization of docs, it'll be nicer to have a distinction between API references, concepts, tutorials, etc.
Describe the solution you'd like
Restructure the docs with some principles from Diátaxis - a framework for structuring technical documentation
Tentative/rough outline:
csp.Struct
(conceptually)csp.DynamicBasket
, csp.Dynamic
)csp.struct
csp.profiler
Also, note that we can create a custom Sidebar. So we can have atomic documentation pages (each page covers one main topic) and have as many pages as we need without worrying about overloading the default wiki TOC.
can use redis-plus-plus which seems full featured enough and easily integrates via vcpkg
we should implement ufunc dispatch so tools like sympy's lambdify can be used with csp.DelayedEdge. Details to follow, we are going to be missing a few functions. Note that it's not strictly necessary to do it this way (I believe you can just override the expected attributes, e.g. edge.log, but I think it's a bit cleaner)
All should be implemented as C++ nodes
@AdamGlustein in case he knows of any fancy implementations, or if stdlib is insufficient
Here's what this looks like in practice. First, you can construct and perform operations symbolically using sympy. Here, we might calculate black scholes.
import sympy as sy
from sympy.stats import Normal as syNormal, cdf
from sympy.utilities.lambdify import lambdify
#spot, strike, vol, days till expiry, interest rate, call or put (1,-1)
spot, strike, vol, dte, rate, cp = sy.symbols('spot strike vol dte rate cp')
T = dte / 260.
N = syNormal('N', 0.0, 1.0)
d1 = (sy.ln(spot / strike) + (0.5 * vol ** 2) * T) / (vol * sy.sqrt(T))
d2 = d1 - vol * sy.sqrt(T)
expr = sy.exp(-rate * T) * (cp * spot * cdf(N)(cp * d1) - cp * strike * cdf(N)(cp * d2))
Then we can extract the variable names and bind them to csp.DelayedEdge
.
from csp import DelayedEdge, ts
syms = list(expr.free_symbols)
names = [s.name for s in syms]
edges = {n: DelayedEdge(ts[float], True) for n in names}
We can then use sympy to apply the symbolic expr
on our delayed edges:
modules = ["scipy", "numpy"]
func = lambdify(syms, expr, modules=modules)
func(*edges.values())
Finally, you can bind these edges to e.g. input adapters. In addition to making this code work, we can add a few helper functions to make the wrapping easier.
This has added utility because right now, csp
does not support dual numbers. So if I wanted e.g. calculate some greeks of the time value expression expr
above, I can do:
delta = diff(expr, spot)
gamma = diff(delta, spot)
theta = diff(expr, dte)
vega = diff(expr, vol)
And I can use the same input adapters and delayed edges for all of these. Note that csp
does not support graph collapsing right now, but we've started investigating it using exprtk
. Being able to do things symbolically (which might result in inefficient computation) but then collapse into pure C++ or exprtk
to regain efficiency would be a very cool and powerful combination.
Describe the bug
If a csp.Struct type is defined with a mutable default ( ie dict ), then all instances of that type will share the same default. Mutate it in one, will mutate for all. This is similar to the age-old python mutable default argument problem, but we have the ability to fix it by deepcopying defaults instead of shallow copying
To Reproduce
class S(csp.Struct):
d : dict = {}
s1 = S()
s2 = S()
s2.d['a'] = 1
print(s1)
> S( d={'a': 1} )
The images linked here are not rendering properly: https://github.com/Point72/csp/wiki/2.-Statistics-Nodes-%28csp.stats%29/#exponential-moving-average
Create a new (crude) example of the pandas extension type functionality to illustrate basic usage.
Right now we rely on policy based data structures for CSP stats code. This tethers us to gcc which is itself a problem, but more critically this reliance on gcc breaks our ABI compatibility with pypi pyarrow on Mac. This compatibility was lucky at best already (we need to move to the C api), but I'd rather have more luck than less 😂.
The immediate repercussion of this reliance is that for the time being, the parquet adapter is disabled on Mac.
I tried installing the latest linux 3.8 wheel in my Windows WSL ubuntu env. It installed fine, but on import I'm hitting missing dependencies off the bat:
>>> import csp
Fatal error on import of /project/_skbuild/linux-x86_64-3.8/cmake-build/cpp/csp/engine/csp_autogen/autogen_types.cpp: failed to import struct module csp.impl.types.autogen_types
Traceback (most recent call last):
File "/home/rob/.local/lib/python3.8/site-packages/csp/impl/types/autogen_types.py", line 3, in <module>
from csp.impl.struct import Struct
File "/home/rob/.local/lib/python3.8/site-packages/csp/impl/struct.py", line 2, in <module>
import ruamel.yaml
ModuleNotFoundError: No module named 'ruamel'
signal 6 (Aborted), address is 0x3e8000001e8 from
[bt]: (0) printBacktrace()
[bt]: (1) sigabrt_handler(int, siginfo_t*, void*)
[bt]: (2) /lib/x86_64-linux-gnu/libc.so.6(+0x46210) [0x7fd56dfd6210]
-packaging:
>>> import csp
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/rob/.local/lib/python3.8/site-packages/csp/__init__.py", line 3, in <module>
from csp.baselib import *
File "/home/rob/.local/lib/python3.8/site-packages/csp/baselib.py", line 18, in <module>
from csp.impl.wiring import DelayedEdge, Edge, OutputsContainer, graph, input_adapter_def, node
File "/home/rob/.local/lib/python3.8/site-packages/csp/impl/wiring/__init__.py", line 3, in <module>
from csp.impl.wiring.delayed_edge import DelayedEdge
File "/home/rob/.local/lib/python3.8/site-packages/csp/impl/wiring/delayed_edge.py", line 8, in <module>
from .graph import graph
File "/home/rob/.local/lib/python3.8/site-packages/csp/impl/wiring/graph.py", line 15, in <module>
from csp.impl.wiring.cache_support.dataset_partition_cached_data import DataSetCachedData, DatasetPartitionCachedData
File "/home/rob/.local/lib/python3.8/site-packages/csp/impl/wiring/cache_support/dataset_partition_cached_data.py", line 12, in <module>
from csp.adapters.output_adapters.parquet import resolve_array_shape_column_name
File "/home/rob/.local/lib/python3.8/site-packages/csp/adapters/output_adapters/parquet.py", line 4, in <module>
from packaging import version
ModuleNotFoundError: No module named 'packaging'
I noticed deprecated use of python setup.py sdist
and python setup.py bdist_wheel
in the Makefile.
Seems that we should be using python -m build
instead according to https://packaging.python.org/en/latest/discussions/setup-py-deprecated/.
I'm trying to build csp
in a VM with 4 GB of RAM and it keeps getting killed by the OOM killer. We should figure out how much RAM is actually needed on the build host and document that.
Is your feature request related to a problem? Please describe.
Due to the policy-based data structures (PBDS) library being gcc specific, I had to change the implementation of Quantile and Rank in cpp/cppnodes/statsimpl.cpp to not use an order statistics tree. Instead, they use a std::multiset to store sorted values, and iterate on that set in linear time to find each quantile/rank when triggered. This is much more inefficient than the O(log n) lookup in the PBDS tree.
Describe the solution you'd like
It would be helpful if someone implemented a balanced (red-black) order statistics tree (OST) natively in csp. The OST is a binary search tree with the additional invariant that each node contains the size of its own subtree (https://en.wikipedia.org/wiki/Order_statistic_tree). This allows for efficient (log n time) insert, find, delete, and index-based lookup.
Describe alternatives you've considered
There are actually 2 data structures that will achieve O(log n) insert, delete and index-based lookup
Additional context
This would be helpful for any non gcc users and would also make the code more readable since we're no longer using 2 different approaches for the same stats calculation.
Discuss and document some concrete user personas, which include knowledge/experience levels, core needs, common pain points, etc.). It can:
csp
makes it easy to build things like chatbots, which react to incoming messages, do some work, and produce outputs, potentially keeping state in between. We have built chatbots with csp
before, but it would be good to make adapters available. Since Slack is quite popular and has a well maintained python SDK, we should write a slack adapter.
Opening this ticket to track, but I'm already writing one which should hopefully be done shortly.
For perspective, we (but mostly @texodus) have setup really great benchmarking which allows us to track granular release-to-release performance metrics. Example:
We have the profiler, but we should setup a benchmark suite and run it in CI (as perspective does).
Is your feature request related to a problem? Please describe.
Using GitHub wiki for docs doesn't allow us to have "reviews" in the documentation contribution workflow.
Describe the solution you'd like
Move the wiki pages into the repository as markdown files, and create an action that moves text over to the wiki. This means we'll treat the wiki as a read-only space.
This also allows us to have linting and testing to ensure consistency.
Describe alternatives you've considered
A docs website is the ideal solution, but is currently not do-able. However, having docs in the repo will allow easier transition to a docs generator if/when that happens.
Additional context
This was discussed with @ptomecek and @robambalu in a meeting today.
Next steps:
Right now csp bootstraps the majority of build dependencies using vcpkg
. However, if we want to eventually add a build of csp to conda-forge we will need to define a conda recipe that gets its build dependencies from conda-forge instead.
From vcpkg.json
we have:
{
"name": "main",
"version-string": "latest",
"dependencies": [
"abseil",
"arrow",
"brotli",
"exprtk",
"gtest",
"librdkafka",
"lz4",
"parquet",
"protobuf",
"rapidjson",
"thrift",
"utf8proc"
],
"overrides": [
{ "name": "arrow", "version": "7.0.0"}
],
"builtin-baseline": "288e8bebf4ca67c1f8ebd49366b03650cfd9eb7d"
}
This corresponds to the following conda-forge
dependencies:
abseil-cpp
arrow-cpp 7.0.0
brotli
exprtk
gtest
librdkafka
lz4
libparquet
protobuf
rapidjson
thrift
utf8proc
The main problem I can see in this dependency list is the pin on arrow-cpp
7.0.0. It looks conda-forge hasn't updated this version in 1 year 8 months. Version 7.0.1 was updated a little more recently, a year ago.
Looking at older arrow-cpp
packages on conda-forge, since they depend on Python, the pin would also restrict us from supporting newer python versions, since python 3.11 and 2.12 don't have arrow-cpp 7.0.0 packages available. It looks like arrow-cpp
dropped its python dependency starting with arrow-cpp 10
.
I don't have context for why the pin on arrow is set the way it is. Can it be loosened at all?
Describe the bug
When following the steps outlined in "Building with Conda on Linux", I observed the following build error. The build failed because the cmake version was less than 3.20.0; however, upon some debugging, it seems the wrong cmake version is being pulled in by skbuild.
(csp) $ which cmake
.../github/csp_venv/envs/csp/bin/cmake
(csp) $ cmake --version
cmake version 3.28.3
Then, upon attempting the build (make build), I get the following error:
CMake Error at CMakeLists.txt:6 (cmake_minimum_required):
CMake 3.20.0 or higher is required. You are running version 3.13.5
Printing out the cmake/version being used I get:
Path to cmake: /usr/bin/cmake3
Cmake version: 3.13.5
It seems that skbuild is using the wrong cmake, different from what is specified by my PATH.
To Reproduce
Expected behavior
make build
should pull in the correct cmake binary and succeed
Desktop (please complete the following information):
We have an adapter for https://symphony.com/ internally. We should open source this.
The documentation should make users aware of the following convenience methods on Edge: apply, pipe and run, with some simple examples of how they can be leveraged to simplify the syntax or work with csp more easily.
apply: https://github.com/Point72/csp/blob/main/csp/impl/wiring/edge.py#L136
pipe: https://github.com/Point72/csp/blob/main/csp/impl/wiring/edge.py#L157
run: https://github.com/Point72/csp/blob/main/csp/impl/wiring/edge.py#L173
The current set of examples can be improved. They need to be triaged into either experience levels (such as "beginner", "intermediate", "advanced") and/or by document type (such as How-tos, Tutorials, or API usage example).
The examples can also be refined to contain more comments and make context clearer, ideally connecting with the corresponding API documentation.
Lastly, it would be interesting to find new sample data sources to use in the examples.
csp
nodes and graphs are type-annotated to return ts["T"]
, and are then connected to other nodes/graphs. Additionally, the return value of these ts["T"]
which is an Edge
, has a number of of convenience methods such as those in #40, a number of operators, and more to come including those in #60. Finally, csp.Struct
is statically typed and should integrate properly in intellisense tools.
We should look to the tools themselves to see how to best integrate:
microsoft/vscode-python#2446
microsoft/python-language-server#260
Describe the bug
It seems like csp.Struct is missing a reference count on the py struct type of nested structs. If the type loses a refcount, it no longer remains valid
To Reproduce
import csp
from csp.impl.struct import defineStruct
class Outter(csp.Struct):
s : csp.Struct
all = []
while True:
sType = defineStruct( 'foo', { 'a' : dict })
all.append(Outter(s = sType(a={ 'foo' : 'bar'})))
print(all)
Is your feature request related to a problem? Please describe.
Logging around lack of memoization of nodes with unhashable type is too noisy (WARN)
https://github.com/Point72/csp/blob/main/csp/impl/mem_cache.py#L209
Describe the solution you'd like
Log at DEBUG level.
Describe alternatives you've considered
None
Additional context
It is alarming to users to see WARNING messages that can safely be ignored.
Describe the bug
Building csp
on Python 3.12 fails due to accessing CPython internals that have been removed.
To Reproduce
Steps to reproduce the behavior:
csp
with a Python 3.12 interpretercsp
itself./Users/goldbaum/Documents/csp/cpp/csp/python/PyNode.cpp: In member function 'void csp::python::PyNode::init(csp::python::PyObjectPtr, csp::python::PyObjectPtr)':
/Users/goldbaum/Documents/csp/cpp/csp/python/PyNode.cpp:64:55: error: 'struct PyGenObject' has no member named 'gi_code'
64 | PyCodeObject * code = ( PyCodeObject * ) pygen -> gi_code;
| ^~~~~~~
See https://github.com/python/cpython/pull/100749/files for the relevant change in CPython
Expected behavior
Build without error
Desktop (please complete the following information):
csp
main branchIs your feature request related to a problem? Please describe.
The csp.typing
Numpy1DArray
and NumpyNDArray
types are confusing for users and lead to a lot of unexpected typing issues. We should be consistent with Numpy and use the base np.ndarray
type for all Numpy time-series objects. With the distinction, you get strange bugs like this:
inp_arr = np.zeros(shape=(4,))
@csp.graph
def arr_1d(x: csp.ts[csp.typing.Numpy1DArray[float]]) -> csp.ts[np.ndarray]:
return x
res = csp.run(arr_1d(arr_1d(csp.const(inp_arr))), starttime=st, endtime=datetime(2020, 2, 7, 9, 1)) # should not raise
This raises an error:
csp.impl.types.instantiation_type_resolver.TSArgTypeMismatchError: In function accept_arr_1d: Expected ts[csp.typing.Numpy1DArray[float]] for argument 'x', got ts[csp.typing.NumpyNDArray[float]]
Describe the solution you'd like
For now, set the Numpy1DArray type to be equal to the NDArray type and make associated changes to the parquet reader/writer such that users need to opt-in to using shape information. By default we will treat the Numpy array as 1-dimensional.
Describe alternatives you've considered
Keeping the two types as separate will continue to lead to weird bugs as I showed above.
Additional context
This change has come up as a discussion topic before as the type distinction led to bugs in the past.
We have a requirement to build against csp libraries for extended C++ adapters using the newer string ABI.
As it stands now, the manylinux wheel is being built with the old CXX string ABI and therefore wont be compatible with existing builds.
Sample issue being hit, from autogenerated code:
In file included from /home/ra7293/user/hfalgo_csp_trading/_skbuild/linux-x86_64-3.11/cmake-build/cpp/csp-trading/adapters/onefeed/csp_autogen/onefeed_types.cpp:2:
/home/ra7293/user/hfalgo_csp_trading/_skbuild/linux-x86_64-3.11/cmake-build/cpp/csp-trading/adapters/onefeed/csp_autogen/onefeed_types.h: In member function 'const std::string& csp::autogen::Header::session_id() const':
/home/ra7293/user/hfalgo_csp_trading/_skbuild/linux-x86_64-3.11/cmake-build/cpp/csp-trading/adapters/onefeed/csp_autogen/onefeed_types.h:455:46: error: static assertion failed
455 | static_assert( sizeof( std::string ) == 8 );
code was generated using the csp wheel's libraries, which have std::string as 8 bytes ( old ABI )
but the compilation we're doing is the new ABI, and so this assertion fails
Describe the bug
In the introduction section of the documentation, under the heading Anatomy of a csp.node the code does not run. We get a python error instead.
To Reproduce
Go to https://github.com/Point72/csp/wiki/0.-Introduction, copy the example code, add from datetime import timedelta
and run it:
from datetime import timedelta
@csp.node # 1
def demo_node(n: int, xs: ts[float], ys: ts[float]) -> ts[float]: # 2
with csp.alarms(): # 3
# Define an alarm time-series of type bool # 4
alarm = csp.alarm(bool) # 5
# 6
with csp.state(): # 7
# Create a state variable bound to the node # 8
s_sum = 0.0 # 9
# 10
with csp.start(): # 11
# Code block that executes once on start of the engine # 12
# one can set timeseries properties here as well, such as # 13
# csp.set_buffering_policy(xs, tick_count=5) # 14
# csp.set_buffering_policy(xs, tick_history=timedelta(minutes=1))# 15
# csp.make_passive(xs) # 16
csp.schedule_alarm(alarm, timedelta(seconds=1), True) # 17
# 18
with csp.stop(): # 19
# code block to execute when the engine is done # 20
# 21
if csp.ticked(xs, ys) and csp.valid(xs, ys): # 22
s_sum += xs * ys # 23
# 24
if csp.ticked(alarm): # 25
csp.schedule_alarm(alarm, timedelta(seconds=1), True) # 26
return s_sum # 27
Expected behavior
Functional code. :) More seriously, adding pass on line 21 would solve the issue. At the same time one could add from datetime import timedelta
to make the code more self-contained.
Error Message
Cell In[8], [line 24](vscode-notebook-cell:?execution_count=8&line=24)
if csp.ticked(xs, ys) and csp.valid(xs, ys): # 22
^
IndentationError: expected an indented block after 'with' statement on line 21
Runtime Environment
0.1.0
3.11.7 | packaged by conda-forge | (main, Dec 23 2023, 14:43:09) [GCC 12.3.0]
linux
We want to make sure releases get github releases along with the sdist and wheels. That will also allow us to upload wheels for more platforms if we hit pypi resource usage limits.
@timkpaine shared https://github.com/finos/perspective/blob/aaff331a2ede998aeae28ea61c806766e1b4c453/.github/workflows/build.yml#L1809 as an example to crib off of.
See below
In file included from /home/goldbaum/github/csp/_skbuild/linux-aarch64-3.11/cmake-build/cpp/csp/engine/csp_autogen/autogen_types.h:6,
from /home/goldbaum/github/csp/cpp/csp/engine/BasketInfo.cpp:2:
In member function 'void csp::Struct::decref()',
inlined from 'void csp::TypedStructPtr<T>::decref() [with T = csp::autogen::DynamicBasketEvents]' at /home/goldbaum/github/csp/cpp/csp/engine/Struct.h:732:20,
inlined from 'csp::TypedStructPtr<T>::~TypedStructPtr() [with T = csp::autogen::DynamicBasketEvents]' at /home/goldbaum/github/csp/cpp/csp/engine/Struct.h:432:19,
inlined from 'void csp::DynamicOutputBasketInfo::addShapeChange(const csp::DialectGenericType&, bool)' at /home/goldbaum/github/csp/cpp/csp/engine/BasketInfo.cpp:167:5:
/home/goldbaum/github/csp/cpp/csp/engine/Struct.h:692:27: warning: pointer used after 'static void csp::Struct::operator delete(void*)' [-Wuse-after-free]
692 | if( --hidden() -> refcount == 0 )
| ~~~~~~~~~~~~^~~~~~~~
In member function 'void csp::Struct::decref()',
inlined from 'void csp::TypedStructPtr<T>::decref() [with T = csp::Struct]' at /home/goldbaum/github/csp/cpp/csp/engine/Struct.h:732:20,
inlined from 'csp::TypedStructPtr<T>::~TypedStructPtr() [with T = csp::Struct]' at /home/goldbaum/github/csp/cpp/csp/engine/Struct.h:432:19,
inlined from 'void csp::DynamicOutputBasketInfo::addShapeChange(const csp::DialectGenericType&, bool)' at /home/goldbaum/github/csp/cpp/csp/engine/BasketInfo.cpp:164:45:
/home/goldbaum/github/csp/cpp/csp/engine/Struct.h:693:20: note: call to 'static void csp::Struct::operator delete(void*)' here
693 | delete this;
| ^~~~
Describe the bug
Running e_11_websocket_output.py
errors out with a type error.
To Reproduce
cd csp/examples
python e_11_websocket_output.py
Traceback (most recent call last):
File "/Users/goldbaum/Documents/csp/csp/examples/e_11_websocket_output.py", line 58, in <module>
csp.run(main, port, num_keys, starttime=datetime.utcnow(), endtime=timedelta(seconds=360), realtime=True)
File "/Users/goldbaum/Documents/csp/csp/impl/wiring/runtime.py", line 183, in run
with ExceptionContext():
File "/Users/goldbaum/Documents/csp/csp/impl/error_handling.py", line 40, in __exit__
raise exc_val.with_traceback(new_tb)
TypeError: baselibimpl.cpp:start:760:TypeError: MyData.fromts() field "angle" expected ts type DOUBLE but got INT64
Expected behavior
Script runs without error
Describe the bug
When there is a type mismatch in the return value of a graph, the message refers to argument 'None' instead of the return value, which is confusing to users, especially for more complicated examples where there are multiple inputs to the graph (especially when some arguments might default to "None").
csp.impl.types.instantiation_type_resolver.TSArgTypeMismatchError: Expected ts[str] for argument 'None', got ts[float]
To Reproduce
import csp
from datetime import datetime
@csp.graph
def graph(x: csp.ts[float]) -> csp.ts[str]:
return x
csp.run(graph, csp.const(1.), starttime=datetime.utcnow())
Expected behavior
Change the error message to:
csp.impl.types.instantiation_type_resolver.TSArgTypeMismatchError: Expected ts[str] for return value, got ts[float]
It would also be nice if it could give you more information about which function it happened in.
Describe the bug
Running e_08_kafka.py
errors out
To Reproduce
Steps to reproduce the behavior:
cd csp/examples
python e_08_kafka.py
Traceback (most recent call last):
File "/Users/goldbaum/Documents/csp/csp/examples/e_08_kafka.py", line 6, in <module>
from csp.adapters.kafka import (
ImportError: cannot import name 'ProtoMessageMapper' from 'csp.adapters.kafka' (/Users/goldbaum/Documents/csp/csp/adapters/kafka.py)
Expected behavior
No error
Is your feature request related to a problem? Please describe.
https://github.com/Point72/csp/wiki/0.-Introduction has great explanatory content. However, it can be too deep for first-time users who want to quickly learn the basics and try some examples.
Describe the solution you'd like
Have a more condensed “Getting started” tutorial with only the basic concepts and link to more detailed contents.
Additional context
Originally proposed by @melissawm, who also created the initial outline.
We should consider switching to the limited python api and build abi3-py38 wheels. This would considerably reduce our CI/CD footprint.
Is your feature request related to a problem? Please describe.
New users can benefit from knowing where the project is headed, what that can expect, and where they can potentially contribute.
Describe the solution you'd like
Discuss potential themes, focus areas, and priorities for the next 12-18, and create a roadmap document. We can then create specific issue tracking roadmap items.
Additional context
There is a preliminary list here: https://github.com/Point72/csp/wiki#roadmap
csp.abs
is accidentally recursive, as it is defining the name abs
but attempts to use the global python builtin abs
in its python definition:
@node(cppimpl=_cspbaselibimpl.abs)
def abs(x: ts[float]) -> ts[float]:
if csp.ticked(x):
return abs(x)
This will be tricky given our use of threading, our various external adapters, reliance on filesystem, etc etc.
There are some references to non-existent or personal email addresses that need to be consolidated, will track them here:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.