zeromq / fszmq Goto Github PK
View Code? Open in Web Editor NEWAn F# binding for the ZeroMQ distributed computing library. For more information, please visit:
Home Page: http://zeromq.github.io/fszmq
License: Mozilla Public License 2.0
An F# binding for the ZeroMQ distributed computing library. For more information, please visit:
Home Page: http://zeromq.github.io/fszmq
License: Mozilla Public License 2.0
Assess impact of the following:
SetLastError=false
with P/Invoke callslibzmq
do more high-level (i.e. multi-part message) processingWeed out unused (and, where possible, deprecated) functions and options, while ensuring new options are supported, and that all option names make sense.
Layer something dynamic (possibly reactive?) over the monitoring API so it is simpler for managed consumers.
Need up-to-date reference documentation for the full fszmq API. This should display well in both VS Intellisense and via FSharp.Formatting (i.e. XMLDoc and Markdown need to be mixed surgically).
libzmq exposes zmq_send
and zmq_recv
, which overlap with zmq_msg_send
and zmq_msg_recv
(respectively). Perform gap analysis and determine whether or not to expose all for functions.
Currently, EAGAIN is hard-coded to 11. This is most likely wrong. Figure out the correct (per-platfrom) values and adjust.
Review zmq.h
and zmq_util.h
(in libzmq
) and zhelpers.h
(part of zguide
) to determine if fszmq
could benefit from any helper functionality.
Update message-level functionality to be consistent with libzmq and overall fszmq design:
Frame
class to Message
Message
modulezmq_msg_*
methodsI cannot dispose a REP server cleanly. The code:
let ctx = new Context()
let c = Context.req ctx
Socket.setOption c (ZMQ.LINGER, 0) // the socket should close instantly
Socket.connect c "tcp://localhost:5555"
// Some if no exceptions, None if there're
let recv socket =
try Some (Socket.recv socket)
with e -> printfn "Exn on recv: %A" e; None
let rec client() = async {
Socket.send c (Encoding.UTF8.GetBytes "Hello!")
match recv c with
| Some msg -> return! client()
| None -> printfn "Loop finished." }
Async.Start (client())
Console.ReadKey() |> ignore
printf "Disposing socket..."
(c :> IDisposable).Dispose()
printfn "done."
printf "Disposing context..."
try (ctx :> IDisposable).Dispose() with e -> printfn "Exn on context dispose: %A" e
printfn "done."
printfn "The end."
I run the console, wait for some seconds, then press any key. It says me the following:
Disposing socket...done.
Disposing context...Exn on recv: System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
at fszmq.C.zmq_msg_recv(IntPtr msg, IntPtr socket, Int32 flags)
at fszmq.SocketModule.Recv(Socket socket)
at [email protected](Socket socket) in L:\git\ZeroMqTest2\ZmqTest.Server\Program.fs:line 15
Loop finished.
So, the recv
has caught the exception and finished as expected. But the Context has hung forever (as we've not got "done. The end." message).
I just wonder if it's a bug or I do things in a wrong way/order?
Should native pointers (for memory, sockets, et cetera) be accessible to consumers?
Adapt the suite of tests found in the zeromq-4x code base.
The current process for generating AssemblyInfo (default mechanism is FAKE) is sub-optimal for the following reasons:
Either extend existing build task or devise a new (template-based?) build task. Ultimately, it would be great to store the most-recently generated variant of this file in source control (as a bit of a sanity check).
Somewhere (probably READ.md
and Index.fsx
) add a note about the documenation being MIT-licensed (as opposed to the source and tests, which are LGPL-licensed).
Master branch needs 4.0.1-updated variant of Socket Monitor API found in 3.2.7 branch.
The build process should compile, test, and package both x86 and x64 variants of fszmq.dll
(actually, fszmq.dll
and associated unit tests are AnyCPU
, but the underlying libzmq.dll
architecture needs to match that of the test runner). This will probably require multiple NuGet packages, unless something clever can be done from the Install.ps1
script.
See also issue #27.
All public API functionality should have test coverage (possibly include random testing).
Ensure optimal experience from C# and VB.NET. But "tests" aren't really the right mechanism. Maybe find a better term/methodology?
It's tacky and cryptic and we just don't need it.
Add support for PLAIN security mechanism as described in the RFC.
At a very high level, this means adding some new socket options and verifying the overall workflow. However, it might be nice to layer a convenience API on top of this functionality.
It might be nice to have a higher-level way of assembling multi-part messages. Some ideas to play with:
libzmq 3.2.1 replaced zmq_ctx_set_monitor
with zmq_socket_monitor
. This should be reflected in both native.fs
and the public API.
Need up-to-date reference documentation for the full fszmq API. This should display well in both VS Intellisense and via FSharp.Formatting (i.e. XMLDoc and Markdown need to be mixed surgically).
Master branch needs to support the Socket Monitor functionality. However, it should do show in a more elegant and obvious way than was implemented in the 3.2.7 branch. Consider doing something with events/observables or, at least, callbacks.
Master branch needs to fully support the new security features in libzmq v4.0.x. Minimally, this means:
Determine if the (few) unimplemented high-performance message functions can be implemented with reasonably close-to-native speed. Specific functions are:
zmq_msg_init_data
zmq_msg_move
zmq_msg_copy
zmq_send_const
It's possible to choose the native DLL that is loaded at runtime. That allows us to have a single Nuget package that includes both x86 and x64 support. Once the native DLL is loaded, all DllImport items use that one. This also allows the nice "Any CPU" support. You can see how this is done with Sqlite (on Windows):
[DllImport("kernel32", SetLastError = true)]
public static extern IntPtr LoadLibrary(string lpFileName);
static MyClassThatUsesTheNativeLibrary()
{
LoadLibrary(Path.Combine(IntPtr.Size > 4 ? "x64" : "x32", "Sqlite.Interop.dll"));
}
update Socket.send
and Socket.recv
to use zmq_send
and zmq_recv
(respectively)
Code base needs narrative-style sample docs (built using FSharp.Formatting).
At a minimum need:
Note: Attempt to connect narrative documentation to zguide samples.
Code base needs narrative-style sample docs (built using FSharp.Formatting). At a minimum need:
Currently, as constructed classes, the three core types (Context
, Socket
, and Message
) use the CLR's default equality and comparison semantics. An analysis should be performed to determine the correct semantics. Possible options include:
Maybe embedded a small variant of the library's logo graphic on the API documentation and non-home-page narrative documentation.
ZMQ.STREAM
constantstream
function to Context
modulethe declaration [<MarshalAs(UnmanagedType.AnsiBStr)>]
in
[<DllImport("libzmq",CallingConvention = CallingConvention.Cdecl)>]
extern int zmq_bind(HANDLE socket,
[<MarshalAs(UnmanagedType.AnsiBStr)>]
string address)
and the like, makes mono crash with the following error
mono helloworld.exe ✱ ◼
Unhandled Exception:
System.Runtime.InteropServices.MarshalDirectiveException: string marshalling conversion 35 not implemented
at (wrapper managed-to-native) fszmq.C:zmq_bind (intptr,string)
at fszmq.SocketModule.Bind (fszmq.Socket socket, System.String address) [0x00000] in <filename unknown>:0
at Program.server (System.Object state) [0x00000] in <filename unknown>:0
at [email protected] (System.Object obj) [0x00000] in <filename unknown>:0
at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: string marshalling conversion 35 not implemented
If I remove such directive, it seems to work :
mono helloworld.exe ⏎ ✱ ◼
Connecting to hello world server...
Sending Hello 0...
Received Hello
Received World 0
Sending Hello 1...
Received Hello
Received World 1
Sending Hello 2...
Received Hello
I am not sure what are those for .
I try to build zeromq but it fails
$ git clone -s https://github.com/zeromq/zeromq4-x
$ cd zeromq4-x
$ ./configure --build=i686-pc-mingw32 --host=i686-pc-mingw32
$ make
Making all in src
make[1]: Entering directory '/e/0MQ/zeromq4-x/src'
make all-am
make[2]: Entering directory '/e/0MQ/zeromq4-x/src'
CXX libzmq_la-address.lo
In file included from err.hpp:40:0,
from address.cpp:21:
windows.hpp:168:21: fatal error: Mstcpip.h: No such file or directory
compilation terminated.
Makefile:746: recipe for target 'libzmq_la-address.lo' failed
make[2]: *** [libzmq_la-address.lo] Error 1
make[2]: Leaving directory '/e/0MQ/zeromq4-x/src'
Makefile:560: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/e/0MQ/zeromq4-x/src'
Makefile:412: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1
How did you manage to build the libzmq.dll?
There's some things worth "borrowing" in both the zeromq4x tests and in the czmq library.
Possibly three packages:
Variant | File | Platform |
---|---|---|
fszmq.x86.nupkg | ||
fszmq.dll |
AnyCPU |
|
libzmq.dll |
x86 |
|
fszmq.x64.nupkg | ||
fszmq.dll |
AnyCPU |
|
libzmq.dll |
x64 |
|
fszmq.nupkg | ||
fszmq.dll |
AnyCPU |
|
(no native library) |
As an alternative, just ship fszmq.dll
-- and let the consumer worry about libzmq.dll
.
NOTE: this doesn't even account for variations around OpenMP, Sodium, or anything else against which ZeroMQ might be optionally compiled.
All written in C by one of the original authors/architects of 0MQ
Currently, the source is Apache-licensed. This should be changed to LGPLv3, which requires:
LICENSE.txt
fileNOTE: narrative docs and samples should be MIT/X11-licensed.
fszmq
has been updated to use a four-segment versioning scheme. This needs to be explained (in docs? in root README.md
).
Socket
, Context
, and Message
can get and/or set various options, each of which is an int
in native code. Perhaps Units of Measure can be used to improve the experience when working with this portion of the API?
Need more tests, both passing and failing, for the following functions of the Z85
module:
encode : byte[] -> string
decode : string -> byte[]
The "fszmq" logo block, in the upper-left corner of each page, in the narrative and API doc should link back to the documentation home page (index.html
).
Solution needs to support VS 2012 and VS 2012 on Win7 and VS 2012 on Win8.
Message, Socket, and Context implement IDisposable, but their Dispose methods are not public. Calling Dispose from C# requires casting the object to IDisposable. Is there a reason for this, or can the accessibility be changed to make usage cleaner?
when starting from a fresh clone, we get the error message in the title.
is there a specific version of Fake to use ?
Develop a more elegant API over the existing polling mechanism
Seems to occur on back-to-back runs. Possibly killing instance after each run will fix. Also happens in VS2012/VS2013 test runner.
Code base needs to be retargetable to CLR 2.0, 3.0, 3.5, 4.0, and 4.5.
Says cannot convert System.Int32 to ProjectBuildAction. Going to use something similar to what SQLite does; it uses 1 (not 2 like I am in the nuget)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.