Giter Club home page Giter Club logo

enchannel-zmq-backend's Introduction

enchannel-zmq-backend

Installation | Usage | Contributors and developers | Learn more about nteract

enchannel-zmq-backend offers the ZeroMQ backend implementation for enchannel.

Technical overview

As a refresher for the reader, enchannel details nteract's lightweight, implementation-flexible specification for communication between a user frontend and a backend, such as a language kernel. The enchannel specification offers a simple description of "what" messages may be passed between frontends and backends, while leaving a developer freedom in "how" to achieve message communication.

enchannel-zmq-backend takes a classic design approach using ZeroMQ, the foundation messaging protocol for the Jupyter project. enchannel-zmq-backend implements backend support for the messaging channels described in the Jupyter messaging specification. This spec explains how front end clients should communicate with backend language kernels which implement the Jupyter messaging specification.

Our backend

enchannel-zmq-backend implements the "how" to communicate messages to and from a backend.

We provide functions to create RxJS Subjects (two way Observables for four of the channels described in the Jupyter messaging specification):

  • shell
  • control
  • iopub
  • stdin

That's it. Functions for four channels; simplicity in action.

Installation

Prerequisite: Node.js and npm

You may use whichever package manager (npm or yarn) best suits your workflow. The nteract team internally uses yarn.

npm install enchannel-zmq-backend
# OR
yarn add enchannel-zmq-backend

Usage

Creating messaging channels

To get access to all of the channels for messaging (shell, control, iopub, and stdin), import and use the createChannels function:

import { createChannels } from 'enchannel-zmq-backend'

The createChannels function accepts two things:

  • an identity

    You'll want to set up your identity, relying on the node uuid package:

    const uuidv4 = require('uuid/v4');
    const identity = uuidv4();
  • a runtime object, such as a kernel (which matches the on-disk JSON). Using spawnteract with this project helps streamline spawning a kernel.

    const runtimeConfig = {
      stdin_port: 58786,
      ip: '127.0.0.1',
      control_port: 58787,
      hb_port: 58788,
      signature_scheme: 'hmac-sha256',
      key: 'dddddddd-eeee-aaaa-dddd-dddddddddddd',
      shell_port: 58784,
      transport: 'tcp',
      iopub_port: 58785
    }

To create the channels object:

const channels = createChannels(identity, runtimeConfig)
const { shell, iopub, stdin, control } = channels;

enchannel-zmq-backend also gives access to all of the channels via a single multipled channel exposed via createMainChannel.

import { createMainChannel } from 'enchannel-zmq-backend';

Similar to the createChannels function, the createMainChannel function accepts both an identity and a runtime object.

const channel = createMainChannel(identity, runtimeConfig);

Messages that are sent via the mutliplexed channel need to define a type property that outlines which channel they should be sent under.

const body = {
    header: {
        msg_id: `execute_9ed11a0f-707e-4f71-829c-a19b8ff8eed8`,
        username: "rgbkrk",
        session: "00000000-0000-0000-0000-000000000000",
        msg_type: "execute_request",
        version: "5.0"
    },
    content: {
        code: 'print("woo")',
        silent: false,
        store_history: true,
        user_expressions: {},
        allow_stdin: false
    }
};
const message = { type: "shell", body };

enchannel-zmq-backend also offers four convenience functions to easily create the messaging channels for control, stdin, iopub, and shell :

import {
  createControlSubject,
  createStdinSubject,
  createIOPubSubject,
  createShellSubject,
} from 'enchannel-zmq-backend';

Creating a subject for the shell channel:

const shell = createShellSubject(identity, runtimeConfig)

Subscribing to messages

Here's an example about how to subscribe to iopub messages:

const iopub = createIOPubSubject(identity, runtimeConfig);
var subscription = iopub.subscribe(msg => {
  console.log(msg);
}

// later, run subscription.unsubscribe()

Since these channels are RxJS Observables, you can use filter, map, scan and many other RxJS operators:

iopub.filter(msg => msg.header.msg_type === 'execute_result')
     .map(msg => msg.content.data)
     .subscribe(x => { console.log(`DATA! ${util.inspect(x)}`)})

Sending messages to the kernel

Executing code will rely on sending an execute_request to the shell channel.

var message = {
  header: {
    msg_id: `execute_9ed11a0f-707e-4f71-829c-a19b8ff8eed8`,
    username: 'rgbkrk',
    session: '00000000-0000-0000-0000-000000000000',
    msg_type: 'execute_request',
    version: '5.0',
  },
  content: {
    code: 'print("woo")',
    silent: false,
    store_history: true,
    user_expressions: {},
    allow_stdin: false,
  },
};

Currently, you'll need to have at least one subscription activated before you can send on a channel.

> shell.subscribe(console.log)
> shell.next(message)
> Message {
  header:
   { username: 'rgbkrk',
     msg_type: 'execute_reply',
     msg_id: '0f6d37f3-56a2-41fd-b3ed-90cc189ac423',
     version: '5.1',
     session: '40472e70-e008-48d1-9537-55837a905c05',
     date: '2016-01-12T00:39:44.686986' },
  parent_header:
   { username: 'rgbkrk',
     session: '00000000-0000-0000-0000-000000000000',
     version: '5.0',
     msg_id: 'execute_9ed11a0f-707e-4f71-829c-a19b8ff8eed8',
     msg_type: 'execute_request' },
  metadata:
   { dependencies_met: true,
     engine: '34d73425-4f04-4b57-9bc7-b46e3100e1fd',
     status: 'ok',
     started: '2016-01-12T00:39:44.684534' },
  content:
   { status: 'ok',
     execution_count: 60,
     user_expressions: {},
     payload: [] } }

Contributors and developers

ZeroMQ Dependency

If you plan to contribute to this project or extend it, you will need to have ZeroMQ installed on your system. The easiest way to do this is to install nteract's zmq-prebuilt binary for your operating system.

Install a local development environment

To set up a development environment, you'll need to install:

Then, fork and clone this repo:

git clone https://github.com/nteract/enchannel-zmq-backend.git
cd enchannel-zmq-backend
yarn

Develop! We welcome new and first time contributors.

Learn more about nteract

nteract animated logo

enchannel-zmq-backend's People

Contributors

alexandercbooth avatar captainsafia avatar carreau avatar greenkeeper[bot] avatar ivanov avatar jdetle avatar jdfreder avatar jdpigeon avatar lgeiger avatar rgbkrk avatar williamstein avatar willingc avatar willwhitney avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

enchannel-zmq-backend's Issues

Interface iterations

  • Technically the iopub should not be a subject. It should only be an Observable. Still have to rely on onCompleted, still a subject.
  • I totally cheated and declared subj.send = subj.onNext to fit our intent rather than RxJS's parlance. I like this better so people can write shell.send(payload). Somebody once told me that adding props at runtime is a performance drag. Once I see that being an issue, we can adapt it. Now we're using RxJS 5 / ES7 Observable parlance.
  • Since these get tied together as subjects, the user is required to subscribe to any they want to onNext ahem send on. Ok, this is still an issue. Sweep it under the rug!

delete all mention of createChannels from the README/docs

You deprecated createChannels completely, but it is the first thing mentioned in the docs in the README.

CoCalc directly uses enchannel-zmq-backend via createChannels, so this was a little bit confusing, and I've been reading all the source code to figure out how to rewrite our code not using createChannels. Doing so wasn't hard, and in fact my code is a lot better just using createMainChannel. That said, you might want to rewrite the docs to reflect this (good) change.

Validate sockets before creating channels object

// TODO: This shouldn't work silently if the socket doesn't actually behave
// like an actual socket
// NOTE: RxJS doesn't error with the fromEvent until there is at least one
// subscriber, which also tells me we might have the wrong behavior
// here as it should go ahead and subscribe unconditionally...
const channels = createMainChannelFromSockets(sockets);


This issue was generated by todo based on a TODO comment in b7d509f. It's been assigned to @captainsafia because they committed the code.

apple silicon (m1 max) npm install error

Error report

error ~/Documents/Github/nteract/applications/desktop/node_modules/zeromq: Command failed.
Exit code: 1
Command: node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)
Arguments:
Directory: ~/Documents/Github/nteract/applications/desktop/node_modules/zeromq
Output:
prebuild-install WARN install No prebuilt binaries found (target=18.4.0 runtime=node arch=arm64 libc= platform=darwin)

gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info find Python using Python version 3.10.9 found at "/opt/homebrew/opt/[email protected]/bin/python3.10"
(node:64123) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
(Use `node --trace-deprecation ...` to show where the warning was created)
gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.10
gyp info spawn args [
gyp info spawn args   '~/Documents/Github/nteract/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '~/Documents/Github/nteract/applications/desktop/node_modules/zeromq/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '~/Documents/Github/nteract/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '~/Library/Caches/node-gyp/18.4.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=~/Library/Caches/node-gyp/18.4.0',
gyp info spawn args   '-Dnode_gyp_dir=~/Documents/Github/nteract/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=~/Library/Caches/node-gyp/18.4.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=~/Documents/Github/nteract/applications/desktop/node_modules/zeromq',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CXX(target) Release/obj.target/zmq/binding.o
../binding.cc:28:10: fatal error: 'zmq.h' file not found
#include <zmq.h>
         ^~~~~~~
1 error generated.
make: *** [Release/obj.target/zmq/binding.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (~/Documents/Github/nteract/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:537:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Darwin 22.2.0

An in-range update of rxjs is breaking the build 🚨

Version 5.3.2 of rxjs just got published.

Branch Build failing 🚨
Dependency rxjs
Current Version 5.3.1
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

rxjs is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build failed Details

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

An in-range update of rxjs is breaking the build 🚨

Version 5.4.1 of rxjs just got published.

Branch Build failing 🚨
Dependency rxjs
Current Version 5.4.0
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

rxjs is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

An in-range update of uuid is breaking the build 🚨

Version 3.1.0 of uuid just got published.

Branch Build failing 🚨
Dependency uuid
Current Version 3.0.1
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

uuid is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 6 commits.

  • c50ac88 Update readme & package version (#198)
  • 082a0b3 v5 support in CLI (#197)
  • 1d56dc9 V5 support (#188)
  • f37f96a (fix) Add .npmignore file to exclude test/ and other non-essential files from packing. (#183)
  • 3b21880 Fix typo (#178)
  • d8e41bd Simple typo fix (#165)

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

An in-range update of sinon is breaking the build 🚨

Version 2.3.5 of sinon just got published.

Branch Build failing 🚨
Dependency sinon
Current Version 2.3.4
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As sinon is β€œonly” a devDependency of this project it might not break production or downstream projects, but β€œonly” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Release Notes Fix: Check configurable on a prop before creating
  • Check configurable on a prop before creating (fixes #1456) (#1462)
Commits

The new version differs by 8 commits.

  • 33f32ac Update docs/changelog.md and set new release id in docs/_config.yml
  • d00ab2b Add release documentation for v2.3.5
  • 1c690aa 2.3.5
  • cc0a3fb Update Changelog.txt and AUTHORS for new release
  • ef2ecec Check configurable on a prop before creating (fixes #1456) (#1462)
  • e7dbfd7 Merge pull request #1460 from gj/patch-1
  • 512018f Fix minor typos
  • d60d4a2 Fix apostrophe typos (#1454)

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.