Giter Club home page Giter Club logo

dtype-next's Introduction

dtype-next

Next generation high performance Clojure toolkit.

Clojars Project

Build Status

New Functionality

Description

dtype-next provides a unified pathway for dealing with contiguous containers of primitive datatypes such as ints and floats. In addition it defines the basis for array programming as found in APL or numpy and a deep Java interface hierarchy with default methods to allow implementing new arrays painless. This interface hierarchy integrates with Java Streams, Spliterators, and various members of the java.util.function package. In addition it extends these concepts to native-heap based containers.

There are namespaces to allow elementwise operations across scalars and arrays, highly optimized reductions across index spaces, and algorithms that operate in index space for use when multiple buffers share an index space.

This library forms the numeric basis that underlies the 'tech.ml.dataset' system. It also defines a language independent ABI which allows zerocopy to C-based systems such as numpy, OpenCV, Julia, TVM and neanderthal.

Additional targets of this library:

  • Small runtime footprint. This is harder than it looks but the main thing is that the system needs to produce the right answers with as little type specific code as necessary.
  • Full native memory support. Malloc, free, memset, memcpy. Just the basics but guaranteed to be available.
  • Graal Native support.
  • Support for JDK-8 through JDK-14.
  • Blogpost, example and involved example of using the FFI architecture across JNA, JDK-16 and GraalNative.

Checkout the Overview, API Documentation, and Cheatsheet.

Native Test

  • In order to get mmap working on the native test I had to grab the larray .so from the uberjar and load it manually. Not a big issue at the end of the day but I was having problems getting graal native to package resources.

Use the scripts to get graal and compile test. The code is located under native-test; so far reader/writing/copying all work for native and jvm heap datasets. Tensors work.

Graal Native

License

Copyright © 2020 Chris Nuernberger

This program and the accompanying materials are made available under the terms of the Eclipse Public License 2.0 which is available at http://www.eclipse.org/legal/epl-2.0.

dtype-next's People

Contributors

ashimapanjwani avatar behrica avatar borh avatar cnuernber avatar harold avatar holyjak avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.