I followed the compiling instructions and got a "hello world" working on my Mac! Hurray!

One thing: I had to type

./bin/configure -Djank_cling_build_dir=build/cling-build

I needed three attempts to figure this out, the info in the Readme led me to
plain ./bin/configure and then ./bin/configure -Djank_cling_build_dir=cling-build.

Maybe it makes sense to clarify this in the Readme for Clojurians (like me) who in the last decade had used make and C++ maybe once and now building Jank with "fingers crossed".

Clojure supports unboxed signatures for fns with up to 4 params. jank can do the same thing, but it comes in two steps:

1. We support type hints on fns and generate the appropriate primitive version
2. When we already have the type/box info for each param, we can generate code to invoke accordingly

Right now, jank tracks boxing info, but not type info. This may be required, if we support unboxed things which aren't numbers. I'm not yet sure if we will.

Alex Miller covers an example of this here: https://youtu.be/s_xjnXB994w?t=2529

jank has support for a new special form, called native/raw. It works in place of Clojure's interop syntax and allows for inline C++. But it also support interpolating jank expressions into that C++. Docs on the rationale and final solution are here: https://github.com/jank-lang/jank/blob/main/DESIGN.md#interop

Similar to #24, clj-kondo doesn't know about native/raw and so it causes linting failures. It also doesn't see that the vars referenced within the native/raw interpolations are referenced, so we often end up with unused param warnings which are incorrect. Addressing this will require changes to clj-kondo, which is all Clojure work, I think.

I use submodules in my .vim dir to manage my plugins which doesn't really work well with the current jank setup. And since I use vim in different environments, it's not feasible to hard code a symlink or mess with environment variables.

jank currently uses folly's string, but they're quite slow to construct and don't support custom allocators (meaning they leak like crazy right now). I think we'll need a custom string, but it can follow what folly is doing, if that doesn't impact allocation speed.

interesting project.

suggestion:

r.e. static typing & templates
have you considered trying to retrofit type annotations in a way which fit the syntax of an existing lisp

e.g.
start out with untyped code that looks like clojure, generating C++ templates:
(defn foo [x y z]...) compiles to template<typename X,typename Y, typename Z> auto foo(X x,Y y, Z z){...}
i.e. an argument without any annotation gets an anonymous type parameter.

Then come up with a retrofitted syntax to add annotations for named type-parameters and actual types (perhaps a parenthesised pair in the argument list could specify (argname type)
e.g. (defn foo [(a int)(b int) c]...) would compile to template<typename C> auto foo(int a,int b, C c){...}

clojures' syntax is pretty interesting IMO, the special use of [] and {} does give your eye something specific to latch onto whilst still being easy to parse and not too far from other lisps'.

another possible inspiration for a type annotation syntax is http://adambard.com/blog/core-typed-vs-haskell/

I do realise 'jank' is NOT designed to be compatible with any existing lisp, being a whole new language with unique goals, but maybe if you more closely follow an existing syntax you're more likely to find other people can intuitively read it - 'principle of least surprise'.

I notice ':' having use in your syntax for types, but in lisps that gets used for keywords. Perhaps you could retain lisp-like keywords and a clojure-like syntax where {:x 0 :y 1 :z 2} is like an 'anonymous struct constructor', initializing named fields, and keyword arguments (which seem like a really nice feature in lisps).

Clojure’s object model is intense. In fact, representing it 1:1 from Java to C++ is impossible, since C++ has stricter rules are duplicate base classes and bases with same-name fields and different types. Furthermore, C++, even with the Boehm GC, is much slower at creating objects than the JVM is. It’s bread and butter work for the JVM. Also, while everything is an Object in the JVM, C++ doesn’t have the same notion. If every jank class were to inherit from the same Object class, it would have very serious performance implications when it comes to allocations.

So jank has two key problems here:

1. Creating new boxed values is slow, compared to the JVM
2. Not every jank type can actually be turned into an object, which sometimes means doing some weird dancing to get from one type, through another, and finally to the object base; this generally requires virtual function calls

So far, I’ve worked around the first one by optimizing other things, so jank can be faster than Clojure in a given program for example, but when measuring just the object creations, Clojure is still around twice as fast. I want to fix this.

This task would entail implementing and benchmarking a few different solutions, all of which move jank’s object model away from C++ inheritance and toward something more dataoriented. This gets us around C++’s virtual inheritance allocation costs, but it can also allow every jank type to be treated as an object, which will not only simplify jank’s runtime code, but will itself be a key optimization.

Right now, I have two key design ideas:

1. A very template-heavy approach, which uses bit fields to keep track of which behaviors an object has, as well as which data storage it has
2. An ECS-based approach, which separates object identity from storage, which would aid in cache performance and data locality issues

So far, I have prototyped the first approach and found object creation is nearly twice as fast. This funding would allow me to implement both of these solutions fully, benchmark them, and research further ways to improve them. Finally, I will integrate the fastest solution into jank and reap the benefits.

bdwgc has support for type descriptors, to describe the layout of objects to minimize scanning time. The header is here: https://github.com/ivmai/bdwgc/blob/master/include/gc/gc_typed.h

We should take full advantage of this until we switch to mmtk.

Reference: https://clojure.org/reference/transients

We have transient data structures, just not runtime objects.

• transient_vector (supported by immer)
• transient_hash_map (supported by immer)
• transient_array_map (needs custom support)
• transient_sorted_map (needs sorted map #57)
• transient_hash_set (supported by immer)
• transient_sorted_set (needs sorted set #57)

Then we need the necessary core fns.

• transient
• persistent!
• assoc!
• dissoc!
• conj!
• pop!
• disj!

This is used by the clojure.core/repeat fn. Should be a straightforward runtime object to cook up, as well as implementing the core fn itself.

As mentioned in #24, CLJS uses ~{} and there's no good reason for jank not to as well. There's some remaining work right now to improve the parsing of interpolated expressions, so that will likely be included along with this ticket.

jank has support for a new special form, called native/raw. It works in place of Clojure's interop syntax and allows for inline C++. But it also support interpolating jank expressions into that C++. Docs on the rationale and final solution are here: https://github.com/jank-lang/jank/blob/main/DESIGN.md#interop

Right now, all of this gets highlighted as a string, in vim/emacs/vscode, but the interpolated forms are Clojure code and should be highlighted accordingly. Ideally normal code completion, repl behavior, etc can work from within those forms, but we can take this one step at a time.

We may be able to make the syntax highlighting changes here in https://tree-sitter.github.io/tree-sitter/ and call it a day. But we might also consider getting into the vim/emacs/vscode configurations for Clojure and then forking them for jank to add this support. Would be your call on how to tackle this. I use vim and would love it to have this working, but we'll want great tooling for everyone, so might as well start with whatever you use.

• vim
• emacs
• vscode
• sublime
• pulsar

Hi!
i am trying to build jank on guix
just run bin/configure and got this

-- Running vcpkg install
-- Running vcpkg install - failed
CMake Error at third-party/vcpkg/scripts/buildsystems/vcpkg.cmake:863 (message):
vcpkg install failed. See logs for more information:
/home/moon/code/jank/build/vcpkg-manifest-install.log
Call Stack (most recent call first):
/gnu/store/5p4fnymnslnydibk7qcisal3443w09pm-cmake-minimal-3.21.4/share/cmake-3.21/Modules/CMakeDetermineSystem.cmake:124 (include)
CMakeLists.txt:17 (project)

CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

jank/build/vcpkg-manifest-install.log
is empty

Thanks!

jank uses a tool called Cling to JIT compile C++ at runtime. Cling is a tool built on Clang/LLVM which uses Clang's C++ compilation abilities, but exposes them in a way that works JIT.

Very recently, Cling released a new version which supports LLVM 13. They were previously on LLVM 9. However, that newest version has issues when I enable optimizations, so I can't actually use it. Fixing this would require digging into the C++ internals of what's going on in Cling and why this is failing. I could also put you into contact with the developer of Cling, whom I know, and let him know you're helping out on jank. He should be able to guide you some.

• root-project/cling#483
• root-project/cling#484

• ______

Most lisp-like languages have one indentation scheme (no puns), which is not up for debate. jank is the same way, and it follows clojure's indentation almost to the letter.

Vim's clojure files are here https://github.com/guns/vim-clojure-static but the indentation logic is ridiculous, so I haven't bothered yet.

In short, every first new line of a form indents 2 spaces and subsequent new lines match the previous. An exception is if one argument is passed, then the next new lines match that argument. Certain "special forms" (function, lambda, if, etc) always indent 2 spaces.

So, examples:

(print!
  1
  2)

(; Subsequent lines match the first argument. ;)
(print!
  1 2
  3 4)

(; If one is supplied on the first line, the rest match up. ;)
(print! 1
        2)

(; Lambdas have the signature on the first line, but the body doesn't indent to match it. ;)
(lambda () ()
  (print! "meow"))

(; If expressions take a condition, but the then/else bodies don't match up to it. ;)
(if (empty? foo)
  (print! "empty")
  (print! "not empty"))

We use immer for persistent, immutable data structures in C++ and we use vcpkg to fetch our C++ packages at build time. Unfortunately, the latest vcpkg version for immer doesn't have some new features he's released, like map transients. Updating this would allow us to use those.

This may already exist somewhere, but jank could really use a fully clojure test suite for every clojure.core function. This would be beneficial for all clojure dialects, so perhaps we keep it in a separate repo. Work on this can start prior to jank actually supporting all of those functions. Usage of clojure.test makes sense, as long as there's no interop.

If this doesn't exist, it'd be quite the undertaking, due to the size of clojure.core. However, it'd also be an excellent sweat bed for jank to run continuously. Worth the effort.

Docs: https://clojure.org/guides/weird_characters#_character_literal

This will need support in a few areas:

• lexing (and tests)
• parsing (and tests)
• runtime object
• string sequence (custom sequence type)

folly, a dependency coming in through vcpkg, fails to build on macOS in CI. I don't have a mac for easy testing to figure out what's going on, so I've disabled macOS builds in CI for now. An example log of the failure is here: https://github.com/jank-lang/jank/actions/runs/4029054748/jobs/6926556586

Help would be appreciated, by anyone with a mac and some interest or knowledge of vcpkg, folly, and C++ compilation.

Automated releases for macOS are blocked on this, naturally.

Haven't dived in yet but really love the concept and look forward to seeing it develop. Not crazy about the name but I appreciate the ambition either way! Hope you don't mind but wanted to show some support.

Hi,

I got jank built in Arch on WSL, but got the following error when running the tests:

===============================================================================
/home/chris/repos/jank/test/cpp/jank/jit/processor.cpp:33:
TEST CASE:  Files

/home/chris/repos/jank/test/cpp/jank/jit/processor.cpp:33: FATAL ERROR: test case CRASHED: SIGABRT - Abort (abnormal termination) signal

===============================================================================
[doctest] test cases: 1 | 0 passed | 1 failed | 31 skipped
[doctest] assertions: 0 | 0 passed | 0 failed |
[doctest] Status: FAILURE!
==27405==
==27405== Process terminating with default action of signal 6 (SIGABRT)
==27405==    at 0x9EF98EC: __pthread_kill_implementation (pthread_kill.c:44)
==27405==    by 0x9EAAEA7: raise (raise.c:26)
==27405==    by 0x9E9453C: abort (abort.c:79)
==27405==    by 0x9BCA832: __gnu_cxx::__verbose_terminate_handler() [clone .cold] (vterminate.cc:95)
==27405==    by 0x9BD6D0B: __cxxabiv1::__terminate(void (*)()) (eh_terminate.cc:48)
==27405==    by 0x9BD6D78: std::terminate() (eh_terminate.cc:58)
==27405==    by 0x9BD7AB6: __cxa_pure_virtual (pure.cc:50)
==27405==    by 0x5A34525: clang::CodeGen::CodeGenTBAA::getBaseTypeInfoHelper(clang::Type const*) (in /home/chris/repos/jank/build/libjankcling.so)
==27405==    by 0x5A337BC: clang::CodeGen::CodeGenTBAA::getBaseTypeInfo(clang::QualType) (in /home/chris/repos/jank/build/libjankcling.so)
==27405==    by 0x5A342D2: clang::CodeGen::CodeGenTBAA::getBaseTypeInfoHelper(clang::Type const*) (in /home/chris/repos/jank/build/libjankcling.so)
==27405==    by 0x5A337BC: clang::CodeGen::CodeGenTBAA::getBaseTypeInfo(clang::QualType) (in /home/chris/repos/jank/build/libjankcling.so)
==27405==    by 0x5A342D2: clang::CodeGen::CodeGenTBAA::getBaseTypeInfoHelper(clang::Type const*) (in /home/chris/repos/jank/build/libjankcling.so)

Got any ideas what might be wrong? I could provide more info if you think it would help 👍

We currently run the in-source tests after compiling, but it would be helpful to smoke test the packaged distributions to ensure all necessary files are in there, in the right place, and that the basic functionality works.

• Run the jank compiler test suite for each packaged distribution
• Run the jank runtime test suite for each packaged distribution

Right now, we just have a compiler suite. The runtime suite should use a jank implementation of clojure.test to test the whole of clojure.core and other namespaces shipped with jank. The runtime suite is blocked on #31.

can you point me to http client / server examples please?

We need to figure out what goes into making editor-friendly REPL servers and cook one up. I think it makes the most sense to write it in jank itself, to dogfood sufficiently. I don't think existing Clojure REPL servers can work, but this should also be investigated.

As I intend for jank to be a monorepo of official tools, this project should be merged into main eventually.

This is blocked on #31 and work with native boxes (sockets, mainly).

I'm not sure how this will work yet, if jank will have a lein plugin or if something else is required. But jank will be adding support for require shortly, so we should be able to start hooking in deps, generating compilation artifacts, etc.

For now, blocked on #31.

• Add basic lein plugin which can run jank
• Add support for a run-main command to jank
• Bundle a jank library and publish it to clojars (verify lein is putting the jank source in the jar, too)
• Pull that jank library down as a lein dependency, require the code, and verify it works (may be some bugs in the jar side of module loading, since it's been tested the least)
• Add support for compiler flags in the project.clj which get passed to jank as command-line flags (see its currently supported flags and expose all of those)

NOTE: Keep in mind that jank will load .jank or .cljc files, but not .clj files. Just like ClojureScript.

Adding a run-main command

The jank CLI options are defined here: https://github.com/jank-lang/jank/blob/main/include/cpp/jank/util/cli.hpp
The source to process them is here: https://github.com/jank-lang/jank/blob/main/src/cpp/jank/util/cli.cpp

Our run-main command will be very similar to the run command, but will load a module, rather than a file. We're using a new command, rather than a flag, since there's no point in loading a specific file from the lein project; we want to load a module, based on the class path we have.

I'd duplicate the run function we have into a run_main and use the rt_ctx to load the module. https://github.com/jank-lang/jank/blob/main/src/cpp/main.cpp#L29

We'll need a new transient string target_module option to go along with the command.

With out module loaded, we'll need to find the var for -main within that class and invoke it. The rt_ctx can be used to find the var. You can use dynamic_call on that to invoke it (don't forget to deref the var to get its root).

Passing in the command-line arguments will be more difficult, since we'll need to add CLI support for a -- flag, which then consumes everything after it. jank uses CL11 for flag handling, so you'll need to check out how to do -- there: https://github.com/CLIUtils/CLI11

The final command for lein jank run foo bar spam should look like this:

jank run-main my.program --class-path <...> -- foo bar spam

Recently, I set up automatic releases, based on the last successful commit: https://github.com/jank-lang/jank/releases

These are just a tarball of the result of using cmake to install all necessary files. In order to make this easy to actually install on another machine, we'd really benefit from continuous packages for various distros and OSs. I'd say start first with whatever you use, if you want to do this.

• arch
• debian
• nix
• macOS (homebrew)
• freebsd
• windows (chocolatey)

I'm having trouble building. When I run jank-configure I get an error that lib/magic_enum/include does not exist. Which is true, but I don't know how to make it exist. lib/magic_enum exists but is empty.

Here's what I got:

/home/lyh/Documents/CS/jank/jank/build/llvm/include/llvm/Support/Signals.h:119:24: error: unknown type name 'uintptr_t'; did you mean '__intptr_t'?
/home/lyh/Documents/CS/jank/jank/build/llvm/lib/Support/Unix/Signals.inc:348:11: error: out-of-line definition of 'CleanupOnSignal' does not match any declaration in namespace 'llvm::sys'

Log here:
https://pastebin.ubuntu.com/p/7GmBgxCyn2/

Wondering if it's time to request an "official" reader conditional key and file extension.

https://clojure.org/guides/reader_conditionals

#?(:clj  (Clojure expression)
   :cljs (ClojureScript expression)
   :cljr (Clojure CLR expression)
   :default (fallthrough expression))

Maybe :jank sounds too project specific, maybe :cljl? Kinda hard to read.

Right now, jank reads single-file applications and it compiles clojure.core prior to the user code every time. Adding support for require will also entail the rest of the details around module loading and caching. This sets us up for being used within leiningen.

• support require
• make class path configurable
• add alias support

We won't be able to add the full ns macro until syntax quoting is done.

I would love to have automatic formatting on save enabled; it would make accepting C++ PRs much easier, too. However, I haven't been able to get clang-format to format C++ in a way that matches jank's current style. It may require changes to clang-format in order to do this. Or perhaps some other tool can do it better.

Hi!
Looking forward to this project!

I'd like to discuss if you already started working on anything related to LSP support, I'm maintainer of clojure-lsp and have a lot of experience with LSP protocol and editors using it so if you agree I can try to help with that :)

Some questions:

• It'd be nice to implement in jank or clojure IMO, and we could use lsp4clj which clojure-lsp uses under the hood and helps a lot with all the communication process and infra, WDYT?

Clojure has sorted-map, which builds a PersistentTreeMap, and sorted-set, which builds a PersistentTreeSet. immer doesn't support these, but there's some good info here: arximboldi/immer#105

jank will need its own version, but hopefully we can benefit from the stdlib.

Overall, since we have hash maps, array maps, and hash sets, I think the sorted variants are low priority.

The hack I put in to make ∀ highlight is not very good. It highlights ∀ even if it's in the middle of another symbol. It also doesn't play nicely with word boundaries, which makes the typical case of (∀) pretty annoying.

Vim doesn't allow adding non-ASCII characters to the iskeyword set, so this will either take a change in vim or a bunch of manual regexing. And that's just for the coloring. I'm not sure how to get past the word boundary issue...

When running

./bin/configure -GNinja -DCMAKE_BUILD_TYPE=Debug -Djank_cling_build_dir=build/cling-build 
-- Running vcpkg install

I get the following error:

-- Running vcpkg install - failed
CMake Error at third-party/vcpkg/scripts/buildsystems/vcpkg.cmake:893 (message):
  vcpkg install failed.  See logs for more information:
  /home/mohv/src/jank/build/vcpkg-manifest-install.log
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/CMakeDetermineSystem.cmake:124 (include)
  CMakeLists.txt:28 (project)

here is the full log

Detecting compiler hash for triplet x64-linux...
Detecting compiler hash for triplet x64-clang-static...
The following packages will be built and installed:
    folly[core,zlib]:x64-clang-static -> 2023.05.22.00#1
    immer:x64-clang-static -> 0.8.0#1
    libguarded:x64-clang-static -> 2019-08-27#3
    magic-enum:x64-clang-static -> 0.9.2
Restored 0 package(s) from /home/mohv/.cache/vcpkg/archives in 8.82 us. Use --debug to see more details.
Installing 1/4 folly:x64-clang-static...
Building folly[core,zlib]:x64-clang-static...
-- [OVERLAY] Loading triplet configuration from: /home/mohv/src/jank/vcpkg-triplet/x64-clang-static.cmake
-- Using cached facebook-folly-v2023.05.22.00.tar.gz.
-- Cleaning sources at /home/mohv/src/jank/third-party/vcpkg/buildtrees/folly/src/3.05.22.00-f3a2ac5bcf.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source /home/mohv/src/jank/third-party/vcpkg/downloads/facebook-folly-v2023.05.22.00.tar.gz
-- Applying patch reorder-glog-gflags.patch
-- Applying patch disable-non-underscore-posix-names.patch
-- Applying patch boost-1.70.patch
-- Applying patch fix-windows-minmax.patch
-- Applying patch fix-deps.patch
-- Applying patch openssl.patch
-- Applying patch strong-symbols.patch
-- Using source at /home/mohv/src/jank/third-party/vcpkg/buildtrees/folly/src/3.05.22.00-f3a2ac5bcf.clean
-- Configuring x64-clang-static
-- Building x64-clang-static-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
    Command failed: /home/mohv/src/jank/third-party/vcpkg/downloads/tools/cmake-3.25.1-linux/cmake-3.25.1-linux-x86_64/bin/cmake --build . --config Debug --target install -- -v -j3
    Working Directory: /home/mohv/src/jank/third-party/vcpkg/buildtrees/folly/x64-clang-static-dbg
    See logs for more information:
      /home/mohv/src/jank/third-party/vcpkg/buildtrees/folly/install-x64-clang-static-dbg-out.log

Call Stack (most recent call first):
  /home/mohv/src/jank/build/vcpkg_installed/x64-linux/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
  /home/mohv/src/jank/build/vcpkg_installed/x64-linux/share/vcpkg-cmake/vcpkg_cmake_install.cmake:16 (vcpkg_cmake_build)
  ports/folly/portfile.cmake:71 (vcpkg_cmake_install)
  scripts/ports.cmake:147 (include)


error: building folly:x64-clang-static failed with: BUILD_FAILED
Elapsed time to handle folly:x64-clang-static: 3.8 min
Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
    https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+folly
You can submit a new issue at:
    https://github.com/microsoft/vcpkg/issues/new?title=[folly]+Build+error&body=Copy+issue+body+from+%2Fhome%2Fmohv%2Fsrc%2Fjank%2Fbuild%2Fvcpkg_installed%2Fvcpkg%2Fissue_body.md

-- Running vcpkg install - failed
CMake Error at third-party/vcpkg/scripts/buildsystems/vcpkg.cmake:893 (message):
  vcpkg install failed.  See logs for more information:
  /home/mohv/src/jank/build/vcpkg-manifest-install.log
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/CMakeDetermineSystem.cmake:124 (include)
  CMakeLists.txt:28 (project)


CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!

I got this when trying to run this on my Macbook Pro M1.

[nix-shell:~/Projects/jank]$ jank-configure
The Meson build system
Version: 0.61.2
Source dir: /Users/pez/Projects/jank
Build dir: /Users/pez/Projects/jank/build
Build type: native build
Project name: jank
Project version: snapshot
C++ compiler for the host machine: clang++ (clang 11.1.0 "clang version 11.1.0")
C++ linker for the host machine: clang++ ld64 530
Host machine cpu family: aarch64
Host machine cpu: arm64

meson.build:8:0: ERROR: Include dir lib/magic_enum/include does not exist.

A full log can be found at /Users/pez/Projects/jank/build/meson-logs/meson-log.txt

Please let me know if I can provide any further detail. Or if I can make any experiments that would help you understand what's going on (or not).

Now that #27 is done, we can add a hash map with transients into jank.

• Add new hash map object
• Add promotion from array map to hash map, based on size
• Codegen hash maps by default, based on size of map (16 or more in Clojure)
• Add hash-map function to clojure.core
• Ensure both map types are map?

This is blocked on #37, which is changing the whole object model. Makes sense to only do this afterward.

As title, thanks.