Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
Home Page: https://jan.ai/
License: GNU Affero General Public License v3.0
What is a data-connector-agent in the first place?
Should we name this service better for users?
Separately:
I know we haven't enabled otel, so maybe we can do something to disable this log. It makes for a very spammy install experience.
Can we:
jan-data-connector-agent-1 | {"timestamp":"2023-09-05T05:50:26.171Z","sequence":128,"loggerClassName":"org.jboss.logmanager.Logger","loggerName":"io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter","level":"SEVERE","message":"Failed to export spans. The request could not be executed. Full error message: Failed to connect to localhost/127.0.0.1:4317","threadName":"OkHttp http://localhost:4317/...","threadId":84,"mdc":{},"ndc":"","hostName":"8ca47fbf537b","processName":"super-connector.jar","processId":1}
Insert a table describing the models supported, performance, backends, etc.
To repro:
/authcloud.jan.ai should use open source Jan (vs. our older hosted version)Should we wait for "Teams share OpenAI Key" to upgrade cloud.jan.ai?
Dark mode looks off on first glance. I suggest we go with default light mode instead
What's the issue?
Env: Mac M1 Pro arm64
Solution: un-dockerize llama.cpp, so we can use Metal
jan-worker-1 |
jan-worker-1 |
jan-graphql-engine-1 | {"detail":{"http_info":{"content_encoding":null,"http_version":"HTTP/1.1","ip":"172.20.0.13","method":"POST","status":200,"url":"/v1/graphql"},"operation":{"query":{"query":"\nmutation chatCompletions($id: uuid = \"\", $data: messages_set_input) {\n update_messages_by_pk(pk_columns: {id: $id}, _set: $data) {\n id\n content\n }\n}\n","variables":{"data":{"content":" Not much, just hanging out here"},"id":"2368262c-5a65-4057-9203-9ec64588ec19"}},"query_execution_time":4.6397458e-2,"request_id":"62a9a2c3-b677-4aae-8b83-7d5c760227ce","request_mode":"single","request_read_time":3.5e-6,"response_size":125,"uncompressed_response_size":125,"user_vars":{"x-hasura-role":"admin"}},"request_id":"62a9a2c3-b677-4aae-8b83-7d5c760227ce"},"level":"info","timestamp":"2023-09-03T0SyntaxError: Unexpected token ':', ": ping - 2"... is not valid JSON
jan-worker-1 | at JSON.parse (<anonymous>)
jan-worker-1 | at worker.js:135:28
jan-worker-1 | at async jsonError (worker.js:228:12) {
jan-worker-1 | stack: SyntaxError: Unexpected token ':', ": ping - 2"...…:135:28
jan-worker-1 | at async jsonError (worker.js:228:12),
jan-worker-1 | message: Unexpected token ':', ": ping - 2"... is not valid JSON
jan-worker-1 | }
jan-worker-1 |
jan-worker-1 |Multi column layout: https://tailwindui.com/components/application-ui/application-shells/multi-column#component-65815af4357dfdcf9be55c390221c5fb
Private Zenhub Image
Right pane slide overs: https://tailwindui.com/components/application-ui/overlays/slide-overs#component-4d1d575bd79eef81afc9a4f2e1900ecb
Private Zenhub Image
Settings page: https://tailwindui.com/components/appl
ication-ui/page-examples/settings-screens
Private Zenhub Image
Current: Jan monorepo is opinionated and heavy with many services, many of which are not critical to the individual enthusiast
e.g. keycloak, minio, services noncritical to the inferencing
Users may not love the many containers, lengthy scaffold time, or they may just not like the particular 3rd party integration tools that we chose.
Ideal: Let's make such services and their respective 3rd integrations hot swappable.
Rough design:
Assumption: right now we force users to install the 2 foundational models (llama 2 and sd 1.5).
Ideal: Let's make both AND/OR either models optional
cloud.jan.ai should be a multi-tenant version of JanNeed to install xcode command line tool
Env: mac pro arm
To repro:
docker compose upctrl+cLogs
jan-data-connector-agent-1 | {"timestamp":"2023-09-04T02:13:34.63Z","sequence":111,"loggerClassName":"org.jboss.logmanager.Logger","loggerName":"io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter","level":"SEVERE","message":"Failed to export spans. The request could not be executed. Full error message: Failed to connect to localhost/127.0.0.1:4317","threadName":"OkHttp http://localhost:4317/...","threadId":29,"mdc":{},"ndc":"","hostName":"0a623d5d26ef","processName":"super-connector.jar","processId":1}
jan-graphql-engine-1 | {"timestamp":"2023-09-04T02:13:35.000+0000","level":"info","type":"startup","detail":{"kind":"migrations-startup","info":"failed waiting for 9691, try increasing HASURA_GRAPHQL_MIGRATIONS_SERVER_TIMEOUT (default: 30)"}}
jan-traefik-1 | time="2023-09-04T02:13:35Z" level=debug msg="Provider event received {Status:die ID:d955a2c5b1f9e7ed8417640a4fef63bc8654b7e17f27e70024d7827a6fc553f0 From:hasura/graphql-engine:v2.31.0.cli-migrations-v3 Type:container Action:die Actor:{ID:d955a2c5b1f9e7ed8417640a4fef63bc8654b7e17f27e70024d7827a6fc553f0 Attributes:map[com.docker.compose.config-hash:e56b1cc19f80980f5e19c0d2fa1a43635bba84ed845ec7832abfd8ee62622c06 com.docker.compose.container-number:1 com.docker.compose.depends_on:data-connector-agent:service_healthy:false com.docker.compose.image:sha256:5a614e0e760a3f91cee5c093cc7c9e2f676004396af7e8dc9214840976fc5d57 com.docker.compose.oneoff:False com.docker.compose.project:jan com.docker.compose.project.config_files:/Users/nicolezhu/eigenvector/jan/docker-compose.yml com.docker.compose.project.working_dir:/Users/nicolezhu/eigenvector/jan com.docker.compose.service:graphql-engine com.docker.compose.version:2.20.2 desktop.docker.io/binds/0/Source:/Users/nicolezhu/eigenvector/jan/app-backend/hasura/migrations desktop.docker.io/binds/0/SourceKind:hostFile desktop.docker.io/binds/0/Target:/migrations desktop.docker.io/binds/1/Source:/Users/nicolezhu/eigenvector/jan/app-backend/hasura/metadata desktop.docker.io/binds/1/SourceKind:hostFile desktop.docker.io/binds/1/Target:/metadata execDuration:30 exitCode:1 image:hasura/graphql-engine:v2.31.0.cli-migrations-v3 name:jan-graphql-engine-1 org.opencontainers.image.ref.name:ubuntu org.opencontainers.image.version:22.04 traefik.enable:true traefik.http.middlewares.graphql-engine.stripprefix.forceslash:false traefik.http.middlewares.graphql-engine.stripprefix.prefixes:/graphql-engine traefik.http.routers.graphql-engine.entrypoints:jan traefik.http.routers.graphql-engine.middlewares:graphql-engine traefik.http.routers.graphql-engine.rule:(PathPrefix(`/graphql-engine`))]} Scope:local Time:1693793615 TimeNano:1693793615863117304}" providerName=docker
dependency failed to start: container jan-graphql-engine-1 is unhealthyShould convert to event later on
something like this would be cool:
Migrated from Linear:
Force reload the page, switch from LLM to SD convo -> blank page without messages loaded
This will need to be taken on by engineers, due to the technical nature of the content.
Env: 4090s on linux x86_64
Convo history:
username
01:47 pm
tell me a lohng story
Llama-2-7B-Chat
01:47 pm
Absolutely! I'd be happy to listen to a long story. Please go ahead and tell me anything that comes to mind. Is there a particular theme or topic you'd like to explore, or would you prefer me to suggest some prompts?
username
01:48 pm
no you tell me a long story about mermaids please
Llama-2-7B-Chat
01:48 pm
### Sure! Here is a long story about mermaids:
username
01:48 pm
where did the story go?
Llama-2-7B-Chat
01:48 pm
[never responds]
Logs:
none - or rather the current logs are so busy that I cannot see what even went wrong
no logs, no nothing.
env: mac m1
Also
env: linux x86 with 4090s
Env: Linux/ubuntu x86_64
Repro:
oviderName=docker
jan-traefik-1 | time="2023-09-05T05:43:15Z" level=debug msg="FIXME: Got an status-code for which error does not match any expected type!!!" error="Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version\": dial unix /var/run/docker.sock: connect: permission denied" module=api status_code=-1
jan-traefik-1 | time="2023-09-05T05:43:15Z" level=error msg="Failed to retrieve information of the docker client and server host: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version\": dial unix /var/run/docker.sock: connect: permission denied" providerName=docker
jan-traefik-1 | time="2023-09-05T05:43:15Z" level=error msg="Provider connection error Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version\": dial unix /var/run/docker.sock: connect: permission denied, retrying in 7.117021959s" providerName=docker
jan-traefik-1 | time="2023-09-05T05:43:22Z" level=debug msg="FIXME: Got an status-code for which error does not match any expected type!!!" status_code=-1 error="Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version\": dial unix /var/run/docker.sock: connect: permission denied" module=api
jan-traefik-1 | time="2023-09-05T05:43:22Z" level=error msg="Failed to retrieve information of the docker client and server host: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version\": dial unix /var/run/docker.sock: connect: permission denied" providerName=docker
jan-traefik-1 | time="2023-09-05T05:43:22Z" level=error msg="Provider connection error Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version\": dial unix /var/run/docker.sock: connect: permission denied, retrying in 12.496205234s" providerName=docker
To repro:
It works but for ~1 minute I get the scary logs:
jan-web-1 | Error refreshing access token TypeError: fetch failed
jan-web-1 | at Object.fetch (node:internal/deps/undici/undici:11576:11)
jan-web-1 | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
jan-web-1 | cause: AggregateError
jan-web-1 | at internalConnectMultiple (node:net:1114:18)
jan-web-1 | at afterConnectMultiple (node:net:1667:5)
jan-web-1 | at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
jan-web-1 | code: 'ECONNREFUSED',
jan-web-1 | [errors]: [ [Error], [Error] ]
jan-web-1 | }
jan-web-1 | }
jan-web-1 | Error refreshing access token TypeError: fetch failed
jan-web-1 | at Object.fetch (node:internal/deps/undici/undici:11576:11)
jan-web-1 | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
jan-web-1 | cause: AggregateError
jan-web-1 | at internalConnectMultiple (node:net:1114:18)
jan-web-1 | at afterConnectMultiple (node:net:1667:5)
jan-web-1 | at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
jan-web-1 | code: 'ECONNREFUSED',
jan-web-1 | [errors]: [ [Error], [Error] ]
jan-web-1 | }
jan-web-1 | }
jan-web-1 | Error refreshing access token TypeError: fetch failed
jan-web-1 | at Object.fetch (node:internal/deps/undici/undici:11576:11)
jan-web-1 | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
jan-web-1 | cause: AggregateError
jan-web-1 | at internalConnectMultiple (node:net:1114:18)
jan-web-1 | at afterConnectMultiple (node:net:1667:5)
jan-web-1 | at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
jan-web-1 | code: 'ECONNREFUSED',
jan-web-1 | [errors]: [ [Error], [Error] ]
jan-web-1 | }
jan-web-1 | }
jan-web-1 | Error refreshing access token TypeError: fetch failed
jan-web-1 | at Object.fetch (node:internal/deps/undici/undici:11576:11)
jan-web-1 | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
jan-web-1 | cause: AggregateError
jan-web-1 | at internalConnectMultiple (node:net:1114:18)
jan-web-1 | at afterConnectMultiple (node:net:1667:5)
jan-web-1 | at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
jan-web-1 | code: 'ECONNREFUSED',
jan-web-1 | [errors]: [ [Error], [Error] ]
jan-web-1 | }
jan-web-1 | }
jan-web-1 | - wait compiling /api/auth/logout/route (client and server)...
jan-web-1 | - event compiled successfully in 181 ms (1812 modules)
jan-web-1 | token is expired, refresh it
jan-web-1 | token is expired, refresh it
jan-web-1 | token is expired, refresh it
jan-web-1 | token is expired, refresh it
jan-web-1 | token is expired, refresh it
jan-web-1 | token is expired, refresh itWhen services are ready, signal to the user that Jan is ready to use.
Log to stdout:
| Service | URL | Credentials |
|---|---|---|
| Keycloak | http://localhost:8088 | Admin credentials are set via the environment variables KEYCLOAK_ADMIN and KEYCLOAK_ADMIN_PASSWORD |
| app-backend (hasura) | http://localhost:8080 | Admin credentials are set via the environment variables HASURA_GRAPHQL_ADMIN_SECRET in file conf/sample.env_app-backend |
| web-client | http://localhost:3000 | Users are signed up to keycloak, default created user is set via conf/keycloak_conf/example-realm.json on keycloak with username: username, password: password |
| llm service | http://localhost:8000 | |
| sd service | http://localhost:8001 |
Right now, when you go to localhost 3000, the web app tab name says "Jan. free blabh blah..." It's legacy seo.
Let's just delete all the SEO metadata and the tab should just say "Jan"
Same thing for the other seo/metadata that I can't think of off of top of my head
install.sh scriptnvidia-smi, cuda, nvidia-driverwget...docker compose upfeat: Install with shell script instead of docker
pros from @louis-jan :
success critieria:
With:
requirement:
Changelog as the last-left-tab on the nav barWe will invest in long-term guides as part of docs site:
Bash script stopped at step 5, without any log
docker info
Client: Docker Engine - Community
Version: 24.0.6
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2
Path: /Users/alandao/.docker/cli-plugins/docker-buildx
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-compose" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-compose: no such file or directory
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-dev" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-dev: no such file or directory
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-extension" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-extension: no such file or directory
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-init" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-init: no such file or directory
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-sbom" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-sbom: no such file or directory
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-scan: no such file or directory
WARNING: Plugin "/Users/alandao/.docker/cli-plugins/docker-scout" is not valid: failed to fetch metadata: fork/exec /Users/alandao/.docker/cli-plugins/docker-scout: no such file or directory
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 2
Server Version: 23.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 1fbd70374134b891f97ce19c70b6e50c7b9f4e0d
runc version: 860f061b76bb4fc671f0f9e900f7d80ff93d4eb7
init version:
Security Options:
seccomp
Profile: builtin
Kernel Version: 6.1.29-0-virt
Operating System: Alpine Linux v3.18
OSType: linux
Architecture: aarch64
CPUs: 10
Total Memory: 15.6GiB
Name: colima
ID: a16ca123-ffbe-4d18-aaf9-8cc0ac00786f
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.