Giter Club home page Giter Club logo

llama.swift's Introduction

hey, I'm alex 👋

I'm a full-stack engineer and creative 🔮

I've been building for the Apple ecosystem for a long time, in Objective-C/C++ and Swift, and for AppKit, UIKit and SwiftUI. I've always been driven to ship polished projects with clean APIs, and back in the day I built PXSourceList and PXListView which shipped in some pretty big apps, including early versions of Sketch.

These days I'm excited about Swift, SwiftUI and TypeScript, and am placing a focus on building out fun, interesting projects that make the most of all the incredible tools and technologies that are out there.

I've worked for various companies both big1 2 and small3 4, and I'm currently available for freelance projects 🙌

If you've ever used any of my open-sourced projects, tips are always appreciated 🙏🏼 and you can also sponsor my work on GitHub

Buy Me A Coffee

Find me around the web:

llama.swift's People

Contributors

0-wiz-0 avatar alexrozanski avatar beiller avatar bengarney avatar blackhole89 avatar deepdiffuser avatar etra0 avatar ggerganov avatar jcelerier avatar jooray avatar kharvd avatar maekawatoshiki avatar marckohlbrugge avatar prusnak avatar rgerganov avatar ronsor avatar simonw avatar wizzard0 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

rsych dbonates drewfranz joshluongo stefanschmidt andrewtsuei chrishabiak kohlivarun5 eirias tahabebek

llama.swift's Issues

Metal support from upstream llama.cpp

Hey there, in case you missed it wanted to point out Metal support merged into llama.cpp in the last few days. It's really fast! What would it take to pull in those changes from upstream here?

Add plugins architecture

GPT4All-J requires different cpp bindings than llama.cpp, and uses a specific version of ggml.

In order to support cases like this it might be useful to have a plugin architecture where llama.swift can load different generation plugins which run prediction in an isolated context (but with the support of the core functionality of llama.swift)

An error occurred while I was configuring

image

Complete error message

> python3 -u /var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert-pth-to-ggml.py /Users/wu/Library/Application Support/com.alexrozanski.LlamaChat/models/4CCA4C99-541C-418C-B5C2-F9A11CEE8896/7B 1
Loading model file /Users/wu/Library/Application Support/com.alexrozanski.LlamaChat/models/4CCA4C99-541C-418C-B5C2-F9A11CEE8896/7B/consolidated.00.pth
Loading vocab file /Users/wu/Library/Application Support/com.alexrozanski.LlamaChat/models/4CCA4C99-541C-418C-B5C2-F9A11CEE8896/tokenizer.model
Writing vocab...

Traceback (most recent call last):
  File "/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert-pth-to-ggml.py", line 11, in <module>
    convert.main(['--outtype', 'f16' if args.ftype == 1 else 'f32', '--', args.dir_model])
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 1144, in main
    OutputFile.write_all(outfile, params, model, vocab)
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 953, in write_all
    for i, ((name, lazy_tensor), ndarray) in enumerate(zip(model.items(), ndarrays)):
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 875, in bounded_parallel_map
    result = futures.pop(0).result()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 438, in result
    return self.__get_result()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 950, in do_item
    return lazy_tensor.load().to_ggml().ndarray
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 489, in load
    ret = self._load()
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 497, in load
    return self.load().astype(data_type)
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 489, in load
    ret = self._load()
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 695, in load
    return UnquantizedTensor(storage.load(storage_offset, elm_count).reshape(size))
  File "/private/var/folders/nx/t5wcvmf92yg7n3l4_mnv98_r0000gp/T/FB2F69BC-BCF7-4DDD-965C-EDF15B81DD2F/convert.py", line 679, in load
    raise Exception("tensor stored in unsupported format")
Exception: tensor stored in unsupported format

Could you give an example of MODEL_PATH?

I saw MODEL_PATH=/path/to/ggml-model-q4_0.bin in README, but not sure how to set the path here .

my model file is : /Users/xxx/llama.swift/models/7B/ggml-model-q4_0.bin

when I run test , it shows:

Enter prompt:
hi
Initializing model... Enter prompt:

and nothing happens

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.