• This HTML5 example uses the Ollama REST API and the browser Speech API to add speech detection and speech synthesis for Ollama.

• Web Demo - Note for the demo to work you need the Ollama chat API to add the Access-Control-Allow-Origin: * header. (See below)

• Ollama REST API

• Browser (Chrome Recommended)

• Docker Desktop

• Install Ollama with GPU acceleration on Windows:

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

• Install the llama2 model:

docker exec -it ollama ollama run llama2

You can create a new Ollama container to add a nginx proxy with the required header. If you don't want to use the proxy, just clone the repository and use the live-preview VS Code extension to preview the index.html and it will work without the header. In the future it would be better to modify the client Golang code to add the header so that websites can talk directly to the Ollama REST API.

• First recreate the docker container with an additional port for the nginx proxy.

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 -p 11435:11435 --name ollama ollama/ollama

• Install the llama2 model:

docker exec -it ollama ollama run llama2

• Add vim to edit configs

apt-get update
apt-get install vim

• Install nginx to run a web proxy which adds the custom header

apt-get install nginx

• Edit the nginx configuration

vi /etc/nginx/sites-available/default

• Add the proxy settings and custom header

server {
 listen 11435;
 location / {
   if ($request_method = 'OPTIONS') {
     add_header 'Access-Control-Allow-Origin' '*';
     add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
     add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
     add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
     return 204;
   }
   proxy_pass http://localhost:11434;
 }
 proxy_hide_header 'Access-Control-Allow-Origin';
 proxy_set_header 'Access-Control-Allow-Origin' "*";
 proxy_hide_header 'Content-Type';
 proxy_set_header 'Content-Type' "application/json";
 proxy_hide_header 'Origin';
 proxy_set_header 'Origin' "https://localhost:11434";
 add_header 'Access-Control-Allow-Origin' '*';
 proxy_hide_header 'Sec-Fetch-Mode';
 proxy_hide_header 'Sec-Fetch-Site';
 proxy_hide_header 'Sec-Fetch-Dest';
 proxy_hide_header 'Origin';
}

• Test if the configuration changes are okay.

nginx -t

• Reload the nginx service

service nginx reload

• Start the nginx proxy server

nginx