⚡ Low GPU memory? No problem. CPU offloading. ⚡

Basic Colab Notebook

Bark Infinity started as a humble 💻 command line wrapper, a CLI 💬. Built from simple keyword commands, it was a proof of concept 🧪, a glimmer of potential 💡.

Bark Infinity evolved 🧬, expanding across dimensions 🌐. Infinite Length 🎵🔄, Infinite Voices 🔊🌈, and a true high point in human history: 🌍 Infinite Awkwardness 🕺. But for some people, the time-tested command line interface was not a good fit. Many couldn't even try Bark 😞, struggling with CUDA gods 🌩 and being left with cryptic error messages 🧐 and a chaotic computer 💾. Many people felt very… UN INFINITE.

🚀 Bark Infinity 🐾 was born in the command line, and Bark Infinity grew within the command line. We live in the era where old fashioned command line applications are wrapped in ✨fancy Gradio Uis🌈 and 🖱️One Click Installers. We all must adapt to a changing world, right? Or do we?

pip

!git clone https://github.com/JonathanFly/bark.git
%cd bark
!pip install -r requirements-pip.txt
!pip install encodec rich-argparse

(I created a requirements-pip.txt file as well, but haven't tested a full pip route. However you should be able to install with that too.)

1. Go here: https://github.com/conda-forge/miniforge#mambaforge
2. Download this: https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Windows-x86_64.exe a. Install the Mambaforge for your OS, not specifically Windows. OSX for OSX etc. b. Don't install Mambaforge-pypy3. (It might work but not what I tested.) Install the one above that, just plain Mambaforge.
3. Install. Then start the miniforge 'Miniforge Prompt' Terminal which is a new program it installed. You will always use this program for Bark.
4. You should see a terminal that says "(base)". Do not move forward until you see that.
5. Type this:

mamba update mamba
mamba install git

Your terminal still says (base). 6. This step is most of the installation time, the "mamba env create -f environment-cuda.yml" line. TType:

git clone https://github.com/JonathanFly/bark.git
cd bark
mamba env create -f environment-cuda.yml 

Okay stop here and see if something went wrong. When it's done it should say somewhere: "To activate this environment, use conda activate bark-infinity-oneclick" Then type.

mamba activate bark-infinity-oneclick

Note I typed "mamba" not "conda", even though the message said the word conda.

7. Okay now instead of (base) you should see (bark-infinity-oneclick). Do not move on if you still see (base) on your screen.
8. Type:

pip install encodec
pip install rich-argparse

Now if type 'dir' should see 'bark_webui.py' in the list tof files. If you don't, something might have gone wrong bin step 6 where you type 'cd bark' 9. Start Bark like this. (Always making sure you start 'Miniforge Prompt') not (base) TO START (Always making sure you start 'Miniforge Prompt') and make sure you are the /bark directory that has thet bark_webgui.py file

Are you done? Maybe not. You can try skipping this step but something in the libararies are bugged, so you porbably need a step 10.

10. (you can try skipping this if you want)

mamba uninstall pysoundfile
pip install soundfile

Okay you are done. Just type:

python bark_perform.py

or

python bark_webui.py

To restart later, start Miniforge Prompt. Then activate bark-infinity-oneclick (you can set it up to actiate automatically as well), and then:

Option 1: Using commands

mamba activate bark-infinity-oneclick
cd bark
python bark_webui.py

Option 2: Run bark-webui.bat from Windows Explorer as normal, non-administrator, user.

(If you do not have an NVIDIA GPU use environment-cpu.yml instead of environment-cuda.yml)

I dipped my toes back into a bit twitter.com/jonathanfly

Discover cool new voices and reuse them. Performers, musicians, sound effects, two party dialog scenes. Save and share them. Every audio clip saves a speaker.npz file with the voice. To reuse a voice, move the generated speaker.npz file (named the same as the .wav file) to the "prompts" directory inside "bark" where all the other .npz files are.

🔊 With random celebrity appearances!

(I accidently left a bunch of voices in the repo, some of them are pretty good. Use --history_prompt 'en_fiery' for the same voice as the audio sample right after this sentence.)

Any length prompt and audio clips. Sometimes the final result is seamless, sometimes it's stable (but usually not both!).

🎵 Now with Slowly Morphing Rick Rolls! Can you even spot the seams in the most earnest Rick Rolls you've ever heard in your life?

Confused Travolta GIF

Can your text-to-speech model stammer and stall like a student answering a question about a book they didn't read? Bark can. That's the human touch. The semantic touch. You can almost feel the awkward silence through the screen.

Are you tired of telling your TTS model what to say? Why not take a break and let your TTS model do the work for you. With enough patience and Confused Travolta Mode, Bark can finish your jokes for you.

Truly we live in the future. It might take 50 tries to get a joke and it's probabably an accident, but all 49 failures are also very amusing so it's a win/win. (That's right, I set a single function flag to False in a Bark and raved about the amazing new feature. Everything here is small potatoes really.)

BARK INFINITY is possible because Bark is such an amazingly simple and powerful model that even I could poke around easily.

For music, I recommend using the --split_by_lines and making sure you use a multiline string as input. You'll generally get better results if you manually split your text, which I neglected to provide an easy way to do because I stayed too late listening to 100 different Bark versions of a scene an Andor and failed Why was 6 afraid of 7 jokes.

Type --help or use the GUI

Usage: bark_perform.py [-h] [--text_prompt TEXT_PROMPT] [--list_speakers LIST_SPEAKERS] [--dry_run DRY_RUN]
                       [--history_prompt HISTORY_PROMPT] [--prompt_file PROMPT_FILE]
                       [--split_input_into_separate_prompts_by {word,line,sentence,string,random,rhyme,pos,regex}]
                       [--split_input_into_separate_prompts_by_value SPLIT_INPUT_INTO_SEPARATE_PROMPTS_BY_VALUE]
                       [--always_save_speaker ALWAYS_SAVE_SPEAKER] [--output_iterations OUTPUT_ITERATIONS]
                       [--output_filename OUTPUT_FILENAME] [--output_dir OUTPUT_DIR] [--hoarder_mode HOARDER_MODE]
                       [--extra_stats EXTRA_STATS] [--text_use_gpu TEXT_USE_GPU] [--text_use_small TEXT_USE_SMALL]
                       [--coarse_use_gpu COARSE_USE_GPU] [--coarse_use_small COARSE_USE_SMALL]
                       [--fine_use_gpu FINE_USE_GPU] [--fine_use_small FINE_USE_SMALL]
                       [--codec_use_gpu CODEC_USE_GPU] [--force_reload FORCE_RELOAD] [--text_temp TEXT_TEMP]
                       [--waveform_temp WAVEFORM_TEMP] [--confused_travolta_mode CONFUSED_TRAVOLTA_MODE]
                       [--silent SILENT] [--seed SEED] [--stable_mode_interval STABLE_MODE_INTERVAL]
                       [--single_starting_seed SINGLE_STARTING_SEED]
                       [--split_character_goal_length SPLIT_CHARACTER_GOAL_LENGTH]
                       [--split_character_max_length SPLIT_CHARACTER_MAX_LENGTH]
                       [--add_silence_between_segments ADD_SILENCE_BETWEEN_SEGMENTS]
                       [--split_each_text_prompt_by {word,line,sentence,string,random,rhyme,pos,regex}]
                       [--split_each_text_prompt_by_value SPLIT_EACH_TEXT_PROMPT_BY_VALUE]
                       [--extra_confused_travolta_mode EXTRA_CONFUSED_TRAVOLTA_MODE]
                       [--semantic_history_starting_weight SEMANTIC_HISTORY_STARTING_WEIGHT]
                       [--semantic_history_future_weight SEMANTIC_HISTORY_FUTURE_WEIGHT]
                       [--semantic_prev_segment_weight SEMANTIC_PREV_SEGMENT_WEIGHT]
                       [--coarse_history_starting_weight COARSE_HISTORY_STARTING_WEIGHT]
                       [--coarse_history_future_weight COARSE_HISTORY_FUTURE_WEIGHT]
                       [--coarse_prev_segment_weight COARSE_PREV_SEGMENT_WEIGHT]
                       [--fine_history_starting_weight FINE_HISTORY_STARTING_WEIGHT]
                       [--fine_history_future_weight FINE_HISTORY_FUTURE_WEIGHT]
                       [--fine_prev_segment_weight FINE_PREV_SEGMENT_WEIGHT]
                       [--custom_audio_processing_function CUSTOM_AUDIO_PROCESSING_FUNCTION]
                       [--use_smaller_models USE_SMALLER_MODELS] [--semantic_temp SEMANTIC_TEMP]
                       [--semantic_top_k SEMANTIC_TOP_K] [--semantic_top_p SEMANTIC_TOP_P]
                       [--semantic_min_eos_p SEMANTIC_MIN_EOS_P]
                       [--semantic_max_gen_duration_s SEMANTIC_MAX_GEN_DURATION_S]
                       [--semantic_allow_early_stop SEMANTIC_ALLOW_EARLY_STOP]
                       [--semantic_use_kv_caching SEMANTIC_USE_KV_CACHING] [--semantic_seed SEMANTIC_SEED]
                       [--semantic_history_oversize_limit SEMANTIC_HISTORY_OVERSIZE_LIMIT]
                       [--coarse_temp COARSE_TEMP] [--coarse_top_k COARSE_TOP_K] [--coarse_top_p COARSE_TOP_P]
                       [--coarse_max_coarse_history COARSE_MAX_COARSE_HISTORY]
                       [--coarse_sliding_window_len COARSE_SLIDING_WINDOW_LEN]
                       [--coarse_kv_caching COARSE_KV_CACHING] [--coarse_seed COARSE_SEED]
                       [--coarse_history_time_alignment_hack COARSE_HISTORY_TIME_ALIGNMENT_HACK]
                       [--fine_temp FINE_TEMP] [--fine_seed FINE_SEED] [--render_npz_samples RENDER_NPZ_SAMPLES]
                       [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]

Examples • Suno Studio Waitlist • Updates • How to Use • Installation • FAQ

Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects. The model can also produce nonverbal communications like laughing, sighing and crying. To support the research community, we are providing access to pretrained model checkpoints, which are ready for inference and available for commercial use.

Bark was developed for research purposes. It is not a conventional text-to-speech model but instead a fully generative text-to-audio model, which can deviate in unexpected ways from provided prompts. Suno does not take responsibility for any output generated. Use at your own risk, and please act responsibly.

2023.05.01

• ©️ Bark is now licensed under the MIT License, meaning it's now available for commercial use!

• ⚡ 2x speed-up on GPU. 10x speed-up on CPU. We also added an option for a smaller version of Bark, which offers additional speed-up with the trade-off of slightly lower quality.

• 📕 Long-form generation, voice consistency enhancements and other examples are now documented in a new notebooks section.

• 👥 We created a voice prompt library. We hope this resource helps you find useful prompts for your use cases! You can also join us on Discord, where the community actively shares useful prompts in the #audio-prompts channel.

• 💬 Growing community support and access to new features here:

  

• 💾 You can now use Bark with GPUs that have low VRAM (<4GB).

2023.04.20

• 🐶 Bark release!

from bark import SAMPLE_RATE, generate_audio, preload_models
from scipy.io.wavfile import write as write_wav
from IPython.display import Audio

# download and load all models
preload_models()

# generate audio from text
text_prompt = """
     Hello, my name is Suno. And, uh — and I like pizza. [laughs] 
     But I also have other interests such as playing tic tac toe.
"""
audio_array = generate_audio(text_prompt)

# save audio to disk
write_wav("bark_generation.wav", SAMPLE_RATE, audio_array)
  
# play text in notebook
Audio(audio_array, rate=SAMPLE_RATE)

text_prompt = """
    추석은 내가 가장 좋아하는 명절이다. 나는 며칠 동안 휴식을 취하고 친구 및 가족과 시간을 보낼 수 있습니다.
"""
audio_array = generate_audio(text_prompt)

text_prompt = """
    ♪ In the jungle, the mighty jungle, the lion barks tonight ♪
"""
audio_array = generate_audio(text_prompt)

text_prompt = """
    I have a silky smooth voice, and today I will tell you about 
    the exercise regimen of the common sloth.
"""
audio_array = generate_audio(text_prompt, history_prompt="v2/en_speaker_1")

By default, generate_audio works well with around 13 seconds of spoken text. For an example of how to do long-form generation, see this example notebook.

pip install git+https://github.com/suno-ai/bark.git

or

git clone https://github.com/suno-ai/bark
cd bark && pip install . 

Note: Do NOT use 'pip install bark'. It installs a different package, which is not managed by Suno.

Bark has been tested and works on both CPU and GPU (pytorch 2.0+, CUDA 11.7 and CUDA 12.0).

On enterprise GPUs and PyTorch nightly, Bark can generate audio in roughly real-time. On older GPUs, default colab, or CPU, inference time might be significantly slower. For older GPUs or CPU you might want to consider using smaller models. Details can be found in out tutorial sections here.

The full version of Bark requires around 12GB of VRAM to hold everything on GPU at the same time. To use a smaller version of the models, which should fit into 8GB VRAM, set the environment flag SUNO_USE_SMALL_MODELS=True.

If you don't have hardware available or if you want to play with bigger versions of our models, you can also sign up for early access to our model playground here.

Bark is fully generative tex-to-audio model devolved for research and demo purposes. It follows a GPT style architecture similar to AudioLM and Vall-E and a quantized Audio representation from EnCodec. It is not a conventional TTS model, but instead a fully generative text-to-audio model capable of deviating in unexpected ways from any given script. Different to previous approaches, the input text prompt is converted directly to audio without the intermediate use of phonemes. It can therefore generalize to arbitrary instructions beyond speech such as music lyrics, sound effects or other non-speech sounds.

Below is a list of some known non-speech sounds, but we are finding more every day. Please let us know if you find patterns that work particularly well on Discord!

• [laughter]
• [laughs]
• [sighs]
• [music]
• [gasps]
• [clears throat]
• — or ... for hesitations
• ♪ for song lyrics
• CAPITALIZATION for emphasis of a word
• [MAN] and [WOMAN] to bias Bark toward male and female speakers, respectively

Requests for future language support here or in the #forums channel on Discord.

• nanoGPT for a dead-simple and blazing fast implementation of GPT-style models
• EnCodec for a state-of-the-art implementation of a fantastic audio codec
• AudioLM for related training and inference code
• Vall-E, AudioLM and many other ground-breaking papers that enabled the development of Bark

Bark is licensed under the MIT License.

Please contact us at [email protected] to request access to a larger version of the model.

• Twitter
• Discord

We’re developing a playground for our models, including Bark.

If you are interested, you can sign up for early access here.

• Bark uses Hugging Face to download and store models. You can see find more info here.

• Bark is a GPT-style model. As such, it may take some creative liberties in its generations, resulting in higher-variance model outputs than traditional text-to-speech approaches.

• Bark supports 100+ speaker presets across supported languages. You can browse the library of speaker presets here. The community also shares presets in Discord. Bark also supports generating unique random voices that fit the input text. Bark does not currently support custom voice cloning.

• Bark is a GPT-style model, and its architecture/context window is optimized to output generations with roughly this length.

• The full version of Bark requires around 12Gb of memory to hold everything on GPU at the same time. However, even smaller cards down to ~2Gb work with some additional settings. Simply add the following code snippet before your generation:

import os
os.environ["SUNO_OFFLOAD_CPU"] = True
os.environ["SUNO_USE_SMALL_MODELS"] = True

• Bark generates audio from scratch. It is not meant to create only high-fidelity, studio-quality speech. Rather, outputs could be anything from perfect speech to multiple people arguing at a baseball game recorded with bad microphones.