Giter Club home page Giter Club logo

ctrl-gce's Introduction

Install and Use CTRL on Google Compute Engine

Scripts + guides on how to set up a virtual machine on Google Compute Engine capable of running and using Salesforce's very large text-generating model CTRL to generate high-quality text based on conditional parameters.

The CTRL model is so large (12 GB on disk, 15.5 GB GPU VRAM when loaded, even more system RAM during runtime) that it will currently not fit into a free Colaboratory or Kaggle Notebook. Therefore, this setup is necessary to play with the model for now.

Machine Setup Instructions

The VM these instructions create is the minimum, lowest-cost configuration powerful enough to run CTRL without going out-of-memory (P100 GPU, 8 vCPU, 30 GB RAM, preemptible). With this configuration, having the VM up will cost $0.51/hr.

  1. Make sure the gcloud command line tool is set up on your local computer and up-to-date (can update via gcloud components update).
  2. Make sure your Google Cloud Platform project tied to your local computer's gcloud has enough quota in the us-central-1 region (8 CPUs and 1 P100; these should be available by default, but request more quota if they aren't)
  3. Make sure your GCE project has billing set up.
  4. On your local computer, run this gcloud command in a terminal which creates a VM with the specs noted above:
gcloud compute instances create ctrl \
  --zone us-central1-c \
  --boot-disk-size 45GB \
  --image-family tf-latest-gpu \
  --image-project deeplearning-platform-release \
  --maintenance-policy TERMINATE \
  --machine-type n1-standard-8 \
  --metadata install-nvidia-driver=True \
  --preemptible \
  --accelerator='type=nvidia-tesla-p100,count=1'
  

You can view the created instance, Start/Stop it, and delete it, in the Google Compute Engine dashboard.

Once created (after waiting a bit for the GPU drivers to install), SSH into the instance. The recommended way to do is via the gcloud command created from the SSH dropdown in the GCE dashboard, which will look like:

gcloud beta compute --project "<PROJECT ID>" ssh --zone "us-central1-c" "ctrl"

While SSHed into the VM you created, download and run the install_gce.sh script from this repo via:

curl -O -s https://raw.githubusercontent.com/minimaxir/ctrl-gce/master/install_gce.sh
sudo sh install_gce.sh

If you'd like to use the model with a sequence length of 512, you can pass 512 as an argument to the install script, e.g.

sudo sh install_gce.sh 512

You're done! Make sure to Stop the instance in the GCE dashboard when you are finished generating text!

Using CTRL

For basic usage, running the command below in the VM will load the model and eventually start an interactive prompt.

sudo python generation.py --model_dir seqlen256_v1.ckpt/

If you are using the 512-length model, instead you would do:

sudo python generation.py --model_dir seqlen512_v1.ckpt/ --generate_num 512

While generating, you can specify a KeyboardInterrupt to stop generation (Ctrl+C on macOS). It's also recommended to clear the terminal (CMD+K on macOS) occasionally as the entire generated text will be output after each added token.

You must include a control code with each interactive prompt. You can see how the control codes are used in the original paper, or refer to the following examples:

Links

The Links control code allows you to specify a URL and/or a prompt text and have the model attempt to extrapolate the corresponding article. Examples:

Links https://www.buzzfeednews.com/article/annehelenpetersen/jeremy-renner-app-trolling-career-hurt-locker-avengers

The actor who played the villain Loki in Avengers: Infinity War has been accused of being a troll. 
 
Actor Jeremy Renner was criticized for his role as Loki, an antagonist from Marvel's upcoming Avengers movie, after he tweeted that fans should "stop trying to make me look bad." 
 
In response to Renner's tweet, many people on Twitter said they were offended by what they perceived as trolling.

Links https://www.nytimes.com/2020/08/29/us/politics/skynet-ai-online-norad.html

Site Mobile Navigation 
 
A New Online Course for the Super-Smart 
 
The University of Washington is offering a new online course on artificial intelligence that will be taught by an expert in machine learning and neural networks. 
 
The course, called “Skynet,” was developed with help from Google, Facebook, Microsoft and other companies. It has been designed to teach students how to use deep learning, which uses computer programs modeled after neurons to analyze large amounts of data — like images or videos — using only basic rules about what each pixel represents. 
 
In addition to teaching them how to build their own programs, it also teaches them how to apply those algorithms to real-world problems. 

Questions

The Questions control code allows you to specify a question; CTRL will attempt to follow up from that question (your mileage will vary). Examples:

Questions Q: Who is the Zodiac Killer?

A: charles manson 

Questions Q: What is the last digit of pi?

A: 0

Reddit Subreddits

Each Reddit subreddit has a different control code; see the bottom of the paper for a full list. You can optionally specify the Title or the self-post Text body. Examples:

Legal Title: [FL] Is it legal to get into legal trouble?

Text: I was driving home from work and saw a police car pull up behind me. The officer asked me for my license and registration. He then proceeded to ask if i had any drugs in the car. 
 
My question is this: 
 
1. Can you be arrested without probable cause or even being charged with anything at all? 
 
2. If so, what are your chances of getting out of this situation? 
 
3. What happens when they arrest you? 
 
4. How do you go about proving that there were no drugs? 
 
5. Do you have an attorney who can help me? 
 
Thanks 
 
Edit: 
 
Thank you all so much for your responses. This has been very helpful as I'm trying to figure out how best to proceed. 

Opinion Text: Recycling is bad for the world.

It takes a lot of energy to recycle and it creates more waste than it saves. 
 
I think that recycling should be banned because we are destroying our planet with it. 
 
Edit: Thanks everyone who has given me some good arguments against recycling. 
 
Edit 2: I'm not saying that people shouldn't recycle; it's just that they need to do so in an environmentally friendly way.

Amazon Reviews

The Reviews tag can be used to generate Amazon reviews at an optional specified rating or starting text.

Reviews Rating: 1.0

I bought this book because it was recommended by a friend. It is not worth the money. The author has no credentials and his writing style is very poor.

Reviews Rating: 5.0\n\nI died

a little inside when I saw the first page of this book. It was so beautiful and it made me feel like I could do anything. But then I read on to see what happened next. And there were no more pages. The book just stopped. No epilogue, nothing. Just an abrupt ending. I'm not sure if it's because there's another one coming out or what, but that's how I feel. It's almost as though she got tired of writing about her life in New York City and decided that she'd write something else instead.

Command Line Arguments

Unlike other text-generating apps, CTRL has a default temperature of 0, meaning the model chooses the best guess when possible (before repetition penalty is applied). Some CLI arguments you can add:

  • --generate_num — Number of tokens to generate (default: 256, can exceed the model window)
  • --temperature — Controls model creativity (default: 0, may want to increase to 0.2)
  • --nucleus — Controls cutoff for nucleus/top-p sampling (default: 0, may want to set to 0.9)
  • --topk — Controls cutoff for top-k sampling (default: 0)
  • --penalty — Repetition penalty factor (default: 1.2)

Notes

  • Since the model is huge, generation is very slow: about 2 tokens per second with the configuration above. (therefore, it takes about 2 minutes for a full generation with default parameters)
  • The BPEs CTRL uses are "longer" that those used in GPT-2. As a result, a 256-token generation in CTRL is about the same decoded length as a 1024-token generation in GPT-2.
  • When using the Links control code, keep in mind that code is conditioned on OpenWebText, which is conditioned on Reddit data. Therefore, there's a bias toward English websites and Reddit-friendly content. Here's a quick spreadsheet of the most popular domains on Reddit, sans some obvious image-oriented websites.
  • If CTRL gets confused by the Links URL, it tends to fall back to a more general news-oriented output.
  • It is recommended to use Google Compute Engine (even if you aren't following this guide) as the model itself is hosted in Google Cloud Storage and thus it's relatively fast to transfer to a VM (>100 Mb/s), and also lowers the cost for Salesforce.

TODO

  • Support/Test domain detection.

Maintainer/Creator

Max Woolf (@minimaxir)

Max's open-source projects are supported by his Patreon. If you found this project helpful, any monetary contributions to the Patreon are appreciated and will be put to good creative use.

Special Thanks

Adam King for identifying a working implementation of loading the model after unexplained setbacks.

License

MIT

Disclaimer

This repo has no affiliation or relationship with the CTRL team and/or Salesforce.

ctrl-gce's People

Contributors

bentona avatar minimaxir avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ctrl-gce's Issues

'ascii' codec can't encode character

I'm regularly getting this error. Any way to circumvent it ?

Traceback (most recent call last):
File "generation.py", line 275, in
print(tokens_generated_so_far)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 289: ordinal not in range(128)

Keras_patch GPU instead of TPU

Hi,
I've been using the generation GCE setup you recommended (Works wonderfully with Python 2.7), but I haven't been able to get finetuning to work. I appreciate thats a totally different setup.

But, on Google Compute with 2x16gb P100's 8x CPU 30GB ram with Nick Walton's MultiGPU pull request to salesforce/ctrl appears like it should work.

I'm still getting OOM issues,

They've fixed the python3 not being supported in recent changes to ctrl, so I've tried reinstalling all the setup, with python3, 3.5 and 2.7.

019-11-06 11:33:42.236866: I tensorflow/core/common_runtime/bfc_allocator.cc:818] total_region_allocated_bytes_: 15928269056 memory_limit_: 15928269210 available bytes: 154 curr_region_allocation_bytes_: 31856538624
2019-11-06 11:33:42.236876: I tensorflow/core/common_runtime/bfc_allocator.cc:824] Stats:
Limit: 15928269210
InUse: 15928269056
MaxInUse: 15928269056
NumAllocs: 4013
MaxAllocSize: 1262254080

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104 Driver Version: 410.104 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 45C P0 28W / 250W | 0MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla P100-PCIE... Off | 00000000:00:05.0 Off | 0 |
| N/A 48C P0 30W / 250W | 0MiB / 16280MiB | 7% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

Anyway, more asking for you help than anything, how come I can get generation to work on the same setup that wont allow finetuning?

Many thanks
Vince.

How to use CTRL as a conditional generation model?

Thanks for this repo. I would like to ask if you know about how i could fine-tune CTRL on a dataset which has prompts (such as a list of headlines of an event) and the article (think of it as a reverse summarization task where you give a summary it generates the entire text for you). I could create a new control code and fine tune on these new articles. but how do i introduce the headlines to the training data? The training data under a control code is just one big text file of articles. Should i place the headlines of the articles before each corresponding article and fine tune on this data?

Python 3 Support

Now that Python 3 support has been merged into the original repo, this repo should move to Python 3. (it should just require changing the Python versions accordingly.)

Support non-interactive mode

This worked like a charm, thanks so much.

I'm wondering about supporting non-interactive mode e.g. maybe an argument that is the name of a file with inputs? Seems like it would involve re-writing some of generation.py. My python is novice-level, but happy to take a stab at this if you'd be interested in collabing on it / it seems within the scope of the project.

error: fastBPE/fastBPE.cpp: no such file or directory

Hi,
Not sure the reason I'm having problems installing fastBPE following your exact steps with google compute engine.
I already tried it on paperspace notebook (v100) and fastBPE installed fine as far as I recall (however the interactive console didn't load, unsure why).

Anyway, I will google about "error: fastBPE/fastBPE.cpp: no such file or directory"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.