Giter Club home page Giter Club logo

transformerviz's Introduction

Transformer Visualizations

Investigations on Transformer Visualizations by the Aether Prototyping and Incubation team.

Getting Started

Development Environment Setup

  • python -m pip install --upgrade pip
  • pip install -r requirements.txt --cache-dir <path-to-pip-cache-dir> (The cache-dir parameter is optional).
  • pip install -e .
  • npm install
  • npm run build

Running

PERSPECTIVE_API_KEY=<perspective-api-key> GPT2_MODEL_VERSION=<gpt2-model-version> NUM_RETURN_SEQUENCES=<number-of-return-sequences> TRANSFORMERS_CACHE=<path-to-transformers-cache> FLASK_APP=transformerviz/server.py flask run --host 0.0.0.0 --port 5000 (The TRANSFORMERS_CACHE environment variable is optional).

  • Point your browser to port 5000 of the server on which the app is running.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

transformerviz's People

Contributors

jlema avatar microsoftopensource avatar

Stargazers

avatar avatar

Watchers

avatar avatar avatar

Forkers

nking92 test-mass-forker-org-1

transformerviz's Issues

ValueError: `top_p` has to be a float > 0 and < 1, but is 0

This is using default values on the text generation prompt. Stack trace below.

127.0.0.1 - - [25/May/2021 09:43:22] "OPTIONS /api/v1/generate_text HTTP/1.1" 200 -
[2021-05-25 09:43:22,350] ERROR in app: Exception on /api/v1/generate_text [POST]
Traceback (most recent call last):
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask\app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask_cors\extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask_compat.py", line 39, in reraise
raise value
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Users\jlema\git\transformerviz\transformerviz\server.py", line 92, in generate_text
num_beams=num_beams)
File "C:\Users\jlema\git\transformerviz\transformerviz\helpers\utils.py", line 64, in generate_text
temperature=temperature, num_beams=num_beams)
File "C:\Users\jlema\git\transformerviz\transformerviz\helpers\utils.py", line 36, in generate_sentences_hf
return_dict_in_generate=True
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\torch\autograd\grad_mode.py", line 15, in decorate_context
return func(*args, **kwargs)
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\transformers\generation_utils.py", line 963, in generate
top_k=top_k, top_p=top_p, temperature=temperature, num_beams=num_beams
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\transformers\generation_utils.py", line 541, in _get_logits_warper
warpers.append(TopPLogitsWarper(top_p=top_p, min_tokens_to_keep=(2 if num_beams > 1 else 1)))
File "c:\users\jlema\miniconda3\envs\transformerviz\lib\site-packages\transformers\generation_logits_process.py", line 185, in init
raise ValueError(f"top_p has to be a float > 0 and < 1, but is {top_p}")
ValueError: top_p has to be a float > 0 and < 1, but is 0

Sorting by Identity Attack doesn't do anything

Prompt: "Millennials have destroyed"

  1. Show score - identity attack
  2. Sort by - identity attack
    Expected - scores are sorted from 0 to 1 on identity attack
    Current - scores are not sorted, follow generation order

RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous

Traceback (most recent call last):
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask_cors\extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask_compat.py", line 39, in reraise
raise value
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Users\nick\code\transformerviz\transformerviz\server.py", line 92, in generate_text
num_beams=num_beams)
File "C:\Users\nick\code\transformerviz\transformerviz\helpers\utils.py", line 64, in generate_text
temperature=temperature, num_beams=num_beams)
File "C:\Users\nick\code\transformerviz\transformerviz\helpers\utils.py", line 36, in generate_sentences_hf
return_dict_in_generate=True
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\torch\autograd\grad_mode.py", line 15, in decorate_context
return func(*args, **kwargs)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\generation_utils.py", line 996, in generate
**model_kwargs,
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\generation_utils.py", line 1848, in beam_sample
output_hidden_states=output_hidden_states,
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\models\gpt2\modeling_gpt2.py", line 901, in forward
return_dict=return_dict,
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\models\gpt2\modeling_gpt2.py", line 627, in forward
input_ids = input_ids.view(-1, input_shape[-1])
RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous

[2021-05-25 09:32:58,071] ERROR in app: Exception on /api/v1/generate_text [POST]
Traceback (most recent call last):
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask_cors\extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask_compat.py", line 39, in reraise
raise value
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Users\nick\code\transformerviz\transformerviz\server.py", line 92, in generate_text
num_beams=num_beams)
File "C:\Users\nick\code\transformerviz\transformerviz\helpers\utils.py", line 64, in generate_text
temperature=temperature, num_beams=num_beams)
File "C:\Users\nick\code\transformerviz\transformerviz\helpers\utils.py", line 36, in generate_sentences_hf
return_dict_in_generate=True
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\torch\autograd\grad_mode.py", line 15, in decorate_context
return func(*args, **kwargs)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\generation_utils.py", line 903, in generate
top_k=top_k, top_p=top_p, temperature=temperature, num_beams=num_beams
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\generation_utils.py", line 541, in _get_logits_warper
warpers.append(TopPLogitsWarper(top_p=top_p, min_tokens_to_keep=(2 if num_beams > 1 else 1)))
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\transformers\generation_logits_process.py", line 185, in init
raise ValueError(f"top_p has to be a float > 0 and < 1, but is {top_p}")
ValueError: top_p has to be a float > 0 and < 1, but is 0

[2021-05-25 13:49:03,204] ERROR in app: Exception on /api/v1/generate_text [POST]
Traceback (most recent call last):
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask_cors\extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask_compat.py", line 39, in reraise
raise value
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Users\nick\code\transformerviz\transformerviz\server.py", line 93, in generate_text
analyzed_sentences = perspective_api_client.analyze_sentences(generated_sentences)
File "C:\Users\nick\code\transformerviz\transformerviz\helpers\utils.py", line 92, in analyze_sentences
response = self.client.comments().analyze(body=analyze_request).execute()
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\googleapiclient_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "C:\Users\nick\anaconda3\envs\transviz\lib\site-packages\googleapiclient\http.py", line 935, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=AIzaSyBwDPnFUSFpmCrAxRDL2c_VztKkvnGiywI+&alt=json returned "Unable to detect language.". Details: "Unable to detect language.">

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.