Giter Club home page Giter Club logo

chatgpt-history-export-to-md's Introduction

Convoviz ๐Ÿ“Š: Visualize your entire ChatGPT data !

Convert your ChatGPT history into well-formatted Markdown files. Additionally, visualize your data with word clouds ๐Ÿ”กโ˜๏ธ, view your prompt history graphs ๐Ÿ“ˆ, and access all your custom instructions ๐Ÿค– in a single location.

GitHub last commit GitHub issues

Features

  • YAML Headers: Optional and included by default.
  • Track message versions: prompt/response edits included.
  • Code Interpreter: Environment code blocks and execution results.
  • Data Visualizations: Word clouds, graphs, and more.
  • Custom Instructions: All your custom instructions in one JSON file.

See examples here.

How to Use ๐Ÿ“–

1. Export Your ChatGPT Data ๐Ÿ—‚

  • Sign in at chat.openai.com.
  • Navigate: Profile Name (bottom left) -> Settings -> Data controls -> Export -> Confirm export.
  • Await email from OpenAI and download the .zip file.

2. Install the tool ๐Ÿ› 

Open a terminal or command prompt and run:

pip install convoviz

to install the package.

3. Run the Script ๐Ÿƒโ€โ™‚๏ธ

With the package installed, run the following command in your terminal:

python -m convoviz

Next, follow the instructions displayed and choose your desired options, the script will handle the rest.

4. Check the Output ๐ŸŽ‰

And that's it! After running the script, head over to the output folder to see your nice word clouds, graphs, and neatly formatted Markdown files. Enjoy !

Tweet Your Discoveries:

Tweet

How to add new conversations โž•

See How to use the JS script for instructions on how to download new conversations.

Share Your Feedback! ๐Ÿ’Œ

I hope you find this tool useful. I'm continuously looking to improve on this, but I need your help for that.

Whether you're a tech wizard or you're new to all this (especially if you're new to all this), I'd love to hear about your journey with the tool. Found a quirk? Have a suggestion? Or just want to send some good vibes? I'm all ears!

Here's how you can share your thoughts:

  1. GitHub Issues: For more specific feedback or if you've stumbled upon a bug, please open an issue. This helps me track and address them effectively.

  2. GitHub Discussions: If you just want to share your general experience, have a suggestion, or maybe a cool idea for a new feature, jump into the discussions page. It's a more casual space where we can chat.

And if you've had a great experience, consider giving the project a star โญ. It keeps me motivated and helps others discover it!

Thank you for being awesome! ๐ŸŒŸ

Notes

This project requires Python 3.8.7 or higher if on Windows. See known issue

This is just a small thing I coded to help me see my convos in beautiful markdown, in Obsidian (my go-to note-taking app).

I wasn't a fan of the clunky, and sometimes paid, browser extensions.

It was also a great opportunity to learn more about Python and type annotations. I had mypy, pyright, and ruff all on strict mode, 'twas fun.

It also works as package, so you can import it in your own projects, and use the models and functions as you wish. I need to add more documentation for that tho. Feel free to reach out if you need help.

I'm working on automating it to add new conversations and updating old ones. Had some luck with a JavaScript bookmarklet, still ironing it out tho. Shouldn't take long.

for an older version with zero dependencies, see https://github.com/mohamed-chs/chatgpt-history-export-to-md/tree/fe13a701fe8653c9f946b1e12979ce3bfe7104b8.

chatgpt-history-export-to-md's People

Contributors

mohamed-chs avatar turnboughsg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chatgpt-history-export-to-md's Issues

Crash on non titled chat

Describe the bug
When there is no title, the script crashes.

TLDK:
fix is check if none, if none add title.:
src\utils.py
def sanitize_title(title: str) -> str:
"""Sanitize the title by replacing disallowed characters with '-'.

Args:
    title (str): The title to sanitize.

Returns:
    str: The sanitized title.
"""
if title is None:
    title = "Untitled"  # or any other default value

sanitized_title: str = DISALLOWED_CHARS_PATTERN.sub("-", title.strip())
return sanitized_title

Invalid input in the very first step

Describe the bug
Invalid input but path is correct

To Reproduce

  1. python -m convoviz /Users/wang/Downloads/20231213.zip
  2. hit enter and it says invalid input

Expected behavior
continue on next configuration

Screenshots
Screenshot 2023-12-13 at 19 31 14

OS

  • OS: MacOS Sonoma 14.1.1
  • Hardware: MBP m1
  • python: 3.10.9

`union` operand not supported by python `3.9`

Describe the bug
The python version used in pyproject.toml

doesn't support union type expression with | operand like here

since this is supported since python 3.10

To Reproduce
Steps to reproduce the behavior:

  1. create a new venv
  2. install convoviz
  3. run convoviz
  4. see error
Traceback (most recent call last):
  File "$HOME/.pyenv/versions/3.9.18/lib/python3.9/runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "$HOME/.pyenv/versions/3.9.18/lib/python3.9/runpy.py", line 147, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "$HOME/.pyenv/versions/3.9.18/lib/python3.9/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "$VENV/lib/python3.9/site-packages/convoviz/__init__.py", line 3, in <module>
    from . import configuration, data_analysis, long_runs, models, utils
  File "$VENV/lib/python3.9/site-packages/convoviz/configuration.py", line 18, in <module>
    from .models import Conversation, Message
  File "$VENV/lib/python3.9/site-packages/convoviz/models/__init__.py", line 3, in <module>
    from ._conversation import Conversation
  File "$VENV/lib/python3.9/site-packages/convoviz/models/_conversation.py", line 26, in <module>
    from ._node import Node
  File "$VENV/lib/python3.9/site-packages/convoviz/models/_node.py", line 15, in <module>
    from ._message import Message  # noqa: TCH001
  File "$VENV/lib/python3.9/site-packages/convoviz/models/_message.py", line 20, in <module>
    class MessageAuthor(BaseModel):
  File "$VENV/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 181, in __new__
    set_model_fields(cls, bases, config_wrapper, types_namespace)
  File "$VENV/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 426, in set_model_fields
    fields, class_vars = collect_model_fields(cls, bases, config_wrapper, types_namespace, typevars_map=typevars_map)
  File "$VENV/lib/python3.9/site-packages/pydantic/_internal/_fields.py", line 120, in collect_model_fields
    type_hints = get_cls_type_hints_lenient(cls, types_namespace)
  File "$VENV/lib/python3.9/site-packages/pydantic/_internal/_typing_extra.py", line 212, in get_cls_type_hints_lenient
    hints[name] = eval_type_lenient(value, globalns, localns)
  File "$VENV/lib/python3.9/site-packages/pydantic/_internal/_typing_extra.py", line 224, in eval_type_lenient
    return typing._eval_type(value, globalns, localns)  # type: ignore
  File "$HOME/.pyenv/versions/3.9.18/lib/python3.9/typing.py", line 292, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "$HOME/.pyenv/versions/3.9.18/lib/python3.9/typing.py", line 554, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Expected behavior
upgrade python version on pyproject.toml or change code to be flexible with 3.9

OS

  • OS: Linux, debian-based, popOS

Convoviz errors out on chats containing DALL-E images.

Convoviz errors out when it encountered a sole chat I had containing images from DALL-E.
Once I deleted that chat, it completed it's conversions.

Traceback (most recent call last):
File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Program Files\Python310\lib\site-packages\convoviz_main
.py", line 5, in
main()
File "C:\Program Files\Python310\lib\site-packages\convoviz\cli.py", line 38, in main
entire_collection = ConversationSet.from_zip(user.configs["zip_filepath"])
File "C:\Program Files\Python310\lib\site-packages\convoviz\models_conversation_set.py", line 55, in from_zip
return cls.from_json(convos_path)
File "C:\Program Files\Python310\lib\site-packages\convoviz\models_conversation_set.py", line 47, in from_json
return cls(array=loads(file.read()))
File "C:\Program Files\Python310\lib\site-packages\pydantic\main.py", line 164, in init
pydantic_self.pydantic_validator.validate_python(data, self_instance=pydantic_self)
pydantic_core._pydantic_core.ValidationError: 45 validation errors for ConversationSet
array.1.mapping.4d7b02ae-cd9e-47de-82d3-4981ecffc4e1.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type

Getting requirements to build wheel ... error

Describe the bug
error when installing

Expected behavior
installing the program
Screenshots
File "{MYFILEDIRECTORY}\Local\Programs\Python\Python312\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "{MYFILEDIRECTORY}\Local\Programs\Python\Python312\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "{MYFILEDIRECTORY}\Local\Programs\Python\Python312\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "{MYFILEDIRECTORY}\Local\Temp\pip-build-env-hz122s8l\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "{MYFILEDIRECTORY}\Local\Temp\pip-build-env-hz122s8l\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "{MYFILEDIRECTORY}\Local\Temp\pip-build-env-hz122s8l\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "{MYFILEDIRECTORY}\Local\Temp\pip-build-env-hz122s8l\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
exec(code, locals())
File "", line 13, in
File "{MYFILEDIRECTORY}\wordcloud_a2b0ea437707470fb309527d01325bc0\versioneer.py", line 1480, in get_version
return get_versions()["version"]
^^^^^^^^^^^^^^
File "{MYFILEDIRECTORY}\wordcloud_a2b0ea437707470fb309527d01325bc0\versioneer.py", line 1412, in get_versions
cfg = get_config_from_root(root)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "{MYFILEDIRECTORY}\wordcloud_a2b0ea437707470fb309527d01325bc0\versioneer.py", line 342, in get_config_from_root
parser = configparser.SafeConfigParser()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

OS
Windows

Additional context
This is my very first time submitting an issue - please let me know if i missed something.

, you're encountering a validation error related to the handling of image data in your conversation history.

Describe the bug
A clear and concise description of what the bug is.

โžก๏ธ https://github.com/mohamed-chs/chatgpt-history-export-to-md/issues/new/choose ๐Ÿ”—

? Enter the path to the zip file : C:\Users\mydjt\Downloads\2024_03_20_chatgpt_export_cd833544d1e9d430427877b3357ec5b40
df5d0062769dde85ac4b09bc0e74a93-2024-03-20-19-53-00.zip
? Enter the path to the output folder : C:\Users\mydjt\Documents\ChatGPT Data
? Enter the message header (#) for messages from 'system' : ### System
? Enter the message header (#) for messages from 'user' : # User
? Enter the message header (#) for messages from 'assistant' : # ChatGPT
? Enter the message header (#) for messages from 'tool' : ### Tool output
? Select the LaTeX math delimiters you want to use : default
? Select the YAML metadata headers you want to include : done (9 selections)
? Select the font you want to use for the word clouds : RobotoSlab-Thin
? Select the color theme you want to use for the word clouds : prism
? Enter custom stopwords (separated by commas) : use, file,

And we're off! ๐Ÿš€๐Ÿš€๐Ÿš€

Loading data ๐Ÿ“‚ ...

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\mydjt\Documents\GitHub\chatgpt-history-export-to-md2\convoviz_main
.py", line 5, in
main()
File "C:\Users\mydjt\Documents\GitHub\chatgpt-history-export-to-md2\convoviz\cli.py", line 38, in main
entire_collection = ConversationSet.from_zip(user.configs["zip_filepath"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mydjt\Documents\GitHub\chatgpt-history-export-to-md2\convoviz\models_conversation_set.py", line 55, in from_zip
return cls.from_json(convos_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mydjt\Documents\GitHub\chatgpt-history-export-to-md2\convoviz\models_conversation_set.py", line 47, in from_json
return cls(array=loads(file.read()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mydjt\Documents\GitHub\chatgpt-history-export-to-md2\env\Lib\site-packages\pydantic\main.py", line 171, in init
self.pydantic_validator.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 182 validation errors for ConversationSet
array.59.mapping.bbb217b1-2cd2-4c3e-a26c-0cf7dbe287e0.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... None, 'metadata': None}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.6/v/string_type
array.59.mapping.bbb21ce6-538c-46a9-b944-19711e2a3611.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... None, 'metadata': None}, input_type=dict]

For further information visit https://errors.pydantic.dev/2.6/v/string_type

array.4333.mapping.5bc365ac-6368-491f-9ad2-3bdd43b40ae8.message.content.parts.3
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.6/v/string_type

To Reproduce
Steps to reproduce the behavior:
1.Follow directions. See above.

Expected behavior
Normal output

Screenshots
If applicable, add screenshots to help explain your problem.

https://share.zight.com/RBukD70Y
https://share.zight.com/ApuPdnyD

OS

  • OS: Windows 11

Additional context
Add any other context about the problem here.

I copied and pasted everything into chatgpt and got this as a reply. Any way to fix this?

It seems like you're trying to use the ChatGPT Data Visualizer (Convoviz) to visualize your ChatGPT conversation data. However, you're encountering a validation error related to the handling of image data in your conversation history. The error suggests that the tool is expecting a string input, but it's receiving a dictionary containing image data instead.

This might be due to a limitation in the current version of Convoviz, which may not fully support conversations that include image data. You might want to check if there are updates to Convoviz that address this issue, or you could try removing the image data from your conversation history before running the tool.

python main.py causes error in configs_dict = get_user_configs()

Describe the bug
I use the step-by-step instructions from https://github.com/mohamed-chs/chatgpt-history-export-to-md

To Reproduce
Steps to reproduce the behavior:

  1. git clone https://github.com/mohamed-chs/chatgpt-history-export-to-md.git
  2. cd chatgpt-history-export-to-md
  3. python setup.py
  4. source .venv/bin/activate
  5. python main.py

(same result when using python3)

Expected behavior
The application should ask where the downloaded data is (or ?)

Output

Traceback (most recent call last):
File "/Users/MYUSERNAME/temp/chatgpt-history-export-to-md/main.py", line 122, in
main()
File "/Users/MYUSERNAME/temp/chatgpt-history-export-to-md/main.py", line 38, in main
configs_dict = get_user_configs()
^^^^^^^^^^^^^^^^^^
File "/Users/MYUSERNAME/temp/chatgpt-history-export-to-md/controllers/configuration.py", line 22, in get_user_configs
default_configs["zip_file"] = get_openai_zip_filepath()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/MYUSERNAME/temp/chatgpt-history-export-to-md/utils/utils.py", line 93, in get_openai_zip_filepath
default_zip_filepath: Path = max(zip_files, key=lambda x: x.stat().st_ctime)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: max() arg is an empty sequence

OS

  • OS: macOS 14.0
  • Python 3.11.6

Additional context
Add any other context about the problem here.

[low priority] tests failing with ModuleNotFoundError

Describe the bug
Build failing here with three errors all caused by ModuleNotFoundError: No module named 'questionary'. E.g.:

____________________ ERROR collecting tests/test_message.py ____________________
ImportError while importing test module '/home/runner/work/chatgpt-history-export-to-md/chatgpt-history-export-to-md/tests/test_message.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_message.py:8: in <module>
    from convoviz.models import Message
convoviz/__init__.py:3: in <module>
    from . import configuration, data_analysis, long_runs, models, utils
convoviz/configuration.py:5: in <module>
    from questionary import (
E   ModuleNotFoundError: No module named 'questionary'

To Reproduce
Steps to reproduce the behavior:

  1. Visit build page and view error.

Potential Fix
/convoviz/configuration.py explicitly imports questionable, which is used in a Message in the CLI. You may have to explicitly import necessary questionable packages in /tests somewhere.

ValueError in data_analysis.py on the call of .generate() of WordCloud

About
Wanted to convert .zip to a .md files

To Reproduce
All default inputs are given by pressing just "Return" for all asked inputs like shown below

? Enter the path to the zip file : /home/xd/Downloads/b455353cfb9d91342741e3be5caf32bac5bfdc3200a5c2ed274d8b606825027
6-2024-02-17-18-37-08.zip
? Enter the path to the output folder : /home/xd/Documents/ChatGPT Data
? Enter the message header (#) for messages from 'system' : ### System
? Enter the message header (#) for messages from 'user' : # Me
? Enter the message header (#) for messages from 'assistant' : # ChatGPT
? Enter the message header (#) for messages from 'tool' : ### Tool output
? Select the LaTeX math delimiters you want to use : default
? Select the YAML metadata headers you want to include : done (9 selections)
? Select the font you want to use for the word clouds : RobotoSlab-Thin
? Select the color theme you want to use for the word clouds : prism
? Enter custom stopwords (separated by commas) : use, file

Expected behavior

Done โœ… ! Check the output ๐Ÿ“ˆ here : file:///home/xd/Documents/ChatGPT%20Data/Graphs ๐Ÿ”—

(more graphs ๐Ÿ“ˆ will be added in the future ...)

Creating weekly wordclouds ๐Ÿ”กโ˜๏ธ :   0%|                                                       | 0/41 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/xd/.local/lib/python3.10/site-packages/convoviz/__main__.py", line 5, in <module>
    main()
  File "/home/xd/.local/lib/python3.10/site-packages/convoviz/cli.py", line 76, in main
    generate_wordclouds(
  File "/home/xd/.local/lib/python3.10/site-packages/convoviz/long_runs.py", line 70, in generate_wordclouds
    week_groups[week].wordcloud(**kwargs).save(
  File "/home/xd/.local/lib/python3.10/site-packages/convoviz/models/_conversation_set.py", line 150, in wordcloud
    return generate_wordcloud(text, **kwargs)
  File "/home/xd/.local/lib/python3.10/site-packages/convoviz/data_analysis.py", line 117, in generate_wordcloud
    ).generate(text)
  File "/home/xd/.local/lib/python3.10/site-packages/wordcloud/wordcloud.py", line 642, in generate
    return self.generate_from_text(text)
  File "/home/xd/.local/lib/python3.10/site-packages/wordcloud/wordcloud.py", line 624, in generate_from_text
    self.generate_from_frequencies(words)
  File "/home/xd/.local/lib/python3.10/site-packages/wordcloud/wordcloud.py", line 453, in generate_from_frequencies
    self.generate_from_frequencies(dict(frequencies[:2]),
  File "/home/xd/.local/lib/python3.10/site-packages/wordcloud/wordcloud.py", line 511, in generate_from_frequencies
    box_size = draw.textbbox((0, 0), word, font=transposed_font, anchor="lt")
  File "/usr/lib/python3/dist-packages/PIL/ImageDraw.py", line 671, in textbbox
    raise ValueError("Only supported for TrueType fonts")
ValueError: Only supported for TrueType fonts

Screenshots
Screenshot from 2024-02-18 00-28-40

OS

  • linux 6.5.0-17-generic, Ubuntu 22.04.1, python3.10.12

No valid content found in message

Describe the bug
after use yb66's code. I meet another problem.

OS

  • OS: MacOS

Additional context
image

python -m convoviz
Welcome to ChatGPT Data Visualizer โœจ๐Ÿ“Š!

Follow the instructions in the command line.

Press 'ENTER' to select the default options.

If you encounter any issues ๐Ÿ›, please report ๐Ÿšจ them here:

โžก๏ธ https://github.com/mohamed-chs/chatgpt-history-export-to-md/issues/new/choose ๐Ÿ”—


? Enter the path to the zip file : /Users/apple/Downloads/75749080309a59310137b8f011a91137c34a6e18fc2683dfe935b8d2458b4d9d-2024-03-26-08-06-14/75749080309a59310137b8f011a91137c34a6e18fc2683dfe935b8d2458b4
d9d-2024-03-26-08-06-14.zip
? Enter the path to the output folder : /Users/apple/Documents/ChatGPT Data
? Enter the message header (#) for messages from 'system' : ### System
? Enter the message header (#) for messages from 'user' : # Me
? Enter the message header (#) for messages from 'assistant' : # ChatGPT
? Enter the message header (#) for messages from 'tool' : ### Tool output
? Select the LaTeX math delimiters you want to use : default
? Select the YAML metadata headers you want to include : done (9 selections)
? Select the font you want to use for the word clouds : RobotoSlab-Thin

? Select the color theme you want to use for the word clouds : prism
? Enter custom stopwords (separated by commas) : use, file,


And we're off! ๐Ÿš€๐Ÿš€๐Ÿš€

Loading data ๐Ÿ“‚ ...

Writing Markdown ๐Ÿ“„ files:   0%|โ–                                                                                                                                          | 1/619 [00:00<00:03, 201.06it/s]
Traceback (most recent call last):
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/site-packages/convoviz/__main__.py", line 5, in <module>
    main()
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/site-packages/convoviz/cli.py", line 56, in main
    entire_collection.save(markdown_folder, progress_bar=True)
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/site-packages/convoviz/models/_conversation_set.py", line 80, in save
    conversation.save(filepath)
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/site-packages/convoviz/models/_conversation.py", line 214, in save
    file.write(self.markdown)
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/site-packages/convoviz/models/_conversation.py", line 192, in markdown
    content = close_code_blocks(node.message.text)
  File "/usr/local/anaconda3/envs/gpt4/lib/python3.10/site-packages/convoviz/models/_message.py", line 90, in text
    raise ValueError(err_msg)
ValueError: No valid content found in message: 7a6cab36-b71a-47e9-a7cb-c7d774776a36

error File "<frozen runpy>", line 198, in _run_module_as_main

? Enter the path to the zip file : C:\Users\djtlm\Documents\GitHub\chatgpt-history-export-to-md\openai-export cd833544d
1e9d430427877b3357ec5b40df5d0062769dde85ac4b09bc0e74a93-2023-11-12-21-19-05.zip
? Enter the path to the output folder : C:\Users\djtlm\Documents\GitHub\chatgpt-history-export-to-md\output
? Enter the message header (#) for messages from 'system' : ### System
? Enter the message header (#) for messages from 'user' : # Me
? Enter the message header (#) for messages from 'assistant' : # ChatGPT
? Enter the message header (#) for messages from 'tool' : ### Tool output
? Select the LaTeX math delimiters you want to use : default
? Select the YAML metadata headers you want to include : done (9 selections)
? Select the font you want to use for the word clouds : RobotoSlab-Thin
? Select the color theme you want to use for the word clouds : prism
? Enter custom stopwords (separated by commas) : use, file,

And we're off! ๐Ÿš€๐Ÿš€๐Ÿš€

Loading data ๐Ÿ“‚ ...

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Users\djtlm\AppData\Local\Programs\Python\Python311\Lib\site-packages\convoviz_main
.py", line 5, in
main()
File "C:\Users\djtlm\AppData\Local\Programs\Python\Python311\Lib\site-packages\convoviz\cli.py", line 38, in main
entire_collection = ConversationSet.from_zip(user.configs["zip_filepath"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\djtlm\AppData\Local\Programs\Python\Python311\Lib\site-packages\convoviz\models_conversation_set.py", line 55, in from_zip
return cls.from_json(convos_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\djtlm\AppData\Local\Programs\Python\Python311\Lib\site-packages\convoviz\models_conversation_set.py", line 47, in from_json
return cls(array=loads(file.read()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\djtlm\AppData\Local\Programs\Python\Python311\Lib\site-packages\pydantic\main.py", line 164, in init
pydantic_self.pydantic_validator.validate_python(data, self_instance=pydantic_self)
pydantic_core._pydantic_core.ValidationError: 50 validation errors for ConversationSet
array.28.mapping.b5afe8ed-de76-4dd5-a0cf-04e7ab3845a9.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.b5afe8ed-de76-4dd5-a0cf-04e7ab3845a9.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.87ef3fb6-ba31-4233-ba3d-e3eece62f9e5.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.87ef3fb6-ba31-4233-ba3d-e3eece62f9e5.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.aa5a68eb-ce8d-4e56-802d-38b43eba4a4f.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.aa5a68eb-ce8d-4e56-802d-38b43eba4a4f.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.75d9db25-b16f-4987-9aa2-f94dc515d4f8.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.75d9db25-b16f-4987-9aa2-f94dc515d4f8.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.a971f12b-6607-48dc-a624-242d80ca7122.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.a971f12b-6607-48dc-a624-242d80ca7122.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.ff5b1d57-9293-458c-8845-7dee437142a7.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.28.mapping.ff5b1d57-9293-458c-8845-7dee437142a7.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.29.mapping.aaa20e83-8756-4729-a9ba-9c45b0961f63.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... None, 'metadata': None}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.107.mapping.aaa2442a-8d2f-42b0-b932-c2db3887f7d8.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... None, 'metadata': None}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.288.mapping.46e1e809-8d4b-45cd-9617-c183a017f7e1.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.288.mapping.46e1e809-8d4b-45cd-9617-c183a017f7e1.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.288.mapping.44778564-808a-4900-a0fb-0b9be15e4d64.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.288.mapping.44778564-808a-4900-a0fb-0b9be15e4d64.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.340.mapping.3358dd23-e3c0-4a9e-a217-51ba0584d2eb.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.340.mapping.3358dd23-e3c0-4a9e-a217-51ba0584d2eb.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.341.mapping.88559ce0-f97f-44fa-a630-ca9fedf38edc.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.341.mapping.88559ce0-f97f-44fa-a630-ca9fedf38edc.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.342.mapping.7a23b8f1-f01c-4b33-ab50-baa58647c7a5.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.3d38b27b-794b-4a0b-9b3e-6e5c3106a510.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.3d38b27b-794b-4a0b-9b3e-6e5c3106a510.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.8364c1b8-7ade-4071-9a84-380d232f0e93.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.8364c1b8-7ade-4071-9a84-380d232f0e93.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.84d7d509-faa2-4ad8-82e5-10ffaa162f9b.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.84d7d509-faa2-4ad8-82e5-10ffaa162f9b.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.957392d5-a02d-4dae-8dab-fd8c0b960fd7.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.957392d5-a02d-4dae-8dab-fd8c0b960fd7.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.b7842137-42c9-4c11-a119-065c7a8e6851.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.b7842137-42c9-4c11-a119-065c7a8e6851.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.345.mapping.e2fe2f68-bb3e-4eae-b430-06f101d347be.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.bc9e0305-a1af-4399-a6db-38f56ec4519b.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.bc9e0305-a1af-4399-a6db-38f56ec4519b.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.bc9e0305-a1af-4399-a6db-38f56ec4519b.message.content.parts.2
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.bc9e0305-a1af-4399-a6db-38f56ec4519b.message.content.parts.3
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.ee61c5c2-5122-4d0d-bb1c-75e126865117.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.ee61c5c2-5122-4d0d-bb1c-75e126865117.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.ee61c5c2-5122-4d0d-bb1c-75e126865117.message.content.parts.2
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.ee61c5c2-5122-4d0d-bb1c-75e126865117.message.content.parts.3
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.82f4daf3-332e-4434-8fbe-356a6b53ff78.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.82f4daf3-332e-4434-8fbe-356a6b53ff78.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.82f4daf3-332e-4434-8fbe-356a6b53ff78.message.content.parts.2
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.82f4daf3-332e-4434-8fbe-356a6b53ff78.message.content.parts.3
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.5bc365ac-6368-491f-9ad2-3bdd43b40ae8.message.content.parts.0
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.5bc365ac-6368-491f-9ad2-3bdd43b40ae8.message.content.parts.1
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.5bc365ac-6368-491f-9ad2-3bdd43b40ae8.message.content.parts.2
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
array.492.mapping.5bc365ac-6368-491f-9ad2-3bdd43b40ae8.message.content.parts.3
Input should be a valid string [type=string_type, input_value={'content_type': 'image_a... generation metadata'}}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/string_type
PS C:\Users\djtlm\Documents\GitHub\chatgpt-history-export-to-md\openai-export cd833544d1e9d430427877b3357ec5b40df5d0062769dde85ac4b09bc0e74a93-2023-11-12-21-19-05>

ImportError: cannot import name 'Unpack' from 'typing'

Describe the bug
python -m convoviz
Traceback (most recent call last):
File "C:\ProgramData\miniforge3\lib\runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "C:\ProgramData\miniforge3\lib\runpy.py", line 146, in _get_module_details
return get_module_details(pkg_main_name, error)
File "C:\ProgramData\miniforge3\lib\runpy.py", line 110, in get_module_details
import(pkg_name)
File "C:\ProgramData\miniforge3\lib\site-packages\convoviz_init
.py", line 3, in
from . import configuration, data_analysis, long_runs, models, utils
File "C:\ProgramData\miniforge3\lib\site-packages\convoviz\configuration.py", line 18, in
from .models import Conversation, Message
File "C:\ProgramData\miniforge3\lib\site-packages\convoviz\models_init
.py", line 3, in
from ._conversation import Conversation
File "C:\ProgramData\miniforge3\lib\site-packages\convoviz\models_conversation.py", line 11, in
from typing import TYPE_CHECKING, Any, ClassVar, Unpack
ImportError: cannot import name 'Unpack' from 'typing' (C:\ProgramData\miniforge3\lib\typing.py)

python --version
Python 3.10.12

KeyError: 'is_user_system_message'

Describe the bug
I was getting a crash part way through the run

Writing Markdown ๐Ÿ“„ files: 3%|โ–ˆโ–ˆ | 12/452 [00:00<00:00, 833.60it/s] Traceback (most recent call last): File "/home/me/tools/code/public/chatgpt-history-export-to-md/main.py", line 227, in <module> main() File "/home/me/tools/code/public/chatgpt-history-export-to-md/main.py", line 103, in main conversation.save_to_dir(markdown_folder) File "/home/me/tools/code/public/chatgpt-history-export-to-md/models/conversation.py", line 329, in save_to_dir file.write(self.markdown) File "/home/me/tools/code/public/chatgpt-history-export-to-md/models/conversation.py", line 286, in markdown markdown = self.yaml_header File "/home/me/tools/code/public/chatgpt-history-export-to-md/models/conversation.py", line 256, in yaml_header "custom_instructions": self.custom_instructions, File "/home/ivan/tools/code/public/chatgpt-history-export-to-md/models/conversation.py", line 239, in custom_instructions if context_message and context_message.metadata["is_user_system_message"]: KeyError: 'is_user_system_message'

To Reproduce
I think some of my data is missing a "is_user_system_message" key?

Additional context
i updated
line 239 in conversation.py to use the get method so it doesnt fail is that key is not there.
if context_message and context_message.metadata.get("is_user_system_message", None):

TypeError in ConversationSet.from_zip: Unexpected Dictionary Instead of String

Describe the bug
When attempting to load a conversation set using Convoviz, a pydantic_core._pydantic_core.ValidationError is encountered. This error arises due to a type mismatch in the ConversationSet model, where a string is expected, but a dictionary is received in message.content.parts. This issue prevents successful data loading and processing.

To Reproduce
Steps to reproduce the behavior:

Use Convoviz to load a conversation set from a .zip file using ConversationSet.from_zip.
Encounter a ValidationError during the process.
The error details point to multiple instances in array where message.content.parts expects a string but receives a dictionary.
Expected behavior
The expected behavior is for Convoviz to successfully load the conversation set without any type mismatch errors. The ConversationSet model should correctly handle the data structure provided in the .zip file.

OS: Windows 10

Additional context
This issue seems to stem from a mismatch between the expected data format in the Pydantic model and the actual data structure being processed. It might require either adjusting the data format or modifying the Pydantic model to align with the actual data structure.

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.