mufeedvh / code2prompt Goto Github PK

Is it possible to add an argument so that any excluded files/folders/extensions are also excluded from the source tree?

I am currently working with some folders that have thousands of .npy files, or .wav, etc, and this means the source tree can get a little bit heavy!

Thanks for this great tool!

Hi, would it be possible to release a new version? Seems to have implemented quite a few features since 1.10. Looking at updating the package in nixpkgs, and this would be appreciated.

Hi, mufeedvh. Thank you for a very nice application.

I'm running into a problem right now when dealing with large code repositories. When I'm dealing with small code repositories, code2prompt works great. But when I'm dealing with large repositories, I have a token overflow problem when interacting with LLM.

So how should we deal with large code repositories? Sending only part of the source code will affect the context. Now it seems that only Gemini 1.5pro can handle about 200m tokens, which is the upper limit.

Can you perform tuning on a large code repository? Or do you have any good suggestions?

The --exclude-files and --exclude-folders flags, as documented in the code2prompt GitHub documentation, do not seem to function when used in the command-line interface (CLI). Attempting to use these flags results in an "unexpected argument" error, suggesting a possible discrepancy between the documentation and the tool's current CLI implementation.

Steps to Reproduce:

1. Attempt to use code2prompt with the --exclude-files flag:
   This results in an error: error: unexpected argument '--exclude-files' found.

2. Attempt to use code2prompt with the --exclude-folders flag:
   Similarly, this results in an error: error: unexpected argument '--exclude-folders' found.

Expected Behavior:
According to the documentation, users should be able to exclude specific files and folders from the prompt generation process by using the --exclude-files and --exclude-folders flags. The expected behavior is for code2prompt to accept these flags without error and to exclude the specified files and folders accordingly.

Actual Behavior:
The tool does not recognize the --exclude-files and --exclude-folders flags, resulting in an "unexpected argument" error message. This discrepancy between the documentation and the tool's behavior suggests a bug in the CLI implementation of these features.

Environment:
code2prompt version: 1.0.0
Operating System: (e.g., Ubuntu 20.04)
Shell: (e.g., bash, zsh)

this occurs in some setups when using SSH. Instead of panicking, it should still write the output to a file.

Thank you for a very nice application.
I am a beginner in programming, so I especially use "clean-up-code.hbs" and "document-the-code.hbs". Thanks to you, I am enjoying a very comfortable development experience.

Now to the main topic.
When '-json' is specified, json is output to the terminal.
What I am hoping for is that it will also be copied to the clipboard.
If I don't specify -json, I expect the same behaviour as if it were saved to the clipboard as text.

Am I wrong in my thinking? Ideas please.

▹▹▹▹▸ Done!                                                                                                                                                               [✓] Prompt copied to clipboard!

However, nothing had been copied to the clipboard. Has this been tested on a linux machine?

Basically, I am a Mac developer. However, I sometimes develop on Ubuntu with Windows WSL2 due to circumstances.

I can use Mac without any problem, but in WSL2 environment, the following commands do not work as expected.

1.code2prompt /home/USER/PJ --exclude="**/package-lock.json"

The purpose is to exclude package-lock.json existing in PJ/server, PJ/client, etc.
However, the output result includes the contents of package-lock.json.
Is there a way to properly use --exclude?

2. code2prompt /home/USER/PJ

I am expecting the output results to be saved to the clipboard as a result of running this command, but sometimes they are not.
So, I use the following command every time.
code2prompt /home/USER/PJ --ouput /home/USER/PJ/output.txt.

It accomplishes the purpose, but it is better to be saved to the clipboard.

Can you give me some advice on how to solve this problem?

P.S.
Very nice app, can't develop anymore without it! I look forward to more in the future.

Translated with DeepL.com (free version)

Hello,
thanks for creating this!
It looks super interesting.
With Gemini having 1 million tokens available. And the possibility to get 2 million soon, it will be amazing to see the amount of tokens for them.
https://ai.google.dev/gemini-api/docs/get-started/tutorial?lang=web#count-tokens

Thanks in advance for considering!
Greetings from Franconia (Germany)

Could a flag be added to skip copying the output to clipboard so code2prompt can be used on a remote without a clipboard.

@mufeedvh This is an awesome tool!

Feature request:

A --relative-paths flag truncates paths to the directory passed as an argument so that parent directories are not included in the file paths of the output.

Today if I run code2prompt projectdir when files are listed in the prompt payload the paths are displayed as $HOME/code/projectdir/somefile.txt instead of /projectdir/somefile.txt. Stripping the parent path saves tokens and keep details of my filesystem and username private.

.gitignore contains /node_modules, .env, etc.
I want to exclude /node_modules as is, but include .env in the output.

For example, to ensure that Claude and ChatGPT understand when .env is present, as shown below.

• root/.env
• root/client/.env
• root/server/.env

My rule of thumb is that strictly directing the location of files when interacting with AI makes development go very smoothly.

So, you need to read .env only when running code2prompt.
Is there a good way to do this?

Hello,

first of all, a cool project!

Larger codebases often significantly exceed even the largest context windows available these days, while offline LLMs are even more troublesome in this regard.

It could be useful to implement an indexing feature, that would not generate a single prompt from the codebase, but instead output multiple smaller prompts containing max. N tokens, with the purpose of creating some kind of code abstraction. This abstract could be afterwards used together with just a single file of code where modifications should be made.

I don't use LLMs for coding very frequently, but this seems like the only plausible approach for fitting large codebases into LLLMs. Have you made any considerations/experiments with this approach and possible implementation into code2prompt?

Hey @mufeedvh,

Thanks for involving yourself into this nice project !

Could you please consider providing precompiled binaries through GitHub Actions ?

Cheers

Description

I encountered a panic when running code2prompt with the following command:
RUST_BACKTRACE=full code2prompt .

Error Message

RUST_BACKTRACE=full code2prompt .
▹▹▹▹▸ Done!                                                                                                                                                                     2024-03-18 19:38:56.031 code2prompt[56653:3892444] -[NSPasteboard _setData:forType:index:usesPboardTypes:] returns false. Type: public.utf8-plain-text, index: 0, object class: __NSCFString.
thread 'main' panicked at /Users/slava/.cargo/registry/src/index.crates.io-6f17d22bba15001f/code2prompt-1.0.0/src/main.rs:142:50:
Failed to copy output to clipboard: "NSPasteboard#writeObjects: returned false"
stack backtrace:
   0:        0x10ec08fed - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hbd71adb7a72f4105
   1:        0x10ea96323 - core::fmt::write::h30346430340bc336
   2:        0x10ec06962 - std::io::Write::write_fmt::heb3d6316c565d5b1
   3:        0x10ec08d79 - std::sys_common::backtrace::print::h67e51ff2e3d5cfbd
   4:        0x10ec0a0a5 - std::panicking::default_hook::{{closure}}::hc666c9a55318d1f1
   5:        0x10ec09e0c - std::panicking::default_hook::hf980b1da49948523
   6:        0x10ec0a73f - std::panicking::rust_panic_with_hook::h683bce980186bbbe
   7:        0x10ec0a4d4 - std::panicking::begin_panic_handler::{{closure}}::ha6dbd11ba0ec8af1
   8:        0x10ec094e9 - std::sys_common::backtrace::__rust_end_short_backtrace::h889430bddc786c98
   9:        0x10ec0a232 - _rust_begin_unwind
  10:        0x10ec287c5 - core::panicking::panic_fmt::hff768cef35397791
  11:        0x10ec28ca5 - core::result::unwrap_failed::h951d84d71b0bada2
  12:        0x10ea8804a - code2prompt::main::h2d635b437be76536
  13:        0x10ea75306 - std::sys_common::backtrace::__rust_begin_short_backtrace::hddd69f0893274fa7
  14:        0x10ea7544c - std::rt::lang_start::{{closure}}::h0f70ef5c3efad0b4
  15:        0x10ec000cd - std::rt::lang_start_internal::hba631f1493ca29ca
  16:        0x10ea8a82c - _main
[1]    56653 abort      RUST_BACKTRACE=full code2prompt

Steps to Reproduce

1. Run the command: RUST_BACKTRACE=full code2prompt .
2. Observe the panic and stack trace.

Additional Context

• Operating System: macOS Monterey v12.7.2 (21G1974)
• Rust Version: rustc 1.76.0 (07dca489a 2024-02-04)
• code2prompt Version: v1.0.0

Expected Behavior

The command should execute without panicking.

Actual Behavior

The command panics with the error message: "Failed to copy output to clipboard: "NSPasteboard#writeObjects: returned false""

I want to exclude files and directories listed in .gitignore

node_modules/
venv/

I've specified them in .gitignore as above, but code2prompt reads them without excluding them.

Should I specify them directly using --exclude?

In that case, should I specify --exclude /**/node_modules/, /**/venv/?

I look forward to your reply.

Can I just add specific files like code2prompt add file1,file2? This feature would be great.