Giter Club home page Giter Club logo

Comments (2)

pseudotensor avatar pseudotensor commented on July 19, 2024

LLM setup things can be controlled:

h2ogpt/src/gen.py

Lines 2151 to 2171 in 1d126ec

other_model_state_defaults = dict(load_8bit=load_8bit, load_4bit=load_4bit, low_bit_mode=low_bit_mode,
load_half=load_half, use_flash_attention_2=use_flash_attention_2,
load_gptq=load_gptq, load_awq=load_awq, load_exllama=load_exllama,
use_safetensors=use_safetensors,
revision=revision, use_gpu_id=use_gpu_id, gpu_id=gpu_id,
compile_model=compile_model,
use_cache=use_cache,
llamacpp_dict=llamacpp_dict,
rope_scaling=rope_scaling,
max_seq_len=max_seq_len,
max_output_seq_len=max_output_seq_len,
exllama_dict=exllama_dict,
gptq_dict=gptq_dict,
attention_sinks=attention_sinks,
sink_dict=sink_dict,
truncation_generation=truncation_generation,
hf_model_dict=hf_model_dict,
force_seq2seq_type=force_seq2seq_type,
force_t5_type=force_t5_type,
trust_remote_code=trust_remote_code,
)

as well as prompt_type, prompt_dict, visible_models, h2ogpt_key

But generic LLM usage things are only controlled by the expert settings in UI or defaults via CLI for all LLMs.

However, I pushed changes so the CLI model_lock can control all the other things too.

h2ogpt/src/evaluate_params.py

Lines 18 to 108 in 8e75086

gen_hyper0 = [
"num_beams",
"max_new_tokens",
"min_new_tokens",
"early_stopping",
"max_time",
"repetition_penalty",
"num_return_sequences",
"do_sample",
"seed",
]
gen_hyper = ["temperature", "top_p", "top_k", "penalty_alpha"] + gen_hyper0
reader_names = [
"image_audio_loaders",
"pdf_loaders",
"url_loaders",
"jq_schema",
"extract_frames",
"llava_prompt",
]
eval_func_param_names = (
["instruction", "iinput", "context", "stream_output", "prompt_type", "prompt_dict"]
+ gen_hyper
+ [
"chat",
"instruction_nochat",
"iinput_nochat",
"langchain_mode",
"add_chat_history_to_context",
"langchain_action",
"langchain_agents",
"top_k_docs",
"chunk",
"chunk_size",
"document_subset",
"document_choice",
"document_source_substrings",
"document_source_substrings_op",
"document_content_substrings",
"document_content_substrings_op",
"pre_prompt_query",
"prompt_query",
"pre_prompt_summary",
"prompt_summary",
"hyde_llm_prompt",
"user_prompt_for_fake_system_prompt",
"json_object_prompt",
"json_object_prompt_simpler",
"json_code_prompt",
"json_code_prompt_if_no_schema",
"json_schema_instruction",
"system_prompt",
]
+ reader_names
+ [
"visible_models",
"visible_image_models",
"h2ogpt_key",
"add_search_to_context",
"chat_conversation",
"text_context_list",
"docs_ordering_type",
"min_max_new_tokens",
"max_input_tokens",
"max_total_input_tokens",
"docs_token_handling",
"docs_joiner",
"hyde_level",
"hyde_template",
"hyde_show_only_final",
"doc_json_mode",
"metadata_in_context",
"chatbot_role",
"speaker",
"tts_language",
"tts_speed",
"image_file",
"image_control",
"response_format",
"guided_json",
"guided_regex",
"guided_choice",
"guided_grammar",
"guided_whitespace_pattern",
]
)

from h2ogpt.

chengchu88 avatar chengchu88 commented on July 19, 2024

Thank you for the detailed explanation, truly appreciate it.

from h2ogpt.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.