use axolotl for trainning

Env

• no internet
• qlora_root c:\qlora
• gguf_root c:\gguf

Dataset 

• c:/qlora/output_dataset/instruction_dataset.parquet

PS:Axolotl

• docker run --gpus '"all"' --rm -it winglian/axolotl:main-latest

PS:Dataset

• docker cp C:\qlora\output_dataset\instruction_dataset.parquet container_name:/workspace/axolotl/examples/instruction_dataset.parquet

PS:LLM

• docker cp C:\Meta-Llama-3.1-8B container_name:/workspace/axolotl/examples/Meta-Llama-3.1-8B/

Axolotl:Qlora

• open ./examples/llama-3/qlora.yml find and modify path of llm and parquet

Axolotl:Trainning

• CUDA_VISIBLE_DEVICES="" python -m axolotl.cli.preprocess examples/llama-3/qlora.yml
• accelerate launch -m axolotl.cli.train examples/llama-3/qlora.yml

Axolotl:Test(need internet)

• accelerate launch -m axolotl.cli.inference examples/llama-3/qlora.yml --lora_model_dir="./outputs/qlora-out" --gradio

Axolotl:Merged

• python3 -m axolotl.cli.merge_lora examples/llama-3/qlora.yml --lora_model_dir="./outputs/qlora-out"

PS:Export

• docker cp container_name:/workspace/axolotl/outputs/qlora-out/merged C:\merged

GGUF

• convert to merged_f16.gguf

Ollama

• ollama run merged