use axolotl for trainning

Env

• no internet
• qlora_root c:\qlora
• gguf_root c:\gguf

Dataset 

• c:/qlora/output_dataset/instruction_dataset.parquet

PS:Axolotl

• docker run --gpus '"all"' --rm -it winglian/axolotl:main-latest

PS:Dataset

• docker cp C:\qlora\output_dataset\instruction_dataset.parquet container_name:/workspace/axolotl/examples/instruction_dataset.parquet

PS:LLM

• docker cp C:\Meta-Llama-3.1-8B container_name:/workspace/axolotl/examples/Meta-Llama-3.1-8B/

Axolotl:Qlora

• open ./examples/llama-3/qlora.yml find and modify path of llm and parquet

Axolotl:Trainning

• CUDA_VISIBLE_DEVICES="" python -m axolotl.cli.preprocess examples/llama-3/qlora.yml
• accelerate launch -m axolotl.cli.train examples/llama-3/qlora.yml

Axolotl:Test(need internet)

• accelerate launch -m axolotl.cli.inference examples/llama-3/qlora.yml --lora_model_dir="./outputs/qlora-out" --gradio

Axolotl:Merged

• python3 -m axolotl.cli.merge_lora examples/llama-3/qlora.yml --lora_model_dir="./outputs/qlora-out"

PS:Export

• docker cp container_name:/workspace/axolotl/outputs/qlora-out/merged C:\merged

GGUF

• convert to merged_f16.gguf

Ollama

• ollama run merged

unsloth

Conda

• conda install -c conda-forge xformers
• pip install xformers
• conda config --add channels conda-forge
• conda update conda

Env

• conda create --name unsloth python=3.11 pytorch-cuda=12.1 pytorch cudatoolkit -c pytorch -c nvidia -y
• conda activate unsloth
• pip install xformers

Install

• pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
• pip install --no-deps trl peft accelerate bitsandbytes