Huggingface model convert gguf

config env

• mkdir model and cd model
• conda create -n gguf
• conda activate gguf
• pip install huggingface_hub

dl model from hf:get model_id,local_dir

• create file:download.py and save
  from huggingface_hub import snapshot_download
  model_id="Qwen/Qwen2-1.5B-Instruct"
  snapshot_download(repo_id=model_id, local_dir="Qwen2-1.5B-Instruct",local_dir_use_symlinks=False, revision="main")
• python download.py

config llama.cpp

• cd ..
• git clone https://github.com/ggerganov/llama.cpp.git
• cd llama.cpp
• pip install -r requirements.txt

run python llama.cpp/convert_hf_to_gguf.py -h

convert

• cd ..
  check path for directory of model
• #full f16 or f32
  python ./llama.cpp/convert_hf_to_gguf.py ./model/qwen2_1.5b_instruct  --outtype f32 --verbose --outfile ./Qwen2-1.5B-Instruct_f32.gguf
• #q2_k,q3_k_l/q3_k_m/q3_k_s,q4_0/q4_1/q4_k_m/q4_k_s,q5_0/q5_1/q5_k_m/q5_k_s,q6_k,q8_0,fp16/f32
  python ./llama.cpp/convert_hf_to_gguf.py ./model/qwen2_1.5b_instruct  --outtype q8_0 --verbose --outfile ./Qwen2-1.5B-Instruct_q8_0.gguf

perplexica update

Check

• docker volume ls | perplexica_backend-dbstore
• docker run -it --rm -v perplexica_backend-dbstore:/data busybox ls -l /data

Backup

• docker run --rm -v perplexica_backend-dbstore:/data -v C:/backup:/backup busybox tar cvf /backup/perplexica_backend-dbstore.tar /data

Remove

• docker remove old container

Update

• goto perplexica of directory and run git pull origin master
• check config.toml and docker-compose.yaml

ReBuild

• goto perplexica of directory and docker compose up -d --build

Recovery

• docker run --rm -v perplexica_backend-dbstore:/data -v C:/backup:/backup busybox tar xvf /backup/perplexica_backend-dbstore.tar -C /data

Check

• docker run -it --rm -v perplexica_backend-dbstore:/data busybox ls -l /data

anythingllm update(update 2024.09.23)

Get-New

• docker pull mintplexlabs/anythingllm

Remove

• docker remove old container

Run

• docker run -d -p 3001:3001 --cap-add SYS_ADMIN -v C:/anythingllm/storage:/app/server/storage -v C:/anythingllm/env.txt:/app/server/.env -e STORAGE_DIR="/app/server/storage" --add-host=host.docker.internal:host-gateway --name anythingllm mintplexlabs/anythingllm

---------------------------------------

use github to pull to docker and update

• git clone https://github.com/Mintplex-Labs/anything-llm
• copy ./docker to ./anythingllm
• into ./anything-llm/anythingllm and rename .env.example to .env
• PS>docker compose up -d (--build)

change port number

• .env
  SERVER_PORT=3001
• docker-compose.yml
  ports:
  - "3001:3001"

change name

• docker-compose.yml
  name: anythingllm
  container_name: anythingllm

Chroma update

 

Backup

• docker volume ls
• docker run --rm -v chroma_chroma-data:/data -v C:/backup:/backup busybox tar cvf /backup/chroma_chroma-data.tar /data

Remove

• docker remove old container

Get-New

• get new version from https://github.com/chroma-core/chroma/releases
• docker pull ghcr.io/chroma-core/chroma:0.5.4
• docker run -p 8000:8000 ghcr.io/chroma-core/chroma:0.5.4

Recovery

• docker run --rm -v chroma_chroma-data:/data -v C:/backup:/backup busybox tar xvf /backup/chroma_chroma-data.tar -C /data
• docker run -it --rm -v chroma_chroma-data:/data busybox ls -l /data

SenseVoice

Get 

• git clone https://github.com/FunAudioLLM/SenseVoice.git

Env

• conda create -n sensevoice python=3.8
• conda activate sensevoice

Install

• cd sensevoice
• pip install -r requirements.txt

Run

• python webui.py

Change port

• demo.launch(server_port=your_portnumber,share=True,server_name='your_ip')

Chroma for AI(update 2024.09.23)

Get1

•  docker pull ghcr.io/chroma-core/chroma:0.5.4

Get2

• git clone https://github.com/chroma-core/chroma.git
• docker compose up -d

Update1

• watch version on https://github.com/chroma-core/chroma
• docker pull ghcr.io/chroma-core/chroma:version

Update2

•  goto chroma of directory
• PS>docker compose up -d --build

-----------------------------------------------

you can use github to pull to docker

• git clone https://github.com/chroma-core/chroma
• cd ./chroma and copy compose-env.windows .env
• rename docker-compose.yml to docker-compose-any.yml
• copy docker-compose.server.example.yml to docker-compose.yml
• PS>docker compose up -d (--build)

LangFlow

Github

• git clone https://github.com/langflow-ai/langflow.git

Docker

• cd langflow
• copy ./docker_example ./langflow (or your_name_for_docker)
• open and config(ChangePort) langflow/langflow  (or your_name_for_docker)/docker-compose.yml
• PowerShell>
  cd langflow/langflow  (or your_name_for_docker)
  docker compose up -d (--build)

Run

• Open http://localhost:7860/store and Click API_Key
• Login https://www.langflow.store/ and get API-Key

ChatTTS

It good than Bark

Github

• git clone https://github.com/2noise/ChatTTS.git
• cd ChatTTS

Env

• conda create -n chattts python=3.8
• conda activate chattts

Install

• pip install --upgrade -r requirements.txt
• pip install -r requirements.txt

Run

• python examples/web/webui.py

Change port, public link

• open ChatTTS\examples\web\webui.py
• find
  demo.launch()
• and add args
  server_name='0.0.0.0'
  server_port=7860
  share=True