llama_cpp_for_radxa_dragon_.../examples
slaren 2bf8d0f7c4
backend : offload large batches to GPU (#6083)
* backend : offload large batches to GPU

* fix hip

* code cleanup

* fix CUDA split buffers

* Update ggml-backend-impl.h

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* cuda : fix memset without set_device

* imatrix : remove sched affix from weight names

* sched : add a new split if the current one has too many inputs
reduce max inputs per split
more cleanup

* update backends

ggml-ci

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2024-03-18 11:03:04 +01:00
..
baby-llama
batched
batched-bench
batched.swift
beam-search
benchmark
convert-llama2c-to-ggml
embedding
export-lora
finetune
gguf
gritlm
imatrix
infill
jeopardy
llama-bench
llama.android
llama.swiftui
llava
lookahead
lookup
main
main-cmake-pkg
parallel
passkey
perplexity
quantize
quantize-stats
save-load-state
server
simple
speculative
sycl
tokenize
train-text-from-scratch
alpaca.sh
base-translate.sh
chat-13B.bat
chat-13B.sh
chat-persistent.sh
chat-vicuna.sh
chat.sh
CMakeLists.txt
gpt4all.sh
json-schema-to-grammar.py
llama.vim
llama2-13b.sh
llama2.sh
llm.vim
make-ggml.py
Miku.sh
pydantic-models-to-grammar-examples.py
pydantic_models_to_grammar.py
reason-act.sh
server-embd.py
server-llama2-13B.sh