Georgi Gerganov d00cbea63c server : host-memory prompt caching (#16391) ... * minor : code style * server : fix prompt similarity calculation * server : initial host-memory prompt caching * cont * server : refactor * cont * cont : make the server task of the slot const * cont : minor [no ci] * server : cache prompts and checkpoints only for completion tasks * server : improve prompt caching logic * cont : fix check for number of cached prompts [no ci] * server : improve caching logic, add -cram CLI arg * server : print prompt mismatch info * cont : better naming [no ci] * server : improve prompt cache loading logic * server : add option to debug the slot contents (#16482) * server : add option to debug the slot contents * Update tools/server/server.cpp --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co> * server : add option to disable prompt cache --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>		2025-10-09 18:54:51 +03:00
..
batched-bench
cvector-generator
export-lora
gguf-split
imatrix
llama-bench	llama : add --no-host to disable host buffers (#16310)	2025-10-06 19:55:53 +02:00
main
mtmd	chat : Granite Docling stopping (#16438)	2025-10-06 18:59:40 +02:00
perplexity
quantize
rpc	rpc : update documentation (#16441)	2025-10-07 06:59:13 +00:00
run	common: introduce http.h for httplib-based client (#16373)	2025-10-01 20:22:18 +03:00
server	server : host-memory prompt caching (#16391)	2025-10-09 18:54:51 +03:00
tokenize
tts	model : Apertus model implementation (#15852)	2025-10-02 20:43:22 +03:00
CMakeLists.txt