Georgi Gerganov d00cbea63c server : host-memory prompt caching (#16391) ... * minor : code style * server : fix prompt similarity calculation * server : initial host-memory prompt caching * cont * server : refactor * cont * cont : make the server task of the slot const * cont : minor [no ci] * server : cache prompts and checkpoints only for completion tasks * server : improve prompt caching logic * cont : fix check for number of cached prompts [no ci] * server : improve caching logic, add -cram CLI arg * server : print prompt mismatch info * cont : better naming [no ci] * server : improve prompt cache loading logic * server : add option to debug the slot contents (#16482) * server : add option to debug the slot contents * Update tools/server/server.cpp --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co> * server : add option to disable prompt cache --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>		2025-10-09 18:54:51 +03:00
..
arg.cpp	server : host-memory prompt caching (#16391)	2025-10-09 18:54:51 +03:00
arg.h
base64.hpp
build-info.cpp.in
chat-parser.cpp	refactor: centralize CoT parsing in backend for streaming mode (#16394)	2025-10-08 23:18:41 +03:00
chat-parser.h	model : Apertus model implementation (#15852)	2025-10-02 20:43:22 +03:00
chat.cpp	refactor: centralize CoT parsing in backend for streaming mode (#16394)	2025-10-08 23:18:41 +03:00
chat.h	server : host-memory prompt caching (#16391)	2025-10-09 18:54:51 +03:00
CMakeLists.txt
common.cpp	llama : add --no-host to disable host buffers (#16310)	2025-10-06 19:55:53 +02:00
common.h	server : host-memory prompt caching (#16391)	2025-10-09 18:54:51 +03:00
console.cpp
console.h
http.h
json-partial.cpp
json-partial.h
json-schema-to-grammar.cpp
json-schema-to-grammar.h
llguidance.cpp
log.cpp
log.h
ngram-cache.cpp
ngram-cache.h
regex-partial.cpp
regex-partial.h
sampling.cpp
sampling.h
speculative.cpp
speculative.h