Minsoo Cheong 6d341ab6c5 speculative : implement stochastic speculative sampling (#5625) ... * (WIP) Implement stochastic speculative decoding * sample from residual distribution on draft accept failure * fix #5657: force greedy sampling with probs when temp is 0 * remove p_accept parameter * fix style * remove unused variables * add srand() in speculative.cpp * replace use of rand() with mt19937 sampling * fixes based on review (@JohannesGaessler) * fix r random generation * randomly select next sequence to verify + fix bug in memory freeing * fix bug in active_seqs sync * fix uniform int distribution initialization * remove warnings from comparison between int and size_t * check grammar in `llama_sample_probability_distribution_impl` * remove malloc code by utilizing vectors * add PR link to README		2024-03-04 20:24:00 +02:00
..
baby-llama	code : normalize enum names (#5697)	2024-02-25 12:09:09 +02:00
batched	ggml, common, examples, tests : fixed type arguments in printf (#5528)	2024-02-18 18:20:12 +02:00
batched-bench	llama : cleanup unused mmq flags (#5772)	2024-03-01 13:39:06 +02:00
batched.swift
beam-search
benchmark
convert-llama2c-to-ggml	ggml, common, examples, tests : fixed type arguments in printf (#5528)	2024-02-18 18:20:12 +02:00
embedding
export-lora	ci : add an option to fail on compile warning (#3952)	2024-02-17 23:03:14 +02:00
finetune	code : normalize enum names (#5697)	2024-02-25 12:09:09 +02:00
gguf
imatrix
infill	convert : automatically fall back to HfVocab if tokenizer.model doesn't exist (#5821)	2024-03-02 12:27:26 -05:00
jeopardy
llama-bench	Support multiple GPUs (split mode) on SYCL backend (#5806)	2024-03-02 19:49:30 +08:00
llama.android	ggml-quants : provide ggml_vqtbl1q_u8 for 64bit compatibility (#5711)	2024-02-25 20:43:00 +02:00
llama.swiftui
llava	code : normalize enum names (#5697)	2024-02-25 12:09:09 +02:00
lookahead
lookup
main	main : support special tokens as reverse/anti prompt (#5847)	2024-03-04 09:57:20 +02:00
main-cmake-pkg
parallel
passkey	llama : fix defrag bugs + add parameter (#5735)	2024-02-27 14:35:51 +02:00
perplexity	ci : fix wikitext url + compile warnings (#5569)	2024-02-18 22:39:30 +02:00
quantize	IQ4_XS: a 4.25 bpw quantization (#5747)	2024-02-27 16:34:24 +02:00
quantize-stats
save-load-state
server	add alias for chat template (#5858)	2024-03-04 12:22:08 +01:00
simple
speculative	speculative : implement stochastic speculative sampling (#5625)	2024-03-04 20:24:00 +02:00
sycl	Support multiple GPUs (split mode) on SYCL backend (#5806)	2024-03-02 19:49:30 +08:00
tokenize
train-text-from-scratch	code : normalize enum names (#5697)	2024-02-25 12:09:09 +02:00
alpaca.sh
base-translate.sh
chat-13B.bat
chat-13B.sh
chat-persistent.sh
chat-vicuna.sh
chat.sh
CMakeLists.txt
gpt4all.sh
json-schema-to-grammar.py	examples : support minItems/maxItems in JSON grammar converter (#5039)	2024-02-19 16:14:07 +02:00
llama.vim
llama2-13b.sh
llama2.sh
llm.vim
make-ggml.py
Miku.sh
pydantic-models-to-grammar-examples.py
pydantic_models_to_grammar.py
reason-act.sh
server-llama2-13B.sh