* fix: revert showing control tokens by default * feat: revert changes to default behavior of llama_token_to_piece; provide overridden declaration to receive "bool special" param to toggle showing control tokens * feat: use the overridden declaration of llama_token_to_piece from common/common.cpp to specify "false" so that control tokens are not shown in chat completion responses" * common : simplify --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> |
||
|---|---|---|
| .. | ||
| base64.hpp | ||
| build-info.cpp.in | ||
| CMakeLists.txt | ||
| common.cpp | ||
| common.h | ||
| console.cpp | ||
| console.h | ||
| grammar-parser.cpp | ||
| grammar-parser.h | ||
| json-schema-to-grammar.cpp | ||
| json-schema-to-grammar.h | ||
| json.hpp | ||
| log.h | ||
| ngram-cache.cpp | ||
| ngram-cache.h | ||
| sampling.cpp | ||
| sampling.h | ||
| stb_image.h | ||
| train.cpp | ||
| train.h | ||