mirror of
https://github.com/tldr-pages/tldr.git
synced 2026-08-30 17:33:50 +08:00
llama-cli: add page (#22660)
Co-authored-by: Sadeed <sadeeedw@gmail.com> Co-authored-by: Ivan Baluta <ivanbaluta.dev@gmail.com>
This commit is contained in:
@@ -0,0 +1,24 @@
|
||||
# llama-cli
|
||||
|
||||
> Command-line interface for LLM inference.
|
||||
> More information: <https://github.com/ggml-org/llama.cpp#llama-cli>.
|
||||
|
||||
- Start up LLM [m]odel and interface:
|
||||
|
||||
`llama-cli {{[-m|--model]}} {{path/to/model}}`
|
||||
|
||||
- Start with setup prompt [c]ontext size:
|
||||
|
||||
`llama-cli {{[-m|--model]}} {{path/to/model}} {{[-c|--ctx-size]}} {{number}}`
|
||||
|
||||
- Start with set offload [n]umber of layer [g]pu [l]ayers:
|
||||
|
||||
`llama-cli {{[-m|--model]}} {{path/to/model}} {{[-ngl|--gpu-layers|--n-gpu-layers]}} {{auto|all|number}}`
|
||||
|
||||
- Start with sending content as [sys]tem prompt from a [f]ile:
|
||||
|
||||
`llama-cli {{[-m|--model]}} {{path/to/model}} {{[-sysf|--system-prompt-file]}} {{path/to/prompt}}`
|
||||
|
||||
- Start with sending content in [f]ile as first prompt:
|
||||
|
||||
`llama-cli {{[-m|--model]}} {{path/to/model}} {{[-f|--file]}} {{path/to/prompt}}`
|
||||
Reference in New Issue
Block a user