I tried running ollama on OpenWRT, here are the logs and tests I did. It seems like using AI on OpenWRT is very possible.
BusyBox v1.37.0 (2026-05-13 22:42:09 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt 25.12.4, r32933-4ccb782af7 Dave's Guitar
-----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
OpenWrt recently switched to the "apk" package manager!
OPKG Command APK Equivalent Description
------------------------------------------------------------------
opkg install <pkg> apk add <pkg> Install a package
opkg remove <pkg> apk del <pkg> Remove a package
opkg upgrade apk upgrade Upgrade all packages
opkg files <pkg> apk info -L <pkg> List package contents
opkg list-installed apk info List installed packages
opkg update apk update Update package lists
opkg search <pkg> apk search <pkg> Search for packages ------------------------------------------------------------------
For more information visit:
https://openwrt.org/docs/guide-user/additional-software/opkg-to-apk-cheatsheet
root@OpenWrt:~# ollama -v
Warning: could not connect to a running Ollama instance
Warning: client version is 0.30.6
root@OpenWrt:~# ollama serve
time=2026-06-08T21:14:21.256+08:00 level=INFO source=routes.go:1919 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: LLAMA_ARG_FIT: LLAMA_ARG_FIT_TARGET: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GO_TEMPLATE:true OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_IGPU_ENABLE: OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY:
OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_TRANSFER_STREAMS:4 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:true ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-06-08T21:14:21.258+08:00 level=INFO source=routes.go:1921 msg="Ollama cloud disabled: false"
time=2026-06-08T21:14:21.259+08:00 level=INFO source=images.go:864 msg="total blobs: 0"
time=2026-06-08T21:14:21.260+08:00 level=INFO source=images.go:871 msg="total unused blobs removed: 0"
time=2026-06-08T21:14:21.261+08:00 level=INFO source=routes.go:1981 msg="Listening on 127.0.0.1:11434 (version 0.30.6)"
time=2026-06-08T21:14:21.262+08:00 level=INFO source=model_list_cache.go:111 msg="model list cache hydration complete" models=0 failures=0 elapsed=664.38µs
time=2026-06-08T21:14:21.263+08:00 level=INFO source=runner.go:60 msg="discovering available GPUs..."
time=2026-06-08T21:14:21.430+08:00 level=INFO source=types.go:50 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="1.8 GiB" available="1.5 GiB"
time=2026-06-08T21:14:21.431+08:00 level=INFO source=routes.go:2031 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096
time=2026-06-08T21:14:23.970+08:00 level=INFO source=model_recommendations.go:177 msg="model recommendations cache sleep scheduled" wait=4h8m45.894495986s consecutive_failures=0
[GIN] 2026/06/08 - 21:14:39 | 200 | 270.752µs | 127.0.0.1 | GET "/api/version"
[GIN] 2026/06/08 - 21:15:18 | 200 | 135.167µs | 127.0.0.1 | HEAD "/"
[GIN] 2026/06/08 - 21:15:18 | 404 | 968.841µs | 127.0.0.1 | POST "/api/show"
time=2026-06-08T21:15:20.475+08:00 level=INFO source=download.go:179 msg="downloading 2af3b81862c6 in 7 100 MB part(s)"
time=2026-06-08T21:26:32.821+08:00 level=INFO source=download.go:376 msg="2af3b81862c6 part 5 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster
connection."
time=2026-06-08T21:26:33.821+08:00 level=INFO source=download.go:376 msg="2af3b81862c6 part 6 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster
connection."
time=2026-06-08T21:26:37.945+08:00 level=INFO source=download.go:297 msg="2af3b81862c6 part 5 attempt 0 failed: Get \"https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/2a/2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20260608%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20260608T131520Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=1cb21119b6801a94eb44c308956adddcbc05da7c02ded7d27a2036a74f994d21\": dial tcp [2606:4700:2ff9::1]:443: connect: network is unreachable, retrying in 1s"
time=2026-06-08T21:26:37.945+08:00 level=INFO source=download.go:297 msg="2af3b81862c6 part 6 attempt 0 failed: Get \"https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/2a/2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20260608%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20260608T131520Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=1cb21119b6801a94eb44c308956adddcbc05da7c02ded7d27a2036a74f994d21\": dial tcp [2606:4700:2ff9::1]:443: connect: network is unreachable, retrying in 1s"
time=2026-06-08T21:30:18.957+08:00 level=INFO source=download.go:179 msg="downloading af0ddbdaaa26 in 1 70 B part(s)"
time=2026-06-08T21:30:21.066+08:00 level=INFO source=download.go:179 msg="downloading c8472cd9daed in 1 31 B part(s)"
time=2026-06-08T21:30:23.197+08:00 level=INFO source=download.go:179 msg="downloading fa956ab37b8c in 1 98 B part(s)"
time=2026-06-08T21:30:25.081+08:00 level=INFO source=download.go:179 msg="downloading 6331358be52a in 1 483 B part(s)"
[GIN] 2026/06/08 - 21:30:28 | 200 | 15m10s | 127.0.0.1 | POST "/api/pull"
[GIN] 2026/06/08 - 21:30:29 | 200 | 1.035429888s | 127.0.0.1 | POST "/api/show"
[GIN] 2026/06/08 - 21:30:29 | 200 | 3.380736ms | 127.0.0.1 | POST "/api/show"
time=2026-06-08T21:30:31.477+08:00 level=INFO source=sched.go:1148 msg="disabling mmap for llama-server load by default" model=/root/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 reason=cpu
time=2026-06-08T21:30:31.478+08:00 level=INFO source=server.go:109 msg="using llama-server for model" model=/root/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816
time=2026-06-08T21:30:31.478+08:00 level=WARN source=server.go:114 msg="requested context size too large for model" num_ctx=4096 n_ctx_train=2048
time=2026-06-08T21:30:31.483+08:00 level=INFO source=llama_server.go:403 msg="starting llama-server" cmd="/usr/lib/ollama/llama-server --model /root/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 --port 46339 --host 127.0.0.1 --no-webui --offline -c 2048 -np 1 --log-verbosity 4 --no-log-prefix --no-log-timestamps --no-jinja --chat-template chatml --no-mmap --flash-attn auto -b 512 -ub 512 --context-shift --keep 4"
time=2026-06-08T21:30:31.485+08:00 level=INFO source=sched.go:613 msg="system memory" total="1.8 GiB" free="1.5 GiB" free_swap="4.9 GiB"
time=2026-06-08T21:30:31.485+08:00 level=INFO source=llama_server.go:886 msg="loading model via llama-server" model=/root/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816
time=2026-06-08T21:30:31.485+08:00 level=INFO source=llama_server.go:1131 msg="waiting for llama-server to start responding"
time=2026-06-08T21:30:31.487+08:00 level=INFO source=llama_server.go:1186 msg="waiting for llama-server to become available" status="llm server not responding"
common_params_print_info: build 1 (6f3a9f3de) with GNU 14.3.0 for Linux aarch64
log_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
device_info:
- CPU : ARMv8 Processor rev 4 (v8l) (1792 MiB, 1792 MiB free)
system_info: n_threads = 4 (n_threads_batch = 4) / 4 | CPU : NEON = 1 | ARM_FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1
|
srv init: using 5 threads for HTTP server
srv init: The UI is disabled
srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
srv start: binding port with default address family
srv llama_server: loading model
srv load_model: loading model '/root/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816'
common_init_result: fitting params to device memory ...
common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only
occurs with -fit on)
common_params_fit_impl: getting device memory data for initial
parameters:
time=2026-06-08T21:30:31.740+08:00 level=INFO source=llama_server.go:1186 msg="waiting for llama-server to become available" status="llm server loading model"
common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
common_memory_breakdown_print: | - Host |
701 = 606 + 44 + 51 |
common_params_fit_impl: projected to use 701 MiB of host memory vs. 1792 MiB of total host memory
common_params_fit_impl: will leave 1091 >= 1024 MiB of system memory, no changes needed
common_fit_params: successfully fit params to free device memory
common_fit_params: fitting params to free memory took 0.52 seconds
llama_model_loader: loaded meta data with 23 key-value pairs and 201 tensors from /root/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = TinyLlama
llama_model_loader: - kv 2: llama.context_length u32 = 2048
llama_model_loader: - kv 3: llama.embedding_length u32 = 2048
llama_model_loader: - kv 4: llama.block_count u32 = 22
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64
llama_model_loader: - kv 7: llama.attention.head_count u32 = 32
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 11: general.file_type u32 = 2
llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6,
6, ...
llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n...
llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 2
llama_model_loader: - kv 21: tokenizer.chat_template str = {% for message in messages %}\n{%
if m...
llama_model_loader: - kv 22: general.quantization_version u32 = 2
llama_model_loader: - type f32: 45 tensors
llama_model_loader: - type q4_0: 155 tensors
llama_model_loader: - type q6_K: 1 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_0
print_info: file size = 606.53 MiB (4.63 BPW)
load: 0 unused tokens
load: printing all EOG tokens:
load: - 2 ('</s>')
load: special tokens cache size = 3
load: token to piece cache size = 0.1684 MB
print_info: arch = llama
print_info: vocab_only = 0
print_info: no_alloc = 0
print_info: n_ctx_train = 2048
print_info: n_embd = 2048
print_info: n_embd_inp = 2048
print_info: n_layer = 22
print_info: n_head = 32
print_info: n_head_kv = 4
print_info: n_rot = 64
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 64
print_info: n_embd_head_v = 64
print_info: n_gqa = 8
print_info: n_embd_k_gqa = 256
print_info: n_embd_v_gqa = 256
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-05
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: f_attn_value_scale = 0.0000
print_info: n_ff = 5632
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: n_expert_groups = 0
print_info: n_group_used = 0
print_info: causal attn = 1
print_info: pooling type = -1
print_info: rope type = 0
print_info: rope scaling = linear
print_info: freq_base_train = 10000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 2048
print_info: rope_yarn_log_mul = 0.0000
print_info: rope_finetuned = unknown
print_info: model type = 1B
print_info: model params = 1.10 B
print_info: general.name = TinyLlama
print_info: vocab type = SPM
print_info: n_vocab = 32000
print_info: n_merges = 0
print_info: BOS token = 1 '<s>'
print_info: EOS token = 2 '</s>'
print_info: UNK token = 0 '<unk>'
print_info: PAD token = 2 '</s>'
print_info: LF token = 13 '<0x0A>'
print_info: EOG token = 2 '</s>'
print_info: max token length = 48
load_tensors: loading model tensors, this can take a while... (mmap = false, direct_io = false)
load_tensors: CPU model buffer size = 606.53 MiB
.....................................................................................
common_init_result: added </s> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 2048
llama_context: n_ctx_seq = 2048
llama_context: n_batch = 512
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = false
llama_context: freq_base = 10000.0
llama_context: freq_scale = 1
llama_context: n_rs_seq = 0
llama_context: n_outputs_max = 1
llama_context: CPU output buffer size = 0.12 MiB
llama_kv_cache: CPU KV buffer size = 44.00 MiB
llama_kv_cache: size = 44.00 MiB ( 2048 cells, 22 layers,
1/1 seqs), K (f16): 22.00 MiB, V (f16): 22.00 MiB
llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 64
llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 64
sched_reserve: reserving ...
sched_reserve: Flash Attention was auto, set to enabled
sched_reserve: resolving fused Gated Delta Net support:
sched_reserve: fused Gated Delta Net (autoregressive) enabled
sched_reserve: fused Gated Delta Net (chunked) enabled
sched_reserve: CPU compute buffer size = 51.01 MiB
sched_reserve: graph nodes = 688
sched_reserve: graph splits = 1
sched_reserve: reserve took 24.03 ms, sched copies = 1
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
srv load_model: initializing slots, n_slots = 1
common_speculative_init: no implementations specified for speculative decoding
slot load_model: id 0 | task -1 | new slot, n_ctx = 2048
srv load_model: prompt cache is enabled, size limit: 8192 MiB
srv load_model: use `--cache-ram 0` to disable the prompt cache
srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
srv load_model: context checkpoints enabled, max = 32, min spacing = 256
srv init: --cache-idle-slots requires --kv-unified, disabling
init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
srv init: init: chat template, thinking = 0
srv llama_server: model loaded
srv llama_server: server is listening on http://127.0.0.1:46339
srv update_slots: all slots are idle
time=2026-06-08T21:30:35.553+08:00 level=INFO source=llama_server.go:1198 msg="llama-server started in 4.07 seconds"
time=2026-06-08T21:30:35.986+08:00 level=INFO source=images.go:354 msg="template selection" model=registry.ollama.ai/library/tinyllama:latest selected=go_template renderer="" parser="" go_template=[completion] chat_template=[completion] harmony=null renderer_parser=null
time=2026-06-08T21:30:35.987+08:00 level=INFO source=sched.go:729 msg="loaded runners" count=1
time=2026-06-08T21:30:35.987+08:00 level=INFO source=llama_server.go:1131 msg="waiting for llama-server to start responding"
time=2026-06-08T21:30:35.991+08:00 level=INFO source=llama_server.go:1198 msg="llama-server started in 4.51 seconds"
[GIN] 2026/06/08 - 21:30:35 | 200 | 6.495644682s | 127.0.0.1 | POST "/api/generate"
[GIN] 2026/06/08 - 21:31:34 | 200 | 225.126µs | 127.0.0.1 | HEAD "/"
[GIN] 2026/06/08 - 21:31:34 | 200 | 1.35647ms | 127.0.0.1 | GET "/api/tags"
[GIN] 2026/06/08 - 21:33:38 | 200 | 107.001µs | 127.0.0.1 | HEAD "/"
[GIN] 2026/06/08 - 21:33:38 | 200 | 2.946398ms | 127.0.0.1 | POST "/api/show"
srv log_server_r: done request: POST /tokenize 127.0.0.1 200
slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
srv get_availabl: updating prompt cache
srv load: - looking for better prompt, base f_keep =
-1.000, sim = 0.000
srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 2048 tokens, 8589934592 est)
srv get_availabl: prompt cache update took 0.05 ms
slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p
-> ?min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id 0 | task -1 | sampler params:
repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 2048
top_k = 40, top_p = 0.900, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.800
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 2048, n_keep = 4, task.n_tokens = 38
slot update_slots: id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
slot init_sampler: id 0 | task 0 | init sampler, took 0.09 ms, tokens: text = 38, total = 38
srv log_server_r: done request: POST /completion 127.0.0.1 200slot print_timing: id 0 | task 0 | prompt eval time = 12001.34 ms / 38 tokens ( 315.82 ms per token, 3.17 tokens per second)
slot print_timing: id 0 | task 0 | eval time = 7634.36 ms / 21 tokens ( 363.54 ms per token, 2.75 tokens per second)
slot print_timing: id 0 | task 0 | total time = 19635.70 ms / 59 tokens
slot print_timing: id 0 | task 0 | graphs reused =
20
slot release: id 0 | task 0 | stop processing: n_tokens = 58, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: done request: POST /tokenize 127.0.0.1 200
[GIN] 2026/06/08 - 21:33:59 | 200 | 21.187439967s | 127.0.0.1 | POST "/api/generate"
[GIN] 2026/06/08 - 21:35:02 | 200 | 113.334µs | 127.0.0.1 | HEAD "/"
[GIN] 2026/06/08 - 21:35:02 | 200 | 428.878µs | 127.0.0.1 | GET "/api/ps"
[GIN] 2026/06/08 - 21:36:53 | 200 | 127.376µs | 127.0.0.1 | HEAD "/"
[GIN] 2026/06/08 - 21:36:53 | 200 | 2.799398ms | 127.0.0.1 | POST "/api/show"
srv log_server_r: done request: POST /tokenize 127.0.0.1 200
slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.600 (> 0.100 thold), f_keep = 0.414
srv get_availabl: updating prompt cache
srv prompt_save: - saving prompt with length 58, total state size = 1.247 MiB (draft: 0.000 MiB)
srv load: - looking for better prompt, base f_keep =
0.414, sim = 0.600
srv update: - cache state: 1 prompts, 1.247 MiB (limits: 8192.000 MiB, 2048 tokens, 380936 est)
srv update: - prompt 0x1981e730: 58 tokens, checkpoints: 0, 1.247 MiB
srv get_availabl: prompt cache update took 5.47 ms
slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p
-> ?min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id 0 | task -1 | sampler params:
repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 2048
top_k = 40, top_p = 0.900, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.800
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
slot launch_slot_: id 0 | task 22 | processing task, is_child
= 0
slot update_slots: id 0 | task 22 | new prompt, n_ctx_slot = 2048, n_keep = 4, task.n_tokens = 40
slot update_slots: id 0 | task 22 | cached n_tokens = 24, memory_seq_rm [24, end)
srv log_server_r: done request: POST /completion 127.0.0.1 200slot init_sampler: id 0 | task 22 | init sampler, took 0.11 ms, tokens: text = 40, total = 40
slot print_timing: id 0 | task 22 | n_decoded = 100, tg =
2.59 t/s
slot print_timing: id 0 | task 22 | n_decoded = 108, tg =
2.58 t/s
slot print_timing: id 0 | task 22 | n_decoded = 116, tg =
2.58 t/s
slot print_timing: id 0 | task 22 | n_decoded = 124, tg =
2.57 t/s
slot print_timing: id 0 | task 22 | n_decoded = 132, tg =
2.56 t/s
slot print_timing: id 0 | task 22 | n_decoded = 140, tg =
2.56 t/s
slot print_timing: id 0 | task 22 | n_decoded = 148, tg =
2.55 t/s
slot print_timing: id 0 | task 22 | n_decoded = 156, tg =
2.54 t/s
slot print_timing: id 0 | task 22 | n_decoded = 164, tg =
2.54 t/s
slot print_timing: id 0 | task 22 | n_decoded = 172, tg =
2.53 t/s
slot print_timing: id 0 | task 22 | n_decoded = 180, tg =
2.53 t/s
slot print_timing: id 0 | task 22 | n_decoded = 188, tg =
2.52 t/s
slot print_timing: id 0 | task 22 | n_decoded = 196, tg =
2.52 t/s
slot print_timing: id 0 | task 22 | n_decoded = 204, tg =
2.51 t/s
slot print_timing: id 0 | task 22 | n_decoded = 212, tg =
2.51 t/s
slot print_timing: id 0 | task 22 | n_decoded = 219, tg =
2.50 t/s
slot print_timing: id 0 | task 22 | n_decoded = 226, tg =
2.49 t/s
slot print_timing: id 0 | task 22 | n_decoded = 233, tg =
2.48 t/s
slot print_timing: id 0 | task 22 | n_decoded = 240, tg =
2.48 t/s
slot print_timing: id 0 | task 22 | n_decoded = 247, tg =
2.47 t/s
slot print_timing: id 0 | task 22 | n_decoded = 254, tg =
2.46 t/s
slot print_timing: id 0 | task 22 | n_decoded = 261, tg =
2.45 t/s
slot print_timing: id 0 | task 22 | n_decoded = 268, tg =
2.45 t/s
slot print_timing: id 0 | task 22 | n_decoded = 275, tg =
2.44 t/s
slot print_timing: id 0 | task 22 | n_decoded = 282, tg =
2.43 t/s
slot print_timing: id 0 | task 22 | n_decoded = 289, tg =
2.43 t/s
slot print_timing: id 0 | task 22 | n_decoded = 296, tg =
2.42 t/s
slot print_timing: id 0 | task 22 | n_decoded = 303, tg =
2.41 t/s
slot print_timing: id 0 | task 22 | n_decoded = 310, tg =
2.41 t/s
slot print_timing: id 0 | task 22 | n_decoded = 317, tg =
2.40 t/s
slot print_timing: id 0 | task 22 | n_decoded = 324, tg =
2.40 t/s
slot print_timing: id 0 | task 22 | n_decoded = 331, tg =
2.39 t/s
slot print_timing: id 0 | task 22 | n_decoded = 338, tg =
2.39 t/s
slot print_timing: id 0 | task 22 | n_decoded = 345, tg =
2.38 t/s
slot print_timing: id 0 | task 22 | n_decoded = 352, tg =
2.38 t/s
slot print_timing: id 0 | task 22 | n_decoded = 359, tg =
2.37 t/s
slot print_timing: id 0 | task 22 | n_decoded = 366, tg =
2.37 t/s
slot print_timing: id 0 | task 22 | n_decoded = 373, tg =
2.36 t/s
slot print_timing: id 0 | task 22 | n_decoded = 380, tg =
2.36 t/s
slot print_timing: id 0 | task 22 | n_decoded = 387, tg =
2.35 t/s
slot print_timing: id 0 | task 22 | n_decoded = 394, tg =
2.35 t/s
slot print_timing: id 0 | task 22 | n_decoded = 401, tg =
2.34 t/s
slot print_timing: id 0 | task 22 | n_decoded = 408, tg =
2.34 t/s
slot print_timing: id 0 | task 22 | n_decoded = 415, tg =
2.34 t/s
slot print_timing: id 0 | task 22 | prompt eval time = 5136.58 ms / 16 tokens ( 321.04 ms per token, 3.11 tokens per second)
slot print_timing: id 0 | task 22 | eval time = 179937.09 ms / 420 tokens ( 428.42 ms per token, 2.33 tokens per second)
slot print_timing: id 0 | task 22 | total time = 185073.66 ms / 436 tokens
slot print_timing: id 0 | task 22 | graphs reused =
437
slot release: id 0 | task 22 | stop processing: n_tokens
= 459, truncated = 0
srv update_slots: all slots are idle
srv log_server_r: done request: POST /tokenize 127.0.0.1 200
[GIN] 2026/06/08 - 21:40:00 | 200 | 3m6s | 127.0.0.1 | POST "/api/generate"
[GIN] 2026/06/08 - 21:40:08 | 200 | 107.126µs | 127.0.0.1 | HEAD "/"
[GIN] 2026/06/08 - 21:40:08 | 200 | 121.501µs | 127.0.0.1 | GET "/api/ps"