Allocates the per-layer key/value cache slabs an incremental
rllm_forward() writes into and attends over. Each slab is a raw vector of
n_ctx * (n_embd / n_head) * n_head_kv f32 values - plain R memory by
default, or Rfmalloc-backed (file-backed, memory-mapped) when a
runtime is given, which makes the cache a disk citizen: it survives in
the runtime's file and its pages are evictable like any other fmalloc
payload. (A quantized cache codec in the TurboQuant/PolarQuant vein can
later replace the f32 slabs without touching the graph.)
Arguments
- model
An
rllm_modelfromrllm_gguf_model().- n_ctx
Maximum number of positions the cache can hold.
- runtime
Optional
Rfmalloc::open_fmalloc()runtime for the slabs.
Value
An environment of class rllm_kv_cache with fields k, v
(per-layer lists of raw vectors), n_ctx, and n_past.
Details
The returned object is an environment, so rllm_forward() can advance its
n_past by reference.