Package index

All functions

rllm_decode() rllm_encode()

Convert between token ids and raw bytes

rllm_forward()

Run a transformer forward pass and return the logits

rllm_generate()

Generate tokens with a KV cache (greedy or sampled)

rllm_gguf_model()

Load a llama-architecture GGUF model for forward passes

rllm_kv_cache()

Create a KV cache for incremental decoding

rllm_quantize_tensor()

Quantize a matrix into an Rfmalloc-backed quantized tensor

rllm_use_ggml() rllm_backend_enabled()

Enable or disable the ggml quantized matmul backend