How Moonlight thinks.
A transparent look at the lifecycle of a prompt inside a local inference engine.
Acquiring the Intelligence
Before Moonlight can think, it needs a brain. Models are downloaded from verified Hugging Face repositories directly to your device's internal storage.
Memory Mapping
When you start a chat, Moonlight uses llama.cpp to map the model weights into your device's RAM. This prepares the neural network for inference without fully uncompressing it, preserving your device's memory.
Tokenization & Context
Your text prompt is converted into tokens (numbers) that the model understands. These tokens are passed into the model along with the system prompt, which defines the AI's behavior.
Local Inference
The model calculates probabilities for the next token based on the mathematical weights stored on your phone. It uses your device's CPU/GPU to perform matrix multiplication, generating the answer one word at a time. The computation happens locally.
De-tokenization
The predicted tokens are converted back into human-readable text and streamed into your chat interface instantly. The entire loop completes within your physical device.