Tiny Language Models Meet Embedded Systems: How a 14.3-Million Parameter AI Powers a Handheld Virtual Fish Tank
By Tech & Embedded Systems Desk
Published: September 2026
Main Facts: Edge AI Shrinks Down to the Microcontroller
In a striking demonstration of how far model distillation and quantization have advanced, a developer known as "pocket-tank" has successfully deployed a 14.3-million-parameter language model directly onto an ESP32-S3 microcontroller. Running entirely offline without a cloud connection, API, or external server, this tiny AI engine makes every high-level decision for a virtual ecosystem housed on a 1.8-inch AMOLED display.
The application is a pocket-sized virtual fish tank where multiple fish swim, forage, grow, and socialize. The model determines each creature’s overarching behavioral goals—deciding whether a fish should eat, hide, explore, socialize, rest, or flee from perceived threats.
Measuring just 7.56 MB in its quantized 4-bit state, the neural network reads directly from flash memory using memory-mapping techniques. The hardware platform—a Waveshare ESP32-S3-Touch-AMOLED-1.8 board featuring an ESP32-S3R8 chip, 16 MB of flash, and 8 MB of octal PSRAM—costs roughly $35 all-in. By leveraging dual-core processing, the microcontroller executes the heavy text-generation inference loop on one core while simultaneously maintaining a smooth, real-time graphical simulation at 25–30 frames per second on the other.
This project bridges the gap between massive cloud-hosted Large Language Models (LLMs) and resource-constrained edge hardware. It proves that complex behavioral reasoning can be achieved on a chip consuming mere fractions of a watt, operating entirely untethered from the internet.
Chronology: From 26 Billion Parameters to a Handheld Microcontroller
The realization of the "pocket-tank" project represents the culmination of several overlapping trends in artificial intelligence, hardware accessibility, and embedded programming:
- The Era of Massive LLMs: For years, bleeding-edge AI has been defined by scaling laws. Models grew from hundreds of millions to hundreds of billions of parameters (such as the roughly 26-billion-parameter teacher model utilized in this distillation pipeline), requiring data centers packed with high-end GPUs to operate.
- The Rise of Local Inference Frameworks: Projects like
llama2.cdemonstrated that transformer-based models could be stripped of extraneous code and executed in plain C on consumer CPUs. This opened the floodgates for running small language models (SLMs) on laptops, single-board computers, and eventually microcontrollers. - Model Distillation and Quantization: Developers began taking massive "teacher" models and training smaller "student" networks to mimic their outputs. Combined with aggressive 4-bit quantization—which squashes the numerical precision of weights to save memory—models shrank from hundreds of megabytes or gigabytes down to sizes manageable for embedded flash memory.
- Integration of Advanced Microcontrollers: The emergence of powerful, low-cost Wi-Fi/Bluetooth microcontrollers like the ESP32-S3—equipped with vector extensions (SIMD), large external PSRAM capacities, and high-speed QSPI flash interfaces—provided the raw compute foundation needed for edge AI.
- The "Pocket-Tank" Deployment: The developer structured the firmware into three distinct layers, integrated LVGL graphical interfaces, mapped the 7.56 MB model file directly into flash memory, and achieved stable, dual-core operation. The project was subsequently published to GitHub as an open-source reference for embedded AI enthusiasts.
Supporting Data: Architecture, Performance, and Metrics
Understanding how a language model functions within the strict limitations of a microcontroller requires examining the hardware specs, model dimensions, and runtime performance metrics.
1. Model Architecture and Distillation Metrics
- Teacher Model: A massive baseline model consisting of approximately 26 billion parameters with a vocabulary size of roughly 262,000 tokens.
- Student Model ("Pocket-Tank"): 14.3 million parameters.
- Dimensions & Topology: Dimension size of 384, utilizing 8 layers and 8 attention heads. This represents roughly 1,818 times fewer parameters than the teacher model.
- Memory Footprint:
- FP32 (32-bit floating-point): ~57 MB (too large for typical microcontroller working memory).
- 4-Bit Quantized: 7.56 MB (fits comfortably into a dedicated flash partition).
- Vocabulary: Compressed to a closed-schema lexicon of just 58 tokens, drastically reducing the output space and preventing hallucination while accelerating inference.
- Training and Fidelity: The student was trained on 51,162 distinct situations labeled by the teacher model.
- The student successfully selects the teacher’s primary goal in 75% of cases (notable given the teacher agrees with its own deterministic outputs 82% of the time).
- Behavioral consistency remains high: a hungry fish chooses the
seek_foodgoal at least 89% of the time regardless of its assigned personality trait, proving that 4-bit quantization did not degrade core behavioral logic.
2. Hardware Specifications
- Microcontroller: ESP32-S3R8 dual-core Xtensa LX7 microprocessor.
- Memory: 16 MB of Flash memory and 8 MB of Octal PSRAM.
- Partition Layout: The model resides in a dedicated 8 MB raw flash partition, flashed once at address
0x290000. - Peripherals & Display: Waveshare ESP32-S3-Touch-AMOLED-1.8 board featuring a 368×448 AMOLED display, capacitive touch controller, Power Management IC (PMIC), and Real-Time Clock (RTC).
- Cost: Approximately $35 USD for the complete board, display, and integrated battery setup.
3. Runtime Benchmarks
- Inference Speed: On physical hardware, generating a decision takes approximately 3.6 seconds at a rate of 12 tokens per second. In QEMU software emulation, a decision takes about 2.3 seconds.
- Graphics Frame Rate: The AMOLED display renders the virtual tank at a smooth 25 to 30 frames per second concurrently on the microcontroller’s secondary core.
- Task Sharing: To optimize performance, a single model instance is shared among a school of four to six fish. Fish are only re-queried by the LLM advisor when their environmental situation changes significantly, preventing unnecessary compute cycles.
Three Separate Layers: Firmware Architecture
To ensure stability, modularity, and cross-platform testing, the project codebase is cleanly divided into three distinct operational layers. Crucially, these layers are compiled identically in both the QEMU simulator environment and the physical ESP32-S3 firmware.
+-------------------------------------------------------------+
| USER INTERACTION |
| (Touch Gestures, Feeding, Trust Meter) |
+-------------------------------------------------------------+
|
v
+-------------------------------------------------------------+
| REFLEXES LAYER (tank.c) |
| Physics, Steering, Schooling, Hunger, Vegetation |
| (Runs every frame, non-blocking) |
+-------------------------------------------------------------+
^ |
| Updates movement |
| based on goals | Sends text state string
| v (Hunger, energy, fear)
+-------------------------------------------------------------+
| LLM ADVISOR (llm/) |
| 4-bit Llama2.c Engine, Shared State Encoder |
| (Executes on Core 2 in background) |
+-------------------------------------------------------------+1. The Reflexes Layer (tank.c)
Operating on every single graphics frame without ever blocking on the language model, this layer handles real-time simulation mechanics. It manages collision physics, steering behaviors, schooling formations, touch gestures, feeding mechanics, a depletion/replenishment hunger economy, and environmental elements like vegetation and algae.
2. The LLM Advisor (llm/)
This is the core artificial intelligence engine—a specialized 4-bit inference engine modeled after llama2.c. It utilizes a word-level tokenizer and a shared state encoder. Operating on the ESP32-S3’s second core, it leverages SIMD (Single Instruction, Multiple Data) dot-product operations and quantized activation layers stored directly in internal SRAM.
Significantly, the model’s weight matrix is memory-mapped directly from flash memory and is never duplicated into RAM. This zero-copy strategy ensures that the virtual fish tank animation remains fluid and uninterrupted while the neural network evaluates a decision in the background.
3. The Progression Layer (progression.c)
This layer manages long-term state variables, tracking user interactions, fish growth cycles (starting from small fry and growing into fully developed virtual fish), and trust metrics. Holding a finger against the glass for three seconds prompts fish that have built up sufficient trust metrics to swim over to the touch point.
Decision-Making Loop and Behavioral Dynamics
The interaction between the deterministic reflexes layer and the stochastic language model creates lifelike emergent behaviors within the virtual aquarium.
Instead of running a complex continuous neural network, the system uses a periodic query-response loop:
- State Generation: Each fish periodically compiles its current internal metrics—hunger levels, energy reserves, fear indices, proximity to other fish, and its assigned personality traits—into a single line of text.
- Model Evaluation: The 4-bit LLM parses this text input and generates a completion string that defines a specific high-level goal and an associated urgency rating.
- Probabilistic Sampling: Rather than greedily picking the most mathematically probable token every time, the model samples from its output probability distribution. When the top two potential choices have closely matched probabilities, the fish visibly hesitates within the simulation before committing to an action.
- Reflex Translation: The reflexes layer takes the model’s chosen goal and translates it into physical steering vectors, seamlessly driving the sprite across the AMOLED screen. There are no hardcoded fallback heuristics overriding the model’s choices; the AI’s decisions dictate the simulation.
Implications: The Future of Offline, Low-Power Edge AI
The success of the "pocket-tank" project carries profound implications for hobbyist electronics, industrial automation, and the broader artificial intelligence landscape.
1. The Death of Cloud Dependency for Small-Scale AI
For years, even basic smart-home devices, toys, and wearables relied on continuous internet connectivity to ping cloud servers (such as OpenAI’s APIs or private LLM endpoints) for decision-making. This introduced latency, privacy vulnerabilities, recurring subscription costs, and total failure upon losing network connectivity. This project demonstrates that consumer-grade microcontrollers costing less than a cup of coffee can run autonomous reasoning engines locally, securely, and indefinitely.
2. Democratizing Model Distillation
While training a 26-billion-parameter model requires immense server farms, distilling that model down to fit a microcontroller is becoming increasingly accessible to individual developers. By leveraging open-source toolchains—such as ESP-IDF v5.4, LVGL graphical libraries, SDL2, and Espressif’s QEMU emulation environments—developers can prototype complex neural network behaviors on their desktop computers before flashing them to physical silicon.
3. Emergent Personality in Embedded Systems
By substituting rigid state machines with small language models, interactive devices can exhibit nuanced, non-repetitive behaviors. In the context of virtual pets, interactive toys, or educational robotics, an AI that hesitates, balances competing internal drives (hunger versus fear), and adapts its behavior based on user interaction provides a vastly superior user experience compared to traditional scripted routines.
4. Hardware Expandability
While the reference Waveshare board includes a comprehensive suite of onboard sensors, the firmware architecture is designed to integrate easily with external modules. Developers looking to expand similar projects can wire in 9-axis Inertial Measurement Units (IMUs) via I²C to track physical device orientation and movement, or integrate Real-Time Clock (RTC) modules to maintain accurate temporal awareness even when the microcontroller is powered down.
Replication and Getting Started
For engineers, students, and makers interested in exploring or replicating the "pocket-tank" architecture, the complete software toolchain, flash scripts, and firmware source code are publicly available in the official GitHub repository.
Requisite Toolchain:
- Framework: ESP-IDF v5.4
- Inference Engine: Custom 4-bit engine inspired by
llama2.c - Graphics & UI: LVGL v9.2.2 and SDL2 for desktop simulation
- Deployment Tools: ESP Web Tools, alongside project scripts
tools/make_installer.pyandtools/director.py - Emulation: Espressif-optimized QEMU build
As edge computing hardware continues to outpace expectations, projects like the pocket-sized AI fish tank signal a new frontier where artificial intelligence is no longer confined to server racks, but embedded directly into the physical objects we hold in our hands.
