New update: Run LLMs Locally
Added Mellum 2.1 from JetBrains.
Added LFM2.5-24B-A2B from Liquid AI.
https://github.com/thomas-0816/talks/blob/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #stablediffusion #qwen3 #qwen3tts #nemotron #glm #localai #opencode #systemone #laya #lev #kev #openjev #clef #lfm2_5 #mellum2_1
New update: Run LLMs Locally
Added EmbeddingGemma2 using llama.cpp.
It includes multi modality, allowing to generate embedding vectors from text, images, video, and audio.
So you can build a search engine for documents and photos.
https://github.com/thomas-0816/talks/blob/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #stablediffusion #qwen3 #qwen3tts #nemotron #glm #localai #opencode #systemone #laya #lev #kev #clef #embeddinggemma
Small update: Run LLMs Locally
Added Maple-Preview 20B-A1B using llama.cpp (6 GB, fastest 20B model).
Added Qwen 3.8 27B configuration for coding agents.
https://github.com/thomas-0816/talks/blob/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #stablediffusion #qwen3 #qwen3tts #nemotron #glm #localai #opencode #systemone #laya #lev #kev #clef
Kolibri Has Landed: A Sovereign Open-Weight Model
https://aleph-alpha.com/en/blog/kolibri-has-landed-a-sovereign-open-weight-model/
Small update: Run LLMs Locally
Added decision model Clef-Flash using llama.cpp.
https://github.com/thomas-0816/talks/blob/main/Run_LLMs_Locally_2026_ThomasBley.pdf
More about Clef from Cloudflare: https://blog.cloudflare.com/clef-decision-models/
#ai #llm #llamacpp #stablediffusion #qwen3 #qwen3tts #nemotron #glm #localai #opencode #systemone #laya #lev #kev #openjev #clef
New slides: Run LLMs Locally
Added decision models Laya, lev, Kev, OpenJev using llama.cpp.
Added testing with curl for TypeSafe System One API compatible endpoints.
Added text to speech with Qwen-3-TTS using llama.cpp.
Added Nemotron 3.5 Lightning.
https://github.com/thomas-0816/talks/blob/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #stablediffusion #qwen3 #qwen3tts #nemotron #glm #localai #opencode #systemone #laya #lev #kev #openjev
Ducks are now loving Symfony!
I've built the Doctrine DBAL for DuckDB.
It brings the whole DuckDB ecosystem to Symfony (types, extensions, parquet, data lakes, etc.).
Now you can run high performance analytics directly on your laptop or webserver together with Symfony and Doctrine ORM!
Releases are available on GitHub and Packagist.
https://github.com/thomas-0816/doctrine-dbal-duckdb
https://packagist.org/packages/thomas-0816/doctrine-dbal-duckdb
New week, new slides: Run LLMs Locally
I added virtualization of OpenCode with Matchlock and Firecracker microVMs,
containerization of OpenCode and llama.cpp with Docker
and a new slide for indirect prompt injection attacks.
Matchlock is a great project for sandboxing, bringing the advantages of containers to virtual machines.
https://codeberg.org/thbley/talks/raw/branch/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #firecracker #docker
Why run LLMs locally?
EU Icons for labelling AI-generated content
https://digital-strategy.ec.europa.eu/en/policies/eu-icons-labelling-ai-generated-content
Diffusion models are coming for text generation :)
4x faster text generation
Same week, small update: Run LLMs Locally
Multi-Token-Prediction (MTP) for Gemma-4-E4B and Gemma-4-26B from Unsloth. After 50% from QAT, this brings another 25-90% improvement in token generation speed.
The OpenCode config slide received a small update to reduce prompt sizes with "rtk" and "opencode-tool-search", reducing default prompt size by 60 percent.
Also added logging all prompts to the parameter list.
https://codeberg.org/thbley/talks/raw/branch/main/Run_LLMs_Locally_2026_ThomasBley.pdf
New week, new slides and small updates: Run LLMs Locally
Added an example to create Mermaid diagrams in llama.cpp UI.
Added QAT (Quantization-Aware Training) variants of Gemma 4 which are 50 percent faster in token generation with my local setup.
Added definitions for Deterministic and Probabilistic results.
https://codeberg.org/thbley/talks/raw/branch/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly #mellum2
New week, beautiful new slides: Run LLMs Locally
Now with Mellum2 from JetBrains!
A very fast coding model, requires only 10 GB RAM.
I also added LFM 2.5 from LiquidAI, updated translations with HY-MT2 from Tencent, added examples for wllama using re-ranking and structured output
and added thinking_budget_tokens to the curl examples.
https://codeberg.org/thbley/talks/raw/branch/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly #jetbrains #mellum2
New week, more slides: Run LLMs Locally
Now including wllama to run GGUF models inside your browser!
wllama uses llama.cpp, WebAssembly and WebGPU, bringing a completely new experience of LLMs into the web.
It has no 4 GB limitation and is faster than Transformers.js.
I also added translations using the HY-MT model from Tencent.
https://codeberg.org/thbley/talks/raw/branch/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly
New week, new slides: Run LLMs Locally
Now including multi-token prediction using Qwen3.6 35B-A3B with Nextn quantization. Also speech recognition using Qwen-3-ASR is now working directly with Llama.cpp and included in the slides.
https://codeberg.org/thbley/talks/raw/branch/main/Run_LLMs_Locally_2026_ThomasBley.pdf
#ai #llm #llamacpp #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp
