LocalLLaMA is a Reddit community focused on running open-source LLMs locally, covering quantization, hardware and inference setups. Recent posts include a Qwen3.8 Flash Next benchmark reaching 262 tok/s on 16GB VRAM, an updated Qwen3.8-Flash GGUF, and a ranking of models runnable on 10-16GB VRAM. Members also urged torrent backups of quantized models and discussed FP32 accumulation limits on RTX consumer cards.
Community asks: where are uncensored, NSFW audio models?An r/LocalLLaMA user asked about progress on uncensored, NSFW audio models, noting an attempt about a year ago appeared to stall. The poster questioned whether the gap reflects a lack of interest, resources or know-how, contrasting it with the rapid stream of new text-model releases.
LocalLLaMA users debate open-weight model rankings and benchmarksA r/LocalLLaMA user challenged community consensus, arguing open-weight models are overrated on the Artificial-Analysis V5 Index, that favorites like GLM 5.3 do not rival Grok, and that Haiku 5.5 actually beats deepseek-v4.1-flash-1017. The poster also said ThreeJS should not be a general benchmark and defended Strata Dev's project direction.
Reddit user reports account ban in sneaky update, reigniting compute-ownership debateA LocalLLaMA user posted that their account was banned in a sneaky update, linking to a social media post and arguing this is another reason to own your own compute. The post offers few details but sparks community discussion on platform account risk and self-hosted compute.
Community debate: models chase unrealistic benchmarksA LocalLLaMA post argues current benchmarks overemphasize edge cases like binary reverse engineering and one-shot complex apps, while daily use needs models that follow instructions accurately without overreaching. The author urges lower weighting for such edge capabilities and questions keeping model progress seemingly endless to support valuations.
Community urges backing up quantized models via torrentsA LocalLLaMA community post warns that models from quant producers may be at risk of removal and urges users to back up quantized weights they want to keep. It asks those able to create torrents and share them via llama.garden, pirateface.co and huggingbay.xyz. The post does not specify the cause or which models are affected.
Qwen3.8 Flash Next hits 262 tok/s on 16GB VRAMA LocalLLaMA benchmark shows Qwen3.8-Flash-Next IQ3_S running on an RTX 5080 with 16GB VRAM and 55GB RAM, reaching about 5,600 tok/s prompt processing and 262 tok/s generation at 262K context. In an agent coding session, 99.5% of prompt tokens hit cache, with 0.2s median time to first token.
Post explains locked FP32 accumulation on RTX consumer cardsA LocalLLaMA post claims Nvidia deliberately throttles FP32-accumulation throughput on consumer GPUs for market segmentation: RTX cards from Turing to Blackwell 2.0 see halved FP16/BF16 (and FP8 on Ada and later) performance, hurting LoRA training, diffusion and LLM prefill, while pro and data-center cards are unaffected.
Community debates overlooked RTX 3090 performance potentialA LocalLLaMA user says an earlier viral warning about the RTX 3090 now makes sense and asks whether others can confirm it. The poster speculates that restrictions originally aimed at mining demand may have been lifted and could benefit LLM workloads. No data is presented.
Community updates quantized Qwen3.8-Flash GGUF modelA community release updated the Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF model, changing about 25% of retained experts while leaving retained weights unmodified versus the base IQ3_S model. Due to a limited expert budget, the model remains English-only and multilingual capabilities were not preserved.
User shares 10 practical uses for Qwen 3.6 27BA LocalLLaMA user posted a guide listing ten things to do with Qwen 3.6 27B via llama.cpp, including building small apps, custom chat frontends, ReAct loops and MCP tools. The author calls the model their first agent and aims to help newcomers get started with open-source LLMs.
It's been too quiet on the other side of the PacificThe r/LocalLLaMA community notes that AI activity on the other side of the Pacific (China) has been unusually quiet lately.
Ranking of models runnable on 10-16GB VRAMA Reddit user ranks LLMs runnable on 10-16GB VRAM, comparing speed, coherency, multilingual handling and instruction-following, with Gemma 4 26B-A4B and Qwen 3.6 35B-A3B leading. The author notes models under 120B rarely know media trivia, with only 200B+ models reliable in his test.
Whistle Releases 16.9 MB Speech-to-Text ToolA LocalLLaMA user shared Whistle, a speech-to-text tool that is only 16.9 MB, highlighting lightweight local operation.
Community asks: what are local Jev-like decision models used for?A LocalLLaMA user asks what practical use cases exist for running a Jev-like decision model locally. They currently use it only for deep eval testing, calling it somewhat a "solution in search of a problem," and want to learn what others do with such models.
LocalLLaMA community discusses REA projectA Reddit post on r/LocalLLaMA seeks community views on the GitHub project REA, noting it hit the top spot on GitHub a few days ago. No further technical details are provided.
A Week in Beijing and Shanghai With China's AI BuildersAn article about the people building AI in Beijing and Shanghai was shared in the LocalLLaMA community for interested readers.
LocalLLaMA community shares progress comparisonAn r/LocalLLaMA post titled "How far we've come" presents a comparison of local open-source model progress, sparking community discussion.
How abliterated models can get you pwnedA LocalLLaMA post warns that using abliterated models (with safety limits removed) may pose security risks and lead to users being pwned. No specific technical details were provided.
Local AI world model: 1B model runs real-time on consumer GPUsA developer released a ~960M-parameter world model that turns images into playable characters with text guidance, running at 12fps on an RTX 5090 (peak 50-60fps) and 30fps on an M5 MacBook. It uses a pure transformer, block causal mask, and diffusion forcing training.
Developer posts update on local Yandex 80B finetuneA developer on Reddit shared progress finetuning Yandex's AliceAI-80B-A3B base model. After the first SFT round the model showed chain-of-thought reasoning and conversation ability but was underfit from insufficient instruct data, so the alpha checkpoint won't be released. Next, roughly 5M more general instruct tokens will be added, training locally on three 32GB V100s.
Low-VRAM users hope for Qwen4 35B and similar modelsA r/LocalLLaMA user joked that owners of 8GB VRAM plus 16GB RAM check the subreddit daily hoping for new models such as Qwen4 35B A3B, while still relying on Gemma 26B QAT at about 26 tokens per second.
LocalLLaMA users complain about Strata spamA user on r/LocalLLaMA complained that the subreddit is flooded with repetitive praise for the model Strata, comparing it to cult-like proselytizing. The post reflects frustration with marketing-style spam in the open-source community.
Blogger says self-hosting AI saves no money but sticks with itA r/LocalLLaMA user shared a blog post arguing that self-hosting AI is not cheaper than using APIs, but still prefers it for reasons such as data privacy. The post sparked debate over the value of local deployment.
Local AI costs €0.12/hour: is paying for frontier models cheaper?A LocalLLaMA user says running Qwen locally on an AMD 7900 XTX plus 9800X3D costs about €0.12 per hour, based on a local electricity price of €0.25/kWh. The user asks whether frontier models are actually cheaper per million tokens, or whether the calculation is wrong.
LocalLLaMA Users Call for Stricter Model Benchmark PostsA user on r/LocalLLaMA complained that low-quality model performance posts, such as "1B model at 832843tok/s," lack reproducible details. The poster urged requiring perplexity/KLD, hardware specs, quantization, and runtime parameters, arguing raw throughput is meaningless if output quality is poor.
Community debates surge of llama.cpp hard forksAn r/LocalLLaMA user criticized the many llama.cpp hard forks optimized for specific GPUs that never submit PRs upstream, instead rebranding and promoting themselves. Consensus held that upstream must meet compatibility needs while downstream forks can be more focused, though low-effort engagement-bait forks remain criticized.
How to give local models reliable web browsingA LocalLLaMA user asks how to add web browsing and search to Deepseek Harness, noting its default webfetch plugin needs specific URLs, many search APIs are paid, and free options like SearXNG are often rate-limited.
User calls for pinned model deployment guides on LocalLLaMAA user proposes a pinned section on the LocalLLaMA forum with detailed deployment guides for each model, covering the best inference engine, minimum hardware requirements, and trusted sources for buying or renting hardware. The user complains that AI search answers are often hallucinated and even make up prices, making accurate information hard to find.
Post claims GPUs now 100% efficientA LocalLLaMA post claims GPUs are now 100% efficient, without providing technical details.
Community shares contrastive language models postA LocalLLaMA user posted about contrastive language models with an image link, offering no further text explanation.
The A.I. Daily