RunByAI

Tag: Reasoning optimization

5 post(s)

October 6, 20260 comment(s)

Introduction to Decoding Strategies: How Big Models "Choose the Next Word"

When a big model completes a sentence, it faces tens of thousands or even tens of thousands of candidate words. Why does it pick out the next one? Why is the same question sometimes answered steadily

Tech Guide large model Reasoning optimization

October 6, 20260 comment(s)

Introduction to Speculative Decoding: How LLMs Speed Up Inference by Guessing and Verifying

Most people who have used local large models have a common experience: typing is too slow. Especially in dialogue based generation, the model must squeeze out each token one by one, making it difficul

Tech Guide large model Reasoning optimization

September 21, 20260 comment(s)

Introduction to KV Cache: Why does big model inference become slower as we chat, and how does caching save computing power

Unlike questions that are answered with just one glance, during multiple rounds of conversation, you will find that the response speed of the large model is not stable: the first sentence is always ha

Tech Guide Reasoning optimization Large Language Model (LLM)

September 20, 20260 comment(s)

Introduction to Speculative Decoding: Drafting Small Models and Making Decisions on Large Models

The inference of large models is slow, often in a counterintuitive way: every time a token is generated, the entire model needs to be run once, and the GPU's large amount of computing power is actuall

Tech Guide Inference Acceleration Large Language Model (LLM)Reasoning optimization

May 30, 20260 comment(s)

Complete Guide to Local Deployment of Large Models: From Hardware Selection to Operational Optimization

With the rapid development of open-source big models, deploying big models locally is no longer the exclusive skill of geeks. This article will systematically introduce the complete process of deployi

Tech Guide Ollama large model GGUF