RunByAI

Tag: Inference Acceleration

4 post(s)

September 23, 20260 comment(s)

Introduction to Reasoning Engines: How vLLM and PagedAttention Improve the Throughput of Large Models

When deploying a large model as a service, "whether it's fast or not, how many people can be served at once" is often more critical than "whether it can run". Besides the model itself, it is often the

Tech Guide Large Language Model (LLM)Inference Acceleration

September 22, 20260 comment(s)

Introduction to Flash Attention: Why Attention Computing Eats Video Memory and How It Saves It

The "attention" of large models may seem like a matrix multiplication, but in reality, it is the most memory intensive and slow down step in training and inference. Flash Attention significantly impro

Tech Guide Large Language Model (LLM)Inference Acceleration

September 20, 20260 comment(s)

Introduction to Speculative Decoding: Drafting Small Models and Making Decisions on Large Models

The inference of large models is slow, often in a counterintuitive way: every time a token is generated, the entire model needs to be run once, and the GPU's large amount of computing power is actuall

Tech Guide Inference Acceleration Large Language Model (LLM)Reasoning optimization

June 9, 20260 comment(s)

New Pattern of AI Chips: Architecture Transformation from Training to Reasoning

In 2026, the AI chip market is undergoing a profound architectural transformation from "training first" to "inference is king". As large models move from the laboratory to large-scale deployment, the

Tech AI chip Inference Acceleration large model