---
title: "Megatron-LM"
created: 2026-06-24
updated: 2026-06-24
type: concept
tags: ["training-framework", "distributed-training", "model-parallelism"]
sources:
  - "[[unlimited-ocr-works-2026]]"
---

# Megatron-LM

Megatron-LM（NVIDIA, arXiv:1909.08053）是训练数十亿参数大语言模型的分布式训练框架，提供张量并行、流水线并行等模型并行策略。Unlimited OCR 的训练 pipeline 基于 Megatron-LM 构建。

## 参考
- [[unlimited-ocr-works-2026]]