Alpa Documentation

Star Fork

Alpa is a system for training and serving large-scale neural networks.

Getting Started

• Install Alpa
• Alpa Quickstart

Tutorials

• Distributed Training with Both Shard and Pipeline Parallelism
• Differences between alpa.parallelize, jax.pmap and jax.pjit
• Performance Tuning Guide
• Serving OPT-175B using Alpa
• ICML’22 Big Model Tutorial
• Using Alpa on Slurm
• Frequently Asked Questions (FAQ)

Benchmark

• Performance Benchmark

Architecture

• Design and Architecture
• Alpa Compiler Walk-Through
• Code Structure of the Intra-op Solver

Developer Guide

• Developer Guide