Alpa
0.2.3.dev17

Getting Started

  • Install Alpa
  • Alpa Quickstart

Tutorials

  • Distributed Training with Both Shard and Pipeline Parallelism
  • Differences between alpa.parallelize, jax.pmap and jax.pjit
  • Serving OPT-175B, BLOOM-176B and CodeGen-16B using Alpa
  • Performance Tuning Guide
  • ICML’22 Big Model Tutorial
  • Using Alpa on Slurm
  • Frequently Asked Questions (FAQ)

Architecture

  • Design and Architecture
  • Alpa Compiler Walk-Through
  • Code Structure of the Intra-op Solver

Benchmark

  • Performance Benchmark

Publications

  • Publications

Developer Guide

  • Developer Guide
Alpa
  • »
  • Alpa Tutorials
  • Edit on GitHub

Alpa Tutorials

Alpa Quickstart

Alpa Quickstart

Distributed Training with Both Shard and Pipeline Parallelism

Distributed Training with Both Shard and Pipeline Parallelism

Differences between alpa.parallelize, jax.pmap and jax.pjit

Differences between alpa.parallelize, jax.pmap and jax.pjit

Download all examples in Python source code: tutorials_python.zip

Download all examples in Jupyter notebooks: tutorials_jupyter.zip

Gallery generated by Sphinx-Gallery

© Copyright 2022, Alpa Developers.

Built with Sphinx using a theme provided by Read the Docs.