Code Structure of the Intra-op Solver

The specific code of the intra-op solver (a.k.a auto-sharding) is scattered in various files of the project. This page contains some pointers to key components of the intra-op solver and help you navigate the complicated code base.


All the links below are based on alpa v0.2.2

Key Pointers

How to Read and Learn the Code

Run some simple examples

You can run the unit tests under and set break points in the python entrance run_auto_sharding_pass. You can start from the most basic ones in

Inspect the sharding strategy

You can print the HLO before and after the run_auto_sharding_pass.

How to Debug