Axolotl debugging guide

The guide provides tips and tricks for debugging Axolotl, a tool for fine-tuning large language models. It covers general debugging tips as well as how to set up debugging in VSCode, both locally and remotely via Docker containers.

General debugging tips

• Use the latest version of Axolotl from the main branch

• Eliminate concurrency by restricting to a single GPU and dataset process

• Use a small dataset and model to speed up iteration time

• Minimize batch size and disable validation

• Clear data preprocessing and HuggingFace caches between runs

Debugging in VSCode

• Do an editable install of Axolotl

• Modify launch.json to:

  • Launch the train command with additional debugging flags

  • Set the working directory to devtools

  • Limit to a single GPU

  • Redirect outputs and caches to temp folders

• Use a pre-launch task in tasks.json to clean up temp folders between runs

• Remote debugging is supported via SSH

• The launch config can be customized for your specific use case

Debugging in Docker

• Clone Axolotl repo on the host running Docker

• Run the desired Axolotl Docker image, mounting the Axolotl directory

• Do an editable install inside the container

• If using a remote host, SSH in with VSCode

• Attach VSCode to the running Docker container

• Debug inside the container as you would locally

By following these tips and configuration examples, you can efficiently debug Axolotl to better understand how the code works behind the scenes. The ability to quickly iterate with a simple test setup is key to productive debugging.