LogoLogo
Continuum Knowledge BankContinuum Applications
  • Introduction
  • Creation of Environment
    • Platform Installation
    • Axolotl Dependencies
    • setup.py objectives
      • script analysis
  • Huggingface Hub
  • Download the dataset
    • Types of Dataset Structures
    • Structuring Datasets for Fine-Tuning Large Language Models
    • Downloading Huggingface Datasets
    • Use Git to download dataset
    • Popular Datasets
    • Download cleaned Alpaca dataset
    • Template-free prompt construction
  • Downloading models
    • Phi 2.0 details
    • Downloading Phi 2.0
    • Available Models
  • Configuration for Training
  • Datasets
  • Model Selection - General
  • Phi 2.0
    • Phi 2.0 - Model Configuration
    • Phi 2.0 - Model Quantization
    • Phi 2.0 - Data Loading and Paths
    • Phi 2.0 - Sequence Configuration
    • Phi 2.0 - Lora Configuration
    • Phi 2.0 - Logging
    • Phi 2.0 - Training Configuration
    • Phi 2.0 - Data and Precision
    • Phi 2.0 - Optimisations
    • Phi 2.0 - Extra Hyperparameters
    • Phi 2.0 - All Configurations
    • Phi 2.0 - Preprocessing
    • Phi 2.0 - Training
    • Uploading Models
  • Llama2
    • Llama2 - Model Configuration
    • Llama2 - Model Quantization
    • Llama2 - Data Loading and Paths
    • Llama2 - Sequence Configuration
    • Llama2 - Lora Configuration
    • Llama2 - Logging
    • Llama2 - Training Configuration
    • Llama2 - Data and Precision
    • Llama2 - Optimisations
    • Llama2 - Extra Hyperparameters
    • Llama2- All Configurations
    • Llama2 - Training Configuration
    • Llama2 - Preprocessing
    • Llama2 - Training
  • Llama3
    • Downloading the model
    • Analysis of model files
      • Model Analysis - Configuration Parameters
      • Model Analysis - Safetensors
      • Tokenizer Configuration Files
        • Model Analysis - tokenizer.json
        • Model Analysis - Special Tokens
    • Llama3 - Model Configuration
    • Llama3 - Model Quantization
    • Llama3 - Data Loading and Paths
    • Llama3 - Sequence Configuration
    • Llama3 - Lora Configuration
    • Llama3 - Logging
    • Llama3 - Training Configuration
    • Llama3 - Data and Precision
    • Llama3 - Optimisations
    • Llama3 - Extra Hyperparameters
    • Llama3- All Configurations
    • Llama3 - Preprocessing
    • Llama3 - Training
    • Full Fine Tune
  • Special Tokens
  • Prompt Construction for Fine-Tuning Large Language Models
  • Memory-Efficient Fine-Tuning Techniques for Large Language Models
  • Training Ideas around Hyperparameters
    • Hugging Face documentation on loading PEFT
  • After fine tuning LLama3
  • Merging Model Weights
  • Merge Lora Instructions
  • Axolotl Configuration Files
    • Configuration Options
    • Model Configuration
    • Data Loading and Processing
    • Sequence Configuration
    • Lora Configuration
    • Logging
    • Training Configuration
    • Augmentation Techniques
  • Axolotl Fine-Tuning Tips & Tricks: A Comprehensive Guide
  • Axolotl debugging guide
  • Hugging Face Hub API
  • NCCL
  • Training Phi 1.5 - Youtube
  • JSON (JavaScript Object Notation)
  • General Tips
  • Datasets
Powered by GitBook
LogoLogo

This documentation is for the Axolotl community

On this page
  • Dependencies
  • Parse Requirements
  • Platform-specific requirements
  • The setup() function is called to configure the package

Was this helpful?

  1. Creation of Environment
  2. setup.py objectives

script analysis

Dependencies

The script starts by importing necessary modules:

  • platform module to retrieve information about the operating system.

  • re module for regular expressions.

  • PackageNotFoundError and version from importlib.metadata for handling package versions.

  • find_packages and setup from setuptools for package discovery and setup configuration.

Parse Requirements

The parse_requirements() function is defined to parse the requirements.txt file:

  • It initializes two lists: _install_requires for standard package requirements and _dependency_links for custom index URLs.

  • It opens the requirements.txt file, reads each line, and strips any whitespace.

  • It checks each line to determine if it is an extra requirement (e.g., "flash-attn", "deepspeed", etc.) or a standard package requirement.

  • If the line starts with "--extra-index-url", it extracts the URL and appends it to _dependency_links.

  • If the line is not an extra requirement and is not empty or a comment, it appends the line to _install_requires.

Platform-specific requirements

  • If the operating system is "Darwin" (macOS), it removes the "xformers==0.0.22" requirement from _install_requires.

  • If the operating system is not "Darwin", it retrieves the version of the "torch" package using version("torch").

  • It appends the "torch" package requirement with the specific version to _install_requires.

  • It parses the "torch" version using a regular expression to extract the major, minor, and patch version numbers.

  • If the "torch" version is greater than or equal to "2.1", it removes the "xformers==0.0.22" requirement and appends "xformers>=0.0.23" to _install_requires.

  • If the "torch" package is not found, it skips this step.

The setup() function is called to configure the package

  • It sets the package name to "axolotl".

  • It specifies the version as "0.4.0".

  • It provides a short description and a long description for the package.

  • It sets the package directory to "src".

  • It uses find_packages() to automatically discover packages within the "src" directory.

  • It sets the install_requires parameter to the install_requires list obtained from parse_requirements().

  • It sets the dependency_links parameter to the dependency_links list obtained from parse_requirements().

  • It defines extra requirements using the extras_require parameter, specifying additional dependencies for different features like "flash-attn", "deepspeed", "mamba-ssm", "auto-gptq", "mlflow", "lion-pytorch", and "galore".

Overall, this setup.py script is responsible for configuring the "axolotl" package, specifying its dependencies, handling platform-specific requirements, and defining extra requirements for optional features.

It provides a flexible and customisable setup for the package, allowing users to install the necessary dependencies based on their specific needs.

Previoussetup.py objectivesNextHuggingface Hub

Last updated 1 year ago

Was this helpful?

Page cover image