> For the complete documentation index, see [llms.txt](https://axolotl.continuumlabs.pro/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://axolotl.continuumlabs.pro/general-tips.md).

# General Tips

### <mark style="color:blue;">General Tips</mark>

While debugging it’s helpful to simplify your test scenario as much as possible. Here are some tips for doing so:

<mark style="color:yellow;">**Make sure you are using the latest version of axolotl**</mark>

This project changes often and bugs get fixed fast. Check your git branch and make sure you have pulled the latest changes from <mark style="color:yellow;">`main`</mark>.

### <mark style="color:green;">**Eliminate concurrency**</mark>

Restrict the number of processes to 1 for both training and data pre-processing:

* Set `CUDA_VISIBLE_DEVICES` to a single GPU, ex: `export CUDA_VISIBLE_DEVICES=0`.
* Set `dataset_processes: 1` in your axolotl config or run the training command with `--dataset_processes=1`.

### <mark style="color:green;">**Use a small dataset**</mark>

Construct or use a small dataset from HF Hub.&#x20;

When using a small dataset, you will often have to make sure <mark style="color:yellow;">`sample_packing: False`</mark> and <mark style="color:yellow;">`eval_sample_packing: False`</mark> to avoid errors.&#x20;

If you are in a pinch and don’t have time to construct a small dataset but want to use from the Huggingface Hub, you can <mark style="color:yellow;">shard the data</mark> (this will still tokenize the entire dataset, but will only use a fraction of the data for training.&#x20;

For example, to shard the dataset into 20 pieces, add the following to your axolotl config):&#x20;

<mark style="color:yellow;">`yaml dataset: ... shards: 20`</mark>

### <mark style="color:green;">**Use a small model**</mark>

A good example of a small model is [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).

### <mark style="color:green;">**Minimize iteration time**</mark>

Make sure the training loop finishes as fast as possible, with these settings.

* <mark style="color:yellow;">`micro_batch_size: 1`</mark>
* <mark style="color:yellow;">`max_steps: 1`</mark>
* <mark style="color:yellow;">val\_set\_size</mark><mark style="color:yellow;">`: 0`</mark>

### <mark style="color:green;">**Clear Caches**</mark>

Axolotl caches certain steps and so does the underlying HuggingFace trainer.&#x20;

You may want to clear some of these caches when debugging.

* <mark style="color:blue;">Data pre-processing:</mark> When debugging data pre-processing, which includes prompt template formation, you may want to delete the directory set in <mark style="color:yellow;">`dataset_prepared_path:`</mark> in your axolotl config. If you didn’t set this value, the default is <mark style="color:yellow;">`last_run_prepared`</mark><mark style="color:yellow;">.</mark>
* Huggingface Hub: If you are debugging data pre-processing, you should clear the relevant HF cache [HuggingFace cache](https://huggingface.co/docs/datasets/cache), by deleting the appropriate <mark style="color:yellow;">`~/.cache/huggingface/datasets/...`</mark> folder(s).
* The recommended approach is to redirect all outputs and caches to a temporary folder and delete selected subfolders before each run. This is demonstrated in the example configuration below.