# General Tips

### <mark style="color:blue;">General Tips</mark>

While debugging it’s helpful to simplify your test scenario as much as possible. Here are some tips for doing so:

<mark style="color:yellow;">**Make sure you are using the latest version of axolotl**</mark>

This project changes often and bugs get fixed fast. Check your git branch and make sure you have pulled the latest changes from <mark style="color:yellow;">`main`</mark>.

### <mark style="color:green;">**Eliminate concurrency**</mark>

Restrict the number of processes to 1 for both training and data pre-processing:

* Set `CUDA_VISIBLE_DEVICES` to a single GPU, ex: `export CUDA_VISIBLE_DEVICES=0`.
* Set `dataset_processes: 1` in your axolotl config or run the training command with `--dataset_processes=1`.

### <mark style="color:green;">**Use a small dataset**</mark>

Construct or use a small dataset from HF Hub.&#x20;

When using a small dataset, you will often have to make sure <mark style="color:yellow;">`sample_packing: False`</mark> and <mark style="color:yellow;">`eval_sample_packing: False`</mark> to avoid errors.&#x20;

If you are in a pinch and don’t have time to construct a small dataset but want to use from the Huggingface Hub, you can <mark style="color:yellow;">shard the data</mark> (this will still tokenize the entire dataset, but will only use a fraction of the data for training.&#x20;

For example, to shard the dataset into 20 pieces, add the following to your axolotl config):&#x20;

<mark style="color:yellow;">`yaml dataset: ... shards: 20`</mark>

### <mark style="color:green;">**Use a small model**</mark>

A good example of a small model is [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).

### <mark style="color:green;">**Minimize iteration time**</mark>

Make sure the training loop finishes as fast as possible, with these settings.

* <mark style="color:yellow;">`micro_batch_size: 1`</mark>
* <mark style="color:yellow;">`max_steps: 1`</mark>
* <mark style="color:yellow;">val\_set\_size</mark><mark style="color:yellow;">`: 0`</mark>

### <mark style="color:green;">**Clear Caches**</mark>

Axolotl caches certain steps and so does the underlying HuggingFace trainer.&#x20;

You may want to clear some of these caches when debugging.

* <mark style="color:blue;">Data pre-processing:</mark> When debugging data pre-processing, which includes prompt template formation, you may want to delete the directory set in <mark style="color:yellow;">`dataset_prepared_path:`</mark> in your axolotl config. If you didn’t set this value, the default is <mark style="color:yellow;">`last_run_prepared`</mark><mark style="color:yellow;">.</mark>
* Huggingface Hub: If you are debugging data pre-processing, you should clear the relevant HF cache [HuggingFace cache](https://huggingface.co/docs/datasets/cache), by deleting the appropriate <mark style="color:yellow;">`~/.cache/huggingface/datasets/...`</mark> folder(s).
* The recommended approach is to redirect all outputs and caches to a temporary folder and delete selected subfolders before each run. This is demonstrated in the example configuration below.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://axolotl.continuumlabs.pro/general-tips.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.