Re0-01: HuggingFace Transformers Trainer

Hyacehila
  2025-12-27 14:00 2025-12-27 14:00 Created      

The questions in this article can also be addressedRe0-02 : HuggingFace TRL SFTTrainer、Re0-03 : HuggingFace TRL DPOTrainerHow the concept of a relatively close read together is developed in different contexts.

Auxiliary code:trainer.py

Foreword: Why do we need to fine-tune?

This set begins with a runable fine-tuning process.

Pre-trained large language models have read a large number of public texts, but these borders are still encountered when used directly:

  • He may not know the rules of your company.
  • It may not know your particular field of expertise.
  • It may not answer the question in the form you want.

Fine-twining Use task data to bind modeling to bring output closer to the specific scene.

Learning objectives of this chapter

This chapter covers:

  1. Hugging Face Ecosystem Basic composition
  2. Oversight fine-tuning (SFT) Basic concepts
  3. Loss Masking Role in SFT
  4. Quantified and PEFT/LoRA How can a normal graphic card fine-tune a model?
  5. DataCollator Role and selection

1. Hugging Face Ecosystem Overview

1.1 Core components

Hugging Face Ecosystem Architecture

graph TD
    A[Transformers<br/>模型加载] --> C[Trainer<br/>训练循环管理]
    B[Datasets<br/>数据加载] --> C
    D[PEFT<br/>参数高效微调] --> C
Component Role Use of this tutorial
transformers Load and use pre-training models [Yes]
datasets Loading and processing data sets [Yes]
peft Efficient fine-tuning of parameters (LoRA et al.) [Yes]
bitsandbytes Quantification of models (4-bit/8-bit) [Yes]
trl Enhanced learning and advanced training Next chapter

1.2 What's Trainer?

Trainer It's a training manager from Hugging Face, which helps you handle:

  • Training cycle (forward, reverse, updated)
  • Assessment and validation
  • Model saving and loading
  • Log Log Log
  • Distributive training

The data and configuration are ready.Trainer The main training process will be taken over.

2. Core concept of oversight fine-tuning (SFT)

2.1 What's SFT?

Supervised Fine-Tuning, SFT It is a process of training models with labeled data.

In the context of the dialogue mandate, our data are usually as follows:

Example of dialogue:

  • User: Please help me summarize the content of this conversation...
  • Assistant: This conversation focused on...

2.2 Los Masking: SFT core

Attention. This is the most important concept of this chapter!

In training, the model needs to predict the next token. But the question is:

  • Models should learn.Generate Answers, not learning.Generate problem
  • If you calculate the whole sequence, Los, the model will try to"Reread"User problems

Solutions: Losing Masking

Enter Sequence [User questions] [Adviser Answers]
Tab [-100] [actual token]
Los Calculator [No] No calculation [Yes] Calculate
  • labels = -100 This is where it is.Do not calculate, Loss(Default Ignored value for PyTorch)
  • Only part of the assistant's answer will be used for training.

2.3 Loss Masking in code

Yes. trainer.py Yes. process_func() function:

# 1. 初始化所有 labels 为 -100（全部忽略）
labels = [-100] * len(input_ids)


2. 计算 prompt 长度
prompt_len = len(prompt_ids)


3. 只有 prompt 之后的部分设置为实际的 token ID
labels[prompt_len:] = input_ids[prompt_len:]

Effects：

  • User problem segment:labels = [-100, -100, -100, ...] ♪ No count, Loss ♪
  • The assistant answers the section:labels = [token1, token2, token3, ...] → Calculating Los

3. Allowing the general graphic card to run large models: quantification and LoRA

3.1 Quantification

Problem: A model of 7B parameters that requires 28GB display in FP32 storage!

Solutions: Compress parameters from high to low accuracy

Precision Number of places per parameter 7B model size Speed
FP32 32 bit 28 GB Benchmark
FP16 16 bit 14 GB Faster.
INT8 8 bit 7 GB Faster.
INT4 4 bit 3.5 GB Fastest

Quantification in code using 4-bit:

from transformers import BitsAndBytesConfig


quant_config = BitsAndBytesConfig(
    load_in_4bit=True,              # 使用 4-bit 量化
    bnb_4bit_quant_type="nf4",      # NF4 量化类型（性能更好）
    bnb_4bit_compute_dtype=torch.float16,  # 计算时用 FP16
)


model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    quantization_config=quant_config,
    device_map="auto"
)

3.2 Why is the training of full parameters after quantification still in need of significant visibility?

It's easy to misperception.

The common judgement is that:"The model has been quantified as 4-bit. It should be less visible when training."

But quantifying the main reduction of the model weightStorage space, and during training, the contents of the material are stored in addition to the weights.

Visible occupancy details

Here. 8B Parameter ModelThe following examples of full-parameter training are provided:

Component Size Annotations
Model weights (after quantification) 4 GB 8B x 0.5 bytes (INT4) = 4 GB
Gradients 16 GB 8B x 2 bytes (FP16) = 16 GB
3. Optimizer status 64 GB 8B x 4 bytes (FP32) x 2 = 64 GB
4. Copies of parameters 16 GB 8B x 2 bytes (FP16) = 16 GB
Total 100 GB It's a far more common consumption scale!

Why do you need so much?

  1. Gradients cannot be quantified!

    • Gradient for updating parameters, FP16/FP32 accuracy must be maintained
    • If the gradient is 4-bit, the re-engineering of the parameters will be seriously distorted and the model will not be able to contain

  2. Optimizer state cannot be quantified!

    • AdamW Optimizer requires storage of first-order kinetic (m) and second-stage kinetic (v)
    • These movements must remain FP32 precisions, otherwise the optimal effects will decline dramatically.

  3. A copy of the parameters requires high precision!

    • FP16/FP32 precision is required to calculate the updated parameter
    • Could not perform numerical operations directly on 4-bit weights

Conclusions

Quantification is just a model."Load"It's easier, but..."Training"Still need a lot of visibility!

To effectively reduce the visibility of training, it is necessary to reduce the number of trainingable parameters.

3.3 LoRA（Low-Rank Adaptation）

Problem: even after quantification, fine-tuning of all parameters will require significant visibility (as noted above, approximately 100 GB)

Solutions: only a small part of the parameters is trained

The Remarkable Advantage of LoRA

Same 8B model, using LoRA training:

Component Training in full parameters LoRA training
Quantified weight (freeze) 4 GB 4 GB
Trainable Parameters 16 GB 0.16 GB (1%)
Gradient 16 GB 0.16 GB (1%)
Optimizer status 64 GB 0.64 GB (1%)
Total 100 GB ~5 GB + Activate Value

ConclusionsLoRA reduces the need for visibility from 100 GB to about 10-15 GB (with active value), and normal graphic cards can be trained!

LoRA works well：

原始模型权重 W（冻结，不训练）
    维度：d × d（例如 4096 × 4096）


LoRA 适配器（可训练）：
    A：d × r（例如 4096 × 64）
    B：r × d（例如 64 × 4096）


最终输出 = W·x + α·(B·A)·x
              ↑        ↑
           原始输出  LoRA调整

Key points：

  • Original weight W frozen, not involved in training
  • Train only two small arrays A and B(r) << d, normally r = 8, 16, 32, 64)
  • The result of B.A. is a low-level matrix, which is a primary weight."Adjustments"
  • Alpha is a scaling factor, controls the level of impact of LoRA

Parameter Volume Comparison：

  • Original: 4096 x 4096 = 16,777,216 parameters
  • LoRA: 4096 x 64 + 64 x 4096 = 524,288 Parameters (%)97% reduction!）

Code configuration:

from peft import LoraConfig, get_peft_model


lora_config = LoraConfig(
    r=64,           # LoRA 秩（越大参数越多）
    lora_alpha=16,  # 缩放因子
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # 应用到哪些层
    lora_dropout=0.01,
    task_type="CAUSAL_LM"
)


model = get_peft_model(model, lora_config)
model.print_trainable_parameters()


输出：trainable params: 83,886,080 || all params: 8,108,634,112 || trainable%: 1.0346%

4. DataCollarator: Data Collator

4.1 What's DataColator?

DataCollator It's responsible for combining multiple samples into one of the bats, only filled in an equal length structure.torchThe matrix method in place is designed to accelerate training, each of which uses a one-dimensional set of two-dimensional set of tensor, then backward positive transmission, loss calculation and reverse transmission:

样本1: [1, 2, 3, 4]        →  padding  →  [1, 2, 3, 4, 0, 0]
样本2: [5, 6, 7, 8, 9, 10] →  不变    →  [5, 6, 7, 8, 9, 10]
                              ↓
                         组成 batch

4.2 Two key DataColator

DataCollator Apply scene Labors Process
DataCollatorForLanguageModeling Pre-training Auto Copy input_ids As labels
DataCollatorForSeq2Seq SFT Keep the pre-treatments. labels- 100--

Attention. Important: SFT must be used DataCollatorForSeq2SeqBecause it respects your pre-treatment. labelsThe government has been working on the issue of the Internet.DataCollatorForLanguageModeling I am not a man.

5. Complete training process

5.1 Flowchart

SFT Training Complete Process

  1. Loading data sets

    dataset = load_dataset("数据集名称")

  2. Load Model + Quantification

    model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    quantization_config=quant_config,
    device_map="auto"
)

  3. Data preprocessing (los Masking)

    train_dataset = preprocess_dataset(tokenizer, max_length, seed, dataset['train'])

  4. Add LoRA adapter

    model = get_peft_model(model, lora_config)

  5. Configure Trainer

    trainer = Trainer(model, dataset, data_collator, args)

  6. Start training.

    trainer.train()

  7. Save Model

    trainer.save_model()

5.2 Summary of key codes

# 1. 加载数据集
dataset = load_dataset("neil-code/dialogsum-test")


2. 加载量化模型
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    quantization_config=quant_config,
    device_map="auto"
)


3. 数据预处理（应用 Loss Masking，在代码文件中对此有着更加详细的解释）
train_dataset = preprocess_dataset(tokenizer, max_length, seed, dataset['train'])


4. 添加 LoRA
model = prepare_model_for_kbit_training(model)
model = get_peft_model(model, lora_config)


5. 配置训练器。在代码中包含了不同的DataCollator方便进一步对这里理解，如果只需要使用，则可以直接使用推荐配置
trainer = Trainer(
    model=model,
    train_dataset=train_dataset,
    data_collator=DataCollatorForSeq2Seq(tokenizer),
    args=training_args
)


6. 训练
trainer.train()

6. Operational code

6.1 Environmental readiness

pip install transformers datasets peft bitsandbytes accelerate

If you use uv Managing the environment. code/ Home build pyproject.toml;notice torch Matches CUDA version.

6.2 Operational training

python trainer.py

6.3 Modification of Configuration

Yes. trainer.py Bottom found configuration parameters:

COLLATOR_TYPE = "visual_check"  # 推荐用于学习，会打印详细信息
MODEL_PATH = "Qwen/Qwen3-8B"
MAX_STEPS = 40  # 训练步数

Summary of this chapter

Concept Annotations
SFT Calculate loss for each subsequent token using a calibrated data model instead of pre-trained
Loss Masking Only the assistant, Ross, is the core difference between SFT and pre-training.
Quantified Compress the size of the model parameters to store space so that the normal graphic card can be placed Down
LoRA Trained only a few parameters, further significantly reducing the need for visibility
DataCollator The sample is a watch that needs to be understood more than Los.

8. Practice hands-on

  1. Run trainer.pyWatch. VisualCheckDataCollator Print information
  2. Try to be COLLATOR_TYPE For "seq2seq",comparison output
  3. Modify MAX_STEPS, observe changes in training time and effectiveness

Next chapter.

In this chapter, we have made it happen manually, Los Masking, which requires:

  • Calculate prompt length
  • Manual settings
  • Select the correct DataCollor

Problem: Can we automate all of this?

Answer: Yes!

In the next chapter, we'll learn. SFTTrainer from TRL LibraryIt can:

  • Autoprocessing Losing
  • Autoselect DataCollor
  • Start flash Attention acceleration

Appendix: Common problems

Q: Why can't I stop talking after my model training?
A: Probably EOS token was not correctly added during preprocessing of data. Use apply_chat_template It's automatic.

What if there's not enough to show?
A: 1) Reduce the value of LoRA by the value of r

References

  • Title: Re0-01: HuggingFace Transformers Trainer
  • Author: Hyacehila
  • Created at : 2025-12-27 14:00:00
  • Link: https://hyacehila.github.io//blog/2025/12/27/Re0HF-01/
  • License: This work is licensed under CC BY-NC-SA 4.0.
Comments
On this page
Re0-01: HuggingFace Transformers Trainer
  1. Foreword: Why do we need to fine-tune?
  2. Learning objectives of this chapter
  3. 1. Hugging Face Ecosystem Overview
    1. 1.1 Core components
    2. 1.2 What's Trainer?
  4. 2. Core concept of oversight fine-tuning (SFT)
    1. 2.1 What's SFT?
    2. 2.2 Los Masking: SFT core
    3. 2.3 Loss Masking in code
  5. 2. 计算 prompt 长度
  6. 3. 只有 prompt 之后的部分设置为实际的 token ID
    1. 3. Allowing the general graphic card to run large models: quantification and LoRA
      1. 3.1 Quantification
      2. 3.2 Why is the training of full parameters after quantification still in need of significant visibility?
      3. 3.3 LoRA（Low-Rank Adaptation）
  7. 输出：trainable params: 83,886,080 || all params: 8,108,634,112 || trainable%: 1.0346%
    1. 4. DataCollarator: Data Collator
      1. 4.1 What's DataColator?
      2. 4.2 Two key DataColator
    2. 5. Complete training process
      1. 5.1 Flowchart
      2. 5.2 Summary of key codes
  8. 2. 加载量化模型
  9. 3. 数据预处理（应用 Loss Masking，在代码文件中对此有着更加详细的解释）
  10. 4. 添加 LoRA
  11. 5. 配置训练器。在代码中包含了不同的DataCollator方便进一步对这里理解，如果只需要使用，则可以直接使用推荐配置
  12. 6. 训练
    1. 6. Operational code
      1. 6.1 Environmental readiness
      2. 6.2 Operational training
      3. 6.3 Modification of Configuration
    2. Summary of this chapter
    3. 8. Practice hands-on
    4. Next chapter.
    5. References