Supervised Fine-Tuning (SFT)
Learn about the hf_sft function within Simplifine’s Train Engine
sft_train(
model_name:str,
dataset_name:str=None,
hf_token:str='',
dataset_config_name:str=None,
data_from_hf:bool=True,
do_split:bool=True,
split_ratio:float=0.2,
use_peft:bool=False,
lora_config:LoraConfig=None,
sft_config:SFTConfig=None,
data:dict={},
wandb_config:wandbConfig=None,
use_ddp:bool=False,
use_zero:bool=True, sft_prompt_config:sftPromptConfig=None
)
The name of the model to be trained.
The name of the dataset to be used for training. Defaults to None.
The Hugging Face token required for accessing private datasets or models. Defaults to an empty string.
The configuration name of the dataset, if applicable. Defaults to None.
A flag to determine whether to load data from Hugging Face. Set to True by default.
A flag to determine whether to split the dataset into training and validation sets. Set to True by default.
The ratio of the dataset to be used for validation. Defaults to 0.2.
A flag to enable Parameter-Efficient Fine-Tuning (PEFT). Defaults to False.
The configuration for LoRA (Low-Rank Adaptation) if use_peft is True. Defaults to None.
The configuration for supervised fine-tuning (SFT). Defaults to None.
A dictionary containing the training data. Defaults to an empty dictionary.
The configuration for Weights and Biases (WandB) logging. Defaults to None.
A flag to enable Distributed Data Parallel (DDP) training._ Defaults to False._
A flag to enable ZeRO (Zero Redundancy Optimizer) for memory optimization. Defaults to True.
The configuration for the SFT prompts. Defaults to None.