Causal Language Modeling Training
Learn about the clm_train
function within Simplifine’s Train Engine
clm_train(
model_name:str,
dataset_name:str=None,
hf_token:str='',
dataset_config_name:str=None,
data_from_hf:bool=True,
do_split:bool=True,
split_ratio:float=0.2,
use_peft:bool=False,
lora_config:LoraConfig=None,
train_args:TrainingArguments=None,
data:dict={},
wandb_config:wandbConfig=None,
use_ddp:bool=False,
use_zero:bool=True,
prompt_config:PromptConfig=None
)
The name or path of the pre-trained model to use.
The name of the dataset to be used for training. Defaults to None
.
The Hugging Face token required for accessing private datasets or models. Defaults to an empty string.
The configuration name of the dataset, if applicable. Defaults to None
.
A flag to determine whether to load data from Hugging Face. Defaults to True
.
A flag to determine whether to split the dataset into training and validation sets. Defaults to True
.
The ratio of the dataset to be used for validation. Defaults to 0.2
.
A flag to enable Parameter-Efficient Fine-Tuning (PEFT). Defaults to False
.
The configuration for LoRA (Low-Rank Adaptation) if use_peft
is True
. Defaults to None
.
The training arguments to customize the training process. Defaults to None
.
A dictionary containing the training data. Defaults to an empty dictionary.
The configuration for Weights and Biases (WandB) logging. Defaults to None
.
A flag to enable Distributed Data Parallel (DDP) training. Defaults to False
.
A flag to enable ZeRO (Zero Redundancy Optimizer) for memory optimization. Defaults to True
.
The configuration for prompts used in the training. Defaults to None
.