Cloud-Based Causal Language Modeling Training
Learn about the clm_train_cloud function within Simplifine’s Train Engine for training models in the cloud
client.clm_train_cloud(
self,
job_name:str,
model_name:str,
dataset_name:str=None,
hf_token:str='',
dataset_config_name:str=None,
data_from_hf:bool=True,
do_split:bool=True,
split_ratio:float=0.2,
use_peft:bool=False,
lora_config:LoraConfig=None,
train_args:TrainingArguments=None,
data:dict={},
wandb_config:wandbConfig=None,
use_ddp:bool=False,
use_zero:bool=True,
prompt_config:PromptConfig=None
)
The name of the training job.
The name or path of the pre-trained model to use.
The name of the dataset to be used for training. Defaults to None.
The Hugging Face token required for accessing private datasets or models. Defaults to an empty string.
The configuration name of the dataset, if applicable. Defaults to None.
A flag to determine whether to load data from Hugging Face. Defaults to True.
A flag to determine whether to split the dataset into training and validation sets. Defaults to True.
The ratio of the dataset to be used for validation. Defaults to 0.2.
A flag to enable Parameter-Efficient Fine-Tuning (PEFT). Defaults to False.
The configuration for LoRA (Low-Rank Adaptation) if use_peft is True. Defaults to None.
The training arguments to customize the training process. Defaults to None.
A dictionary containing the training data. Defaults to an empty dictionary.
The configuration for Weights and Biases (WandB) logging. Defaults to None.
A flag to enable Distributed Data Parallel (DDP) training. Defaults to False.
A flag to enable ZeRO (Zero Redundancy Optimizer) for memory optimization. Defaults to True.
The configuration for prompts used in the training. Defaults to None.