LoRAConfig

class swift.tuners.lora.LoRAConfig(swift_type=None, model_key_mapping=None, peft_type=None, auto_mapping=None, base_model_name_or_path=None, revision=None, task_type=None, inference_mode=False, r=8, target_modules=None, lora_alpha=8, lora_dropout=0.0, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern=<factory>, alpha_pattern=<factory>, megatron_config=None, megatron_core='megatron.core', loftq_config=<factory>, use_dora=False, layer_replication=None, runtime_config=<factory>, lora_dtype=None, lorap_lr_ratio=16.0, lorap_emb_lr=1e-06, use_qa_lora=False, use_merged_linear=False, enable_lora=None)[源代码]

The configuration class for the loRA module.

参数:

use_qa_lora (bool) -- Use QA-LoRA:[Quantization-Aware Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2309.14717) instead of LoRA. QA-LoRA only supports AutoGPTQ quantized models. Deprecated, do not use this argument.
lora_dtype (str) -- The dtype for all lora modules, supported values are fp32, fp16, bf16. Default value is None, which means follow the dtype of original module's weight.
lorap_lr_ratio (float) -- The lr_ratio argument for [LoRA+](https://arxiv.org/abs/2402.12354)