ResTuningConfig

class swift.tuners.restuning.ResTuningConfig(swift_type=None, model_key_mapping=None, dims=None, root_modules=None, root_modules_hook='input', stem_modules=None, stem_modules_hook='output', target_modules=None, target_modules_hook='input', target_hidden_pos=None, tuner_cfg=None, use_upsample=False, upsample_out_channels=None, zero_init_last=False, use_bypass=True)

The configuration class for the ResTuning module.

ResTuning is a flexible parameter-efficient and memory-efficient tuning paradigm framework. 'Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone' by Jiang et al.(2023) See

参数:

• dims (Union[List[int], int]) -- The dimensions of the hidden states

• root_modules (str) -- The root module to be replaced, can a regex string

• root_modules_hook (str) -- The hook type of root modules, can be "input" or "output"

• stem_modules (Union[List[str], str]) -- The stem modules to be replaced, can a regex string or name list of full match format

• stem_modules_hook (Union[List[str], str]) -- The hook type of stem modules, can be "input" or "output"

• target_modules (str) -- The target module to be replaced, can a regex string

• target_modules_hook (str) -- The hook type of target modules, can be "input" or "output"

• tuner_cfg (Union[List[Dict], Dict, str]) -- The configuration of the tuning module, can a string or customized config

• use_upsample (bool) -- Whether to use auxiliary upsample module

• upsample_out_channels (List[int]) -- The channels if use_upsample

• zero_init_last (bool) -- Use zero to initialize the last Linear in every sub tuner.