AdapterConfig

class swift.tuners.adapter.AdapterConfig(swift_type=None, model_key_mapping=None, dim=None, target_modules=None, hidden_pos=None, method_name='forward', adapter_length=128, act_layer='gelu')

The configuration class for the adapter module.

Adapters project input tokens by an MLP layer. 'Parameter-Efficient Transfer Learning for NLP' by Houlsby et al.(2019) See http://arxiv.org/abs/1902.00751

参数:

• dim (int) -- The dimension of the hidden states

• target_modules (Union[str, List[str]]) -- The feedforward module to be replaced. in regex format if this argument is str, else will match with end with if List[str].

• hidden_pos (Union[str, int]) -- The position of the hidden state to be passed into the adapter, can be int (args) or str (kwargs)

• method_name (str) -- The method to be replaced, default is forward

• adapter_length (int) -- The length of the adapter length (intermediate length)

• act_layer (str) -- The activation layer of the adapter