PromptConfig

class swift.tuners.prompt.PromptConfig(swift_type=None, model_key_mapping=None, dim=None, target_modules=None, embedding_pos=None, attention_mask_pos=None, attention_mask_value=0.0, prompt_length=16, attach_front=True, extract_embedding=False)[源代码]

The configuration class for the prompt module.

Visual prompt tuning (VPT) is proposed to initialize tunable prompt tokens and prepend to the original tokens in the first layer or multiple layers. 'Visual Prompt Tuning' by Jia et al.(2022) See https://arxiv.org/abs/2203.12119

Here we apply the VPT to other fields.

参数:

  • dim (Union[int, List[int]]) -- The dimension of the hidden states, use list if there are up-sample blocks or down-sample blocks

  • target_modules (str) -- The layer module to be replaced, in regex format

  • embedding_pos (Union[str, int]) -- The position of the embedding tensor

  • attention_mask_pos (Union[str, int]) -- The position of the attention mask

  • attention_mask_value (Union[float, int, bool]) -- The value to pad to the attention mask

  • prompt_length (int) -- The length of the prompt tokens

  • attach_front (bool) -- When set to True, prompt is attached in front of the embedding

  • extract_embedding (bool) -- Whether the embedding is extracted at final stage to keep the same dims with inputs