HubApi

class swift.hub.api.HubApi(endpoint=None)[源代码]

Model hub api interface.

__init__(endpoint=None)[源代码]

The ModelScope HubApi。

参数:

endpoint (str, optional) -- The modelscope server http|https address. Defaults to None.

login(access_token)[源代码]
Login with your SDK access token, which can be obtained from

https://www.modelscope.cn user center.

参数:

access_token (str) -- user access token on modelscope.

返回:

to authenticate yourself to ModelScope open-api git_token: token to access your git repository.

返回类型:

cookies

备注

You only have to login once within 30 days.

create_model(model_id, visibility=5, license='Apache License 2.0', chinese_name=None, original_model_id='')[源代码]

Create model repo at ModelScope Hub.

参数:
  • model_id (str) -- The model id

  • visibility (int, optional) -- visibility of the model(1-private, 5-public), default 5.

  • license (str, optional) -- license of the model, default none.

  • chinese_name (str, optional) -- chinese name of the model.

  • original_model_id (str, optional) -- the base model id which this model is trained from

返回:

Name of the model created

抛出:
  • InvalidParameter -- If model_id is invalid.

  • ValueError -- If not login.

返回类型:

str

备注

model_id = {owner}/{name}

delete_model(model_id)[源代码]

Delete model_id from ModelScope.

参数:

model_id (str) -- The model id.

抛出:

ValueError -- If not login.

备注

model_id = {owner}/{name}

get_model(model_id, revision=None)[源代码]

Get model information at ModelScope

参数:
  • model_id (str) -- The model id.

  • revision (str optional) -- revision of model.

返回:

The model detail information.

抛出:

NotExistError -- If the model is not exist, will throw NotExistError

返回类型:

str

备注

model_id = {owner}/{name}

push_model(model_id, model_dir, visibility=5, license='Apache License 2.0', chinese_name=None, commit_message='upload model', tag=None, revision='master', original_model_id=None, ignore_file_pattern=None)[源代码]

Upload model from a given directory to given repository. A valid model directory must contain a configuration.json file.

This function upload the files in given directory to given repository. If the given repository is not exists in remote, it will automatically create it with given visibility, license and chinese_name parameters. If the revision is also not exists in remote repository, it will create a new branch for it.

This function must be called before calling HubApi's login with a valid token which can be obtained from ModelScope's website.

参数:
  • model_id (str) -- The model id to be uploaded, caller must have write permission for it.

  • model_dir (str) -- The Absolute Path of the finetune result.

  • visibility (int, optional) -- Visibility of the new created model(1-private, 5-public). If the model is not exists in ModelScope, this function will create a new model with this visibility and this parameter is required. You can ignore this parameter if you make sure the model's existence.

  • license (str, defaults to None) -- License of the new created model(see License). If the model is not exists in ModelScope, this function will create a new model with this license and this parameter is required. You can ignore this parameter if you make sure the model's existence.

  • chinese_name (str, optional, defaults to None) -- chinese name of the new created model.

  • commit_message (str, optional, defaults to None) -- commit message of the push request.

  • tag (str, optional, defaults to None) -- The tag on this commit

  • revision (str, optional, default to DEFAULT_MODEL_REVISION) -- which branch to push. If the branch is not exists, It will create a new branch and push to it.

  • original_model_id (str, optional) -- The base model id which this model is trained from

  • ignore_file_pattern (Union[List[str], str], optional) -- The file pattern to ignore uploading

抛出:
  • InvalidParameter -- Parameter invalid.

  • NotLoginException -- Not login

  • ValueError -- No configuration.json

  • Exception -- Create failed.

list_models(owner_or_group, page_number=1, page_size=10)[源代码]

List models in owner or group.

参数:
  • owner_or_group (str) -- owner or group.

  • page_number (int, optional) -- The page number, default: 1

  • page_size (int, optional) -- The page size, default: 10

抛出:

RequestError -- The request error.

返回:

{"models": "list of models", "TotalCount": total_number_of_models_in_owner_or_group}

返回类型:

dict

list_model_revisions(model_id, cutoff_timestamp=None, use_cookies=False)[源代码]

Get model branch and tags.

参数:
  • model_id (str) -- The model id

  • cutoff_timestamp (int) -- Tags created before the cutoff will be included. The timestamp is represented by the seconds elapsed from the epoch time.

  • use_cookies (Union[bool, CookieJar], optional) -- If is cookieJar, we will use this cookie, if True, will load cookie from local. Defaults to False.

返回:

Return list of branch name and tags

返回类型:

Tuple[List[str], List[str]]

get_model_branches_and_tags(model_id, use_cookies=False)[源代码]

Get model branch and tags.

参数:
  • model_id (str) -- The model id

  • use_cookies (Union[bool, CookieJar], optional) -- If is cookieJar, we will use this cookie, if True, will load cookie from local. Defaults to False.

返回:

Return list of branch name and tags

返回类型:

Tuple[List[str], List[str]]

get_model_files(model_id, revision=None, root=None, recursive=False, use_cookies=False, headers={})[源代码]

List the models files.

参数:
  • model_id (str) -- The model id

  • revision (Optional[str], optional) -- The branch or tag name.

  • root (Optional[str], optional) -- The root path. Defaults to None.

  • recursive (Optional[str], optional) -- Is recursive list files. Defaults to False.

  • use_cookies (Union[bool, CookieJar], optional) -- If is cookieJar, we will use this cookie, if True, will load cookie from local. Defaults to False.

  • headers (dict | None) -- request headers

返回:

Model file list.

返回类型:

List[dict]

static fetch_meta_files_from_url(url, out_path, chunk_size=1024, mode='reuse_dataset_if_exists')[源代码]

Fetch the meta-data files from the url, e.g. csv/jsonl files.