Uploading AI Models

The dyff-api repository contains scripts to aid in uploading new models in the scripts/ directory. The main script is generate-huggingface-resource-scripts.py. This script should be run from inside the scripts/ directory.

The script itself generates new scripts that, when run, create the Model, InferenceService, and InferenceSession resource associated with the new AI model. The general workflow is:

  1. Generate the Model creation script

  2. Run the Model creation script and note the ID of the created model

  3. Generate the InferenceService creation script, providing the Model ID

  4. Run the InferenceService creation script and note the ID of the created service

  5. Generate the InferenceSession creation script, providing the InferenceService ID

  6. Run the InferenceSession creation script. This script enters a polling loop until the session is up, then runs a test request to make sure the system is working. It does not terminate the session after completion.

Command-line options

The available command-line options are documented in the generate-huggingface-resource-scripts.py file.

In each of these steps you should keep the command-line options from the previous step and add the additional options needed by the current step. Some of the earlier options are not needed in later steps, but there’s no harm in keeping them.

Prerequisites

Before running any scripts, set the required environment variables in your terminal:

export DYFF_API_TOKEN=<your-api-token>

# If using Staging
export DYFF_API_ENDPOINT=https://staging-api.dyff.io/v0

# If using Production
export DYFF_API_ENDPOINT=https://api.dyff.io/v0