Submissions

A Submission is a submission of an inference system to a challenge by a team.

All constituent resources must already exist before creating a submission. Creating a Submission indicates that the participating team wishes to submit an entry as an official challenge submission. Most challenges limit the number of submissions that can be made in a given time interval.

Creating a Submission

Teams submit entries to specific tasks within a challenge using challenges.submit():

from dyff.schema.requests import SubmissionCreateRequest
from dyff.schema.platform import EntityIdentifier

# First, create an InferenceService to submit
inference_service = dyffapi.inferenceservices.create(...)

# Submit the service to a challenge task
submission = dyffapi.challenges.submit(
    challenge.id,
    task.id,
    SubmissionCreateRequest(
        account=account,
        team=team.id,
        submission=EntityIdentifier.of(inference_service),
    ),
)

Submission Fields

• team - ID of the Team making the submission

• submission - EntityIdentifier of the resource being submitted (typically an InferenceService)

• challenge - ID of the challenge (set automatically)

• task - Key of the task within the challenge (set automatically)

• pipelineRun - ID of the PipelineRun created to assess the submission (set automatically)

Submission Workflow

When a submission is created:

1. The system validates that the submitted resource meets the task’s execution environment requirements (CPU, memory, accelerators)

2. A PipelineRun is created to execute the task’s assessment pipeline

3. The pipeline evaluates the submitted system against the task’s criteria

4. Results are recorded and scores are computed

Resource Validation

For InferenceService submissions, the system validates that the service’s resource requests fit within at least one of the task’s defined execution environments. If the requested resources exceed all available environments, the submission is rejected.

Submission Limits

Tasks can define submission limits through their rules:

• submissionCycleDuration - Duration of each submission cycle (e.g., 1 day)

• submissionLimitPerCycle - Maximum submissions per team per cycle

• openingTime / closingTime - When submissions are accepted