Challenges

This section covers the Dyff Challenges feature, which enables organized competitions where teams submit AI systems to be assessed against standardized benchmarks.

A Challenge is a collection of assessments on which participating teams compete to achieve the best performance.

Challenge Workflow

The challenge workflow involves the following resources:

  • Challenge - The competition itself, containing one or more tasks

  • ChallengeTask - Individual assessments within a challenge (embedded in Challenge)

  • Team - Groups of participants who enter and compete in challenges

  • Submission - Entries submitted by teams to challenge tasks

The typical flow is:

  1. A challenge organizer creates a Challenge with one or more ChallengeTask items

  2. Teams register to participate by creating a Team entity

  3. Teams submit their AI systems (typically InferenceService resources) as Submission entries

  4. Each submission triggers an assessment Pipeline that evaluates the submitted system

  5. Results are collected and scores are computed

Prerequisites

You must use the Dyff API Client to work with challenges. You will need an API token that grants the necessary permissions.