Resource Families

Another way of organizing resources is to create a resource Family. A Family is like a namespace for a collection of resources of the same type. In the context of the Family, you can create string “tags” that are aliases for specific entities, and you can add, change, and delete tags freely. If you’re familiar with Docker, the concept is similar to tagging Docker images.

One important use case for families is resource versioning. For example, if you have a test dataset for measuring a certain property and you need to update it periodically to maintain its validity, you could create a Family of Dataset resources and tag each version 1.0, 1.1, 2.0, or 2025-Jan, 2025-Feb, etc.

Note

If you use families for resource versioning, you should not change what a version tag points to after that tag has been created. Apply the same best practices that are recommended for Docker image tags. You can use (mutable) keyword tags like latest to identify versions that are important at a given moment.

Create a Family

Each Family contains “members” of a single kind, which you specify when you create the Family and which cannot be changed. Some resource kinds cannot be family members, such as workflows with a “lifetime” like Evaluation and InferenceSession.

family = dyffapi.families.create(
    FamilyCreateRequest(
        account=account,
        memberKind=FamilyMemberKind.Dataset,
    )
)

Edit family members

The edit_members() function is used to assign tags to entities. Assigning a value to an existing key overwrites the existing value.

dataset: Dataset = ...
dataset_tiny: Dataset = ...

dyffapi.families.edit_members(
    family.id,
    {
        "regular": FamilyMemberBase(
            entity=EntityIdentifier.of(dataset), description="Regular size"
        ),
        "tiny": FamilyMemberBase(
            entity=EntityIdentifier.of(dataset_tiny), description="Tiny size"
        ),
    },
)

Retrieve a tagged entity

You can retrieve a member of a family in a single operation.

family_id: str = ...
entity: DyffEntityType = dyffapi.families.get_member(family_id, "tag")

The return type matches the .memberKind property of the Family. For example, if family.memberKind == "Dataset", then the returned entity will be a Dataset.

Note

The .get_member() function will return the entity that currently has the given tag in the version of the Family that exists in the Dyff datastore. This may be a different entity than the one referenced in any local copies of the Family that you have.

If you want to retrieve the entity that is referenced in a specific (local) Family instance, you can use code like the following:

member = family.members["tag"]
entity = dyffapi.get(member.entity)

Delete family members

To delete a family member, set its value to None.

dyffapi.families.edit_members(family.id, {"regular": None})