Dyff¶
Dyff is a cloud platform for running evaluations of AI/ML systems. The advantages of Dyff are scalable and reproducible evaluations without revealing information about test datasets or systems under test.
Why is Dyff important?¶
Between new regulations on AI/ML and increased consumer awareness of its shortcomings, we are moving toward a world where verifiable performance scores for AI/ML systems are economically valuable. Dyff demonstrates a path toward an economically sustainable evaluation ecosystem.
Scalable and reproducible: Thorough evaluations need to be a routine part of system deployment. Dyff provides the infrastructure to support this.
Data privacy: It is easy to score well on any given evaluation by training on the test data. Dyff runs evaluations without revealing any information about the test data, prolonging the data’s useful life and giving low marginal cost for evaluations.
System privacy: Test data privacy requires that test data never leave the Dyff system. This means that system creators must submit their systems to run on Dyff, too. These systems can be extremely valuable business assets. Raw outputs from a system could be used to reverse-engineer it. Dyff does not reveal any information about the system under test. This also protects against malicious test systems that reveal input data in their outputs.