Dyff¶
Dyff is a cloud platform for creating and running analyses of AI systems. The advantages of Dyff are scalable and reproducible evaluations without revealing information about test datasets or systems under test.
Why is Dyff important?¶
Between new regulations on AI/ML and increased consumer awareness of its shortcomings, we are moving toward a world where verifiable performance scores for AI/ML systems are economically valuable. Dyff demonstrates a path toward an economically sustainable evaluation ecosystem.
Scalable and reproducible: Thorough evaluations need to be a routine part of system deployment. Dyff provides the infrastructure to support this.
Understandable results for non-expert stakeholders: Dyff includes a Web app that facilitates straightfoward presentation of the important conclusions of AI system evaluations and makes the results discoverable at stable URIs.
Data privacy: It is easy to score well on any given evaluation by training on the test data. Dyff runs evaluations without revealing any information about the test data, prolonging the data’s useful life and giving low marginal cost for evaluations.
System privacy: Test data privacy requires that test data never leave the Dyff system. This means that system creators must submit their systems to run on Dyff, too. These systems can be extremely valuable business assets. Raw outputs from a system could be used to reverse-engineer it. Dyff does not reveal any information about the system under test. This also protects against malicious test systems that reveal input data in their outputs.