FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Matthew Arnold,Rachel K. E. Bellamy,Michael Hind,Stephanie Houde,Sameep Mehta,Aleksandra Mojsilovic,Ravi Nair,K. Natesan Ramamurthy,Alexandra Olteanu,David Piorkowski,Darrell C. Reimer,John T. Richards,Jason Tsay,Kush R. Varshney +13 more
443
TL;DR: This paper envisiones an SDoC for AI services to contain purpose, performance, safety, security, and provenance information to be completed and voluntarily released by AI service providers for examination by consumers.
read more
Abstract: Accuracy is an important concern for suppliers of artificial intelligence (AI) services, but considerations beyond accuracy, such as safety (which includes fairness and explainability), security, and provenance, are also critical elements to engender consumers’ trust in a service. Many industries use transparent, standardized, but often not legally required documents called supplier's declarations of conformity (SDoCs) to describe the lineage of a product along with the safety and performance testing it has undergone. SDoCs may be considered multidimensional fact sheets that capture and quantify various aspects of the product and its development to make it worthy of consumers’ trust. In this article, inspired by this practice, we propose FactSheets to help increase trust in AI services. We envision such documents to contain purpose, performance, safety, security, and provenance information to be completed by AI service providers for examination by consumers. We suggest a comprehensive set of declaration items tailored to AI in the Appendix of this article.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Signs of the Time: Making AI Legible
Joseph Lindley,Paul Coulton,Haider Ali Akmal,Franziska Pilling +3 more
- 12 Aug 2020
TL;DR: This paper argues that design’s role in improving AI legibility is critical and presents the presentation of prototypical icons designed to enhanceAI legibility.
•Posted Content
Green Lighting ML: Confidentiality, Integrity, and Availability of Machine Learning Systems in Deployment.
Abhishek Gupta,Erick Galinkin +1 more
TL;DR: Automated systems for validating privacy and security of models need to be developed, which will help to lower the burden of implementing these hand-offs and increasing the ubiquity of their adoption.
4
More than Model Documentation: Uncovering Teachers' Bespoke Information Needs for Informed Classroom Integration of ChatGPT
Hariharan Subramonyam
- 11 May 2024
TL;DR: Teachers face significant information gaps when integrating ChatGPT in classrooms, lacking clarity on its capabilities for bespoke learning tasks and diverse learners' needs, highlighting the need for interactive model documentation to bridge pedagogical and technical knowledge.
4
Workflow provenance in the lifecycle of scientific machine learning
Renan Souza,Leonardo Guerreiro Azevedo,Vítor Lourenço,Elton F. de S. Soares,Raphael Melo Thiago,Rafael Brandão,Daniel Civitarese,Emilio Vital Brazil,Marcio Ferreira Moreno,Patrick Valduriez,Marta Mattoso,Renato Cerqueira,Marco A. S. Netto +12 more
TL;DR: In this article, the authors leverage workflow provenance techniques to build a holistic view to support the lifecycle of scientific ML, with a W3C PROV compliant data representation and a reference system architecture.
Proceedings Article
Learning From Arabic Corpora But Not Always From Arabic Speakers: A Case Study of the Arabic Wikipedia Editions
TL;DR: In this article, the authors present a case study focusing specifically on differences among the Arabic Wikipedia editions (Modern Standard Arabic, Egyptian, and Moroccan) and document issues in the Egyptian Arabic Wikipedia with automatic creation/generation and translation of content pages from English without human supervision.
4
References
The Market for “Lemons”: Quality Uncertainty and the Market Mechanism
TL;DR: In this paper, the authors present a struggling attempt to give structure to the statement: "Business in under-developed countries is difficult"; in particular, a structure is given for determining the economic costs of dishonesty.
•Book
A Theory of Human Motivation
Abraham H. Maslow
- 01 Jan 2013
Abstract: 1. The integrated wholeness of the organism must be one of the foundation stones of motivation theory. 2. The hunger drive (or any other physiological drive) was rejected as a centering point or model for a definitive theory of motivation. Any drive that is somatically based and localizable was shown to be atypical rather than typical in human motivation. 3. Such a theory should stress and center itself upon ultimate or basic goals rather than partial or superficial ones, upon ends rather than means to these ends. Such a stress would imply a more central place for unconscious than for conscious motivations. 4. There are usually available various cultural paths to the same goal. Therefore conscious, specific, local-cultural desires are not as fundamental in motivation theory as the more basic, unconscious goals. 5. Any motivated behavior, either preparatory or consummatory, must be understood to be a channel through which many basic needs may be simultaneously expressed or satisfied. Typically an act has more than one motivation. 6. Practically all organismic states are to be understood as motivated and as motivating. 7. Human needs arrange themselves in hierarchies of prepotency. That is to say, the appearance of one need usually rests on the prior satisfaction of another, more pre-potent need. Man is a perpetually wanting animal. Also no need or drive can be treated as if it were isolated or discrete; every drive is related to the state of satisfaction or dissatisfaction of other drives. 8. Lists of drives will get us nowhere for various theoretical and practical reasons. Furthermore any classification of motivations
19.7K
Another look at measures of forecast accuracy
Rob J. Hyndman,Anne B. Koehler +1 more
TL;DR: In this paper, the mean absolute scaled error (MESEME) was proposed as the standard measure for comparing forecast accuracy across multiple time series across different time series types, and was used in the M-competition as well as the M3competition.
5K
The Limitations of Deep Learning in Adversarial Settings
Nicolas Papernot,Patrick McDaniel,Somesh Jha,Matt Fredrikson,Z. Berkay Celik,Ananthram Swami +5 more
- 21 Mar 2016
TL;DR: This work formalizes the space of adversaries against deep neural networks (DNNs) and introduces a novel class of algorithms to craft adversarial samples based on a precise understanding of the mapping between inputs and outputs of DNNs.
5K
•Posted Content
Towards A Rigorous Science of Interpretable Machine Learning
Finale Doshi-Velez,Been Kim +1 more
TL;DR: This position paper defines interpretability and describes when interpretability is needed (and when it is not), and suggests a taxonomy for rigorous evaluation and exposes open questions towards a more rigorous science of interpretable machine learning.
4K