Hi Aaron, thanks for taking the time to speak with redditors!
I'm curious about how you, Wikimedia, and Wikimedians evaluate ORES. Based on your work with Huggle/Snuggle, it seems like there are two kinds of evaluation for any human-facing machine learning system: (a) recognition and (b) action.
The first way involves the classic machine learning questions of precision/recall: how well can the system detect what the Wikimedians consider to be abuse, and does it have unfair biases? As I understand it, you've designed ORES in such a way that community members can contest and reshape this part of the system.
The second way to evaluate a system has less to do with what it can recognize and much more to do with what people and computers do with that knowledge: as you wrote in your Snuggle paper, one could deploy all sorts of interventions at the end of an AI system that recognizes vandalism or abuse: one could ban people, remove their comments, or offer them mentorship. These interventions also need to be evaluated, but this evaluation requires stepping back from the question "did this make the right decision here" to ask "what should we do when we recognize a situation of a certain kind, and do those interventions achieve what we hope?"
As AI ethics becomes more of a conversation, it seems to me that almost all of the focus is on the first kind of evaluation: the inner workings and recognition of AI systems rather than the use that then follows that recognition. Is that also true for ORES? When you evaluate Wikimedia's work, do you think of evaluation in these two ways, and if so, what are your thoughts on evaluating the ethics of the outcomes of AI-steered interventions?
natematias4 karma
Hi Aaron, thanks for taking the time to speak with redditors!
I'm curious about how you, Wikimedia, and Wikimedians evaluate ORES. Based on your work with Huggle/Snuggle, it seems like there are two kinds of evaluation for any human-facing machine learning system: (a) recognition and (b) action.
The first way involves the classic machine learning questions of precision/recall: how well can the system detect what the Wikimedians consider to be abuse, and does it have unfair biases? As I understand it, you've designed ORES in such a way that community members can contest and reshape this part of the system.
The second way to evaluate a system has less to do with what it can recognize and much more to do with what people and computers do with that knowledge: as you wrote in your Snuggle paper, one could deploy all sorts of interventions at the end of an AI system that recognizes vandalism or abuse: one could ban people, remove their comments, or offer them mentorship. These interventions also need to be evaluated, but this evaluation requires stepping back from the question "did this make the right decision here" to ask "what should we do when we recognize a situation of a certain kind, and do those interventions achieve what we hope?"
As AI ethics becomes more of a conversation, it seems to me that almost all of the focus is on the first kind of evaluation: the inner workings and recognition of AI systems rather than the use that then follows that recognition. Is that also true for ORES? When you evaluate Wikimedia's work, do you think of evaluation in these two ways, and if so, what are your thoughts on evaluating the ethics of the outcomes of AI-steered interventions?
View HistoryShare Link