AI analysis of social media poses a double-edged sword for social work and addressing the needs of at-risk youths, said Desmond Upton Patton, senior associate dean, Innovation and Academic Affairs, Columbia University. Speaking in the opening plenary session of the PEARC21 conference, Patton noted that on the one hand, it offers a powerful potential tool for understanding and preventing violence. On the other hand, when applied thoughtlessly or without social context, it runs a high risk of false inference, over-surveillance and further victimizing the communities it intends to help.
The PEARC conference series provides a forum for discussing challenges, opportunities and solutions among the broad range of participants in the research computing community. This community-driven effort builds on successes of the past, and aims to grow and be more inclusive by involving additional local, regional, national and international cyberinfrastructure and research computing partners spanning academia, government and industry. PEARC21, Evolution Across All Dimensions, was offered this year as a virtual event (July 19-22).
Selective Reporting, Building Mythology
“I’m a social worker at heart; I have gone on an eight-year journey to understand social media in an environmental context,” Patton said. His work has focused on Gakirah Barnes, a Chicago teen who had been largely demonized and mythologized in the press as the archetypal “gang banger.”
Certainly, like teens everywhere today, youths in the heavily Black and Latinx neighborhoods of the South and West sides of Chicago live a large portion of their lives on social media. Gang members of rival crews act out their grievances online, and sometimes those arguments translate into real-world violence. But selective reporting on seemingly aggressive tweets often misses the neighborhood culture and personal history needed to make sense of those tweets. Another challenge is determining which aggressive tweets defuse emotions and which build on them; Some teens who brag about gang membership and violence online are mostly following Disney princesses.
“Even if you’re from the same neighborhood, you might not have the same interpretation of what’s happening online,” Patton said. In particular, selective reporting of angry grief can seem like imminent violence when stripped of the context of intervening tweets, let alone personal history.
Barnes was notorious, at least in the Chicago press. The facts are difficult to glean. She was credited with killing or shooting 20 people, but had never been arrested. She was murdered in 2014 near her home. And she was a prolific Twitter user, among the 98th percentile in terms of tweets.
“I’ve studied her tweets online and I’m still struggling as a researcher to demarcate what I read about her and what I think about her,” Patton said. Analysis of online behavior “is not easy and should not be done lightly … I made lots of mistakes in my interpretation of Gakirah that could have been really misused.”
CASM: Applying Social Context to NLP
As do many natural language processing projects, Patton’s group’s CASM method—Contextual Analysis of Social Media—started with expert annotation of the content of tweets. He turned to young people living in the affected communities and to students. Even so, baseline impressions, without context, often missed important details. Adding contextual information such as the original post being reacted to, facts about the author and their peer network, offline events, virality and engagement were needed to provide accurate annotation.
Working with Kathleen McKeown of Columbia’s Natural Language Processing Group and Shih-Fu Chang of its Digital Video Multimedia Laboratory, Patton started with an initial labeling of social media entries as “loss,” “aggression” or “other,” the latter including a number of concepts from mood to neighborhood to health to substance abuse. In a series of reports, the team began with a training dataset of 616 tweets, a development dataset of 102 tweets, and a test of 102 tweets. Eventually the work expanded to utilize 4,936 labeled tweets by Barnes and her top communicators to analyze about a million unlabeled related tweets from 279 Twitter users.
“In the initial run, none of the gold-standard tools could work because they could not understand the context or language,” Patton said. Many posts the experts scored as loss were instead labeled as aggressive or threatening. “It was scary having a tool that was misfiring in this way.”
The initial algorithm, when optimized using speech tags developed from qualitative analysis, performed at an F-measure (a statistical measure of accuracy that takes into account both the ratio of true positives to true positives plus false negatives and the ratio of true positives to true plus false positives) of 62 percent. Expanding the dataset improved this, but only to an F-measure of 70 percent.
“Again, what does it mean to have an accurate tool in this particular context, and should we even use this tool at all” given the modest accuracy, he asked. Any such tool could easily be misused, particularly given the uneven way in which online surveillance has been applied to Black versus White youths. Still, he added, parents in communities affected by gang violence “usually do not care how you do it” if you keep their kids safe. “You rarely find an ethical framework that toggles between these issues.”
Patton and his collaborators are continuing the work, trying to find ways of bringing the accuracy of their ML algorithms up to an acceptable level—as well as defining what is acceptable, and what uses are ethical. They will share code—though not social media data, to protect study subjects’ identities—with researchers interested in the problem.
"social" - Google News
July 22, 2021 at 06:02AM
https://ift.tt/3y1a8xh
PEARC21 Plenary Session: AI for Innovative Social Work - HPCwire
"social" - Google News
https://ift.tt/38fmaXp
https://ift.tt/2WhuDnP
Bagikan Berita Ini
0 Response to "PEARC21 Plenary Session: AI for Innovative Social Work - HPCwire"
Post a Comment