Wikipedia’s wealth of cited information comes from a global community of more than 250,000 editors who contribute content each month. Upholding community standards and fostering diversity and inclusion are major goals for the Wikimedia Foundation — goals that depend on creating software that reflects and reinforces the community’s values.

Over the past year, the foundation has partnered with researchers at Princeton’s Center for Information Technology Policy (CITP) to explore how creative uses of technology can help Wikimedia embrace and elevate a broader group of voices on Wikipedia and in other knowledge-sharing projects. Wikimedia was among the first participants in the case study series of CITP’s tech policy clinic.

The clinic, now in its second academic year, seeks to strengthen ties between Princeton researchers and policymakers in government, industry and the nonprofit sector — a central focus for CITP, which was founded in 2005 as a joint initiative of the School of Engineering and Applied Science and the School of Public and International Affairs.

The clinic has proven to be “transformational” for CITP and for policy work at Princeton, said Matthew Salganik, the director of CITP and a professor of sociology. “It expands on our tradition of public service, and enables exciting pathways to bring our research and people out into the world, and also bring the ideas and problems of the world into CITP to fuel our research and teaching.”

The case studies are a key part of this work, allowing faculty, postdocs and students to engage directly on tech policy issues with organizations such as Wikimedia, The New York Times and the Federal Election Commission. The clinic also hosts policy roundtables on specific topics and trains students through independent work seminars and a summer fellowship. (Due to COVID-19, these activities have been held virtually since March 2020.)

Clinic Lead Mihir Kshirsagar calls the clinic’s work “a virtuous loop of activities,” in which real-world problems inform research and teaching, and researchers’ findings and approaches help guide tech policy decisions.

Kshirsagar joined Princeton after serving as a civil prosecutor for the New York Attorney General’s Bureau of Internet and Technology, where he litigated cases related to online consumer protection — including being the lead trial counsel for a 2018 settlement that was the largest consumer payout by an internet service provider in U.S. history.

“From his experience as a lawyer and prosecutor, [Kshirsagar] brings a different way of looking at the world, and that experience and intellectual framework are a real complement to the different kinds of experiences and frameworks that people at CITP already have,” said Salganik.

“Something that I did again and again in litigation was break down [technologies] that were complex, to show how they actually make a difference on the ground,” said Kshirsagar, who is also a lecturer in computer science. Leading the clinic is “a way that I can help improve the relationship between technology and society by working closely with technologists and developing research that answers the problems that people have today.”

Toby Negrin, Wikimedia’s chief product officer, approached Kshirsagar and Salganik after learning about All Our Ideas, a survey-based platform for social data collection that was co-created by Salganik and was itself partly inspired by Wikipedia.

In their efforts to improve community decision-making, Negrin and his colleagues were concerned about the issue of “loud voices,” in which people who are the most “dedicated and vociferous” may dominate discussions, but do not necessarily represent the broader community’s sentiment, said Negrin. Males from North America and Western Europe are overrepresented among Wikipedia’s editors, and the largely unstructured nature of its online community discussions can hinder efforts to welcome editors from diverse backgrounds.

“As a foundation, we would like to make sure that we are listening to everybody, but we also need to do it in a way that’s compatible with what the community wants,” said Negrin. “It’s both an exploration and an end — it’s an attempt to understand and then come up with solutions that we can then discuss with our community, that hopefully can be implemented in software.”

Wikimedia’s case study spurred a year-long research collaboration led by Ashley Gorham, a postdoctoral research associate at CITP and the University Center for Human Values. Gorham, who is trained as a political theorist, is applying her knowledge about the organization of democratic communities to investigate how Wikipedia’s editors participate in discourse about the platform’s rules and future directions, and how social and technical tools might be used to promote inclusivity.

“Bringing these conceptual models from other disciplines to what’s basically a consumer internet site is incredibly helpful,” said Negrin. His team also participated in a second case study on how Wikimedia might better assess the value of its content, such as the appearance of Wikipedia articles in the knowledge panels of Google search results and the use of Wikidata on a wide range of websites.

In another early case study, Robin Berjon, the head of data governance at The New York Times, sought to examine how the company could benefit from effective advertising while preserving readers’ privacy.

“It’s not necessarily that we’re going to change a specific policy,” said Kshirsagar. “The impact is harder to measure because it’s actually more fundamental than that. We’re helping people to examine how they do their work in a different way.”

In late 2019, the clinic also convened a daylong policy roundtable with officials from the Federal Trade Commission. Princeton researchers examining “dark patterns,” or deceptive practices related to online shopping or privacy, brought the issue to the attention of the FTC. These tactics can be harmful to consumers, and are challenging to detect and regulate at large scales. The roundtable discussion helped shape a research paper co-authored by Kshirsagar along with Jonathan Mayer, an assistant professor of computer science and public affairs, and Arunesh Mathur, a postdoctoral research fellow at CITP. This spring, the FTC will convene a public virtual workshop on the topic.

While many fruitful collaborations have occurred virtually, Salganik said that resuming in-person events when it is safe to do so will be extremely valuable. “The clinic’s work really benefits from having people in a room together — students, faculty, postdocs and research fellows with different backgrounds, along with policymakers. … I’m excited to see what kinds of things we can do when we’re all back together in person,” he said.

Student fellowship helps grow and diversify the tech policy field

In summer 2020, CITP launched the Public Interest Technology Summer Fellowship, open to rising juniors and seniors from Princeton and other schools in the Public Interest Technology University Network. Kshirsagar and Tithi Chattopadhyay, CITP’s associate director, led a three-day virtual bootcamp that introduced the nine fellows to key areas of law, policy and computer science. The fellows then worked on tech policy projects with officials at the Federal Trade Commission, the Consumer Financial Protection Bureau, and the New York City Mayor’s Office of the Chief Technology Officer.

Interest in tech-related public service careers is growing, “but unlike many other [tech] careers, the path is not very clear” for students, said Chattopadhyay. The fellowship places a special emphasis on supporting students from underrepresented groups, and helps fellows “understand how to navigate a career in public service,” she added.

Michelle Ly, a 2020 fellow who is now a senior at Stanford University, said that she valued being connected, albeit virtually, to a cohort of students who are passionate about public interest technology. Ly worked with data scientists at the Consumer Financial Protection Bureau to help streamline how the bureau collects data to determine whether a company has violated consumer protection laws.

Ly, who had not considered working in government until she began her studies at Stanford and heard about public interest tech, said the fellowship experience gave her insights into potential careers. Through readings and conversations, her mentors provided a big-picture view of the bureau’s work, including the close collaborations between its data scientists and enforcement lawyers.

“Seeing that partnership between people of different skill sets in order to reach a common goal of consumer protection in the financial sector was inspiring,” said Ly, who is now conducting thesis research on digital privacy and is considering applying to law school.

Seminars offer new avenues for cross-disciplinary research

Kshirsagar teaches an independent work seminar in the Department of Computer Science that offers undergraduates an opportunity to address questions in tech policy. The seminar’s main goals, he said, are to introduce policy topics to computer science concentrators, and to teach students how to collaborate on research. This fall, he also plans to lead a seminar for graduate students in the School of Public and International Affairs.

“We’ll have two or three students work on a particular aspect of the same puzzle, each doing their independent work, and then they put it together into something bigger,” said Kshirsagar.

During the first seminar in fall 2019, Jordan Heinzel-Nelson investigated the use of facial recognition technology by law enforcement agencies. She learned to sort through academic studies on racial biases in facial recognition and “boil them down to the most easily communicable understanding of the results,” said Heinzel-Nelson, now a senior. She and her classmate also conducted interviews with police to assess how the technology is used and how it might be used in the future.

Advised by Kshirsagar, Heinzel-Nelson is now conducting senior thesis research on the potential of computer vision and satellite imagery to improve evaluation practices for organizations that promote agriculture and food security in developing countries.

Heinzel-Nelson describes herself as “one of those rare computer science students who does not want to go into the field of tech,” and said that her work with Kshirsagar has helped her realize her belief in “the power of technology to improve human existence.” 

“Mihir likes to focus on how the user is eventually going to interact with whatever technology we’re creating. I think that’s super important,” she said, adding that she would like to see academic research engage more deeply with real-world uses of technology.

Princeton’s tech policy clinic has already made strides in that direction, said Kshirsagar. Princeton is one of only a few academic institutions that has established “a dedicated way in which technologists are engaging in the policy world.”

This approach is “critically important, both for the research and teaching mission of the University, and for the benefit of society,” said Salganik. “This is just the beginning. We hope this can become a model for other centers at Princeton and for other centers at universities around the world.”