Using VSD in practice involves conducting three types of investigations: empirical, value, and technical. In this section, we describe each type of investigation in detail.
The first type of investigation in VSD is empirical investigation. The purpose of empirical investigation is to go out into the world, a.k.a. the field, to ask questions and measure the social context into which you plan to introduce technology. The key motivation behind empirical investigation is that you cannot hope to design technology that incorporates values without first understanding who your users are, the broader social context of their lives, the values that they hold, and the problems that you want to solve. The following list summarizes key aspects of empirical investigations and key questions that they aim to answer.
There are many important things to note about empirical investigation. First are the kinds of expertise that are involved – not computer and data science, but the humanities, social sciences, and environmental sciences. This highlights a key point about VSD, and about building ethical technology in general: it often requires engaging a team with a diverse set of skills, not just technologists.
Just because VSD calls for skills like sociology, economics, ethics, and psychology, does not necessarily mean you need sociologists, economists, ethicists, and psychologists on your team. It can certainly help to engage experts when possible, but computer and data scientists can and should learn and apply a diverse range of skills themselves. Crucially, this includes learning how to find and incorporate quality research from other areas, including the social sciences, humanities, and arts, that relates to your project.
A second key point concerns the term “stakeholders”. Stakeholders are the people who will be impacted by a proposed piece of technology. There are direct stakeholders, who are typically people who would use your technology and would be colloquially referred to as “users”. Additionally, your technology may have indirect stakeholders, i.e., people who are not “users”, but are nonetheless impacted by your technology. For example, the passenger of a self-driving car would be a direct stakeholder, while nearby pedestrians on the street would be indirect stakeholders (if the car were to malfunction, they could be impacted).
The second type of investigation in VSD is value investigation. The purpose of value investigation is to take the data and knowledge gleaned from the empirical investigations and synthesize and analyze it in order to identify the values and ethical considerations that are (or could be) relevant to the technology. These are the ethics and value considerations that should inform the technology design – e.g., supporting autonomy, improving health, protecting privacy, empowering people, increasing access, providing security, and avoiding bias.
At this point in the investigation, it might appear difficult to maximally support all the values involved. For example, there could be a tension between trying to maximize users’ health outcomes and maximize privacy protections. But that is okay – resolving value tensions is part of the third investigation. The following list summarizes key aspects of value investigations and key questions that they aim to answer.
Notice that at this point in the VSD process, computer and data scientists reenter the picture, but they are not alone. The technologists bring with them the perspective of what can be built and achieved via technology. But this is not enough. It must be augmented by knowledge and skills drawn from other fields of inquiry. For example, ethical analysis and critical theory skills are necessary to synthesize, interpret, and identify relevant values, concerns, and opportunities on the basis of the empirical research and informed by technological understanding. Similarly, legal and policy expertise is necessary to identify potential regulatory or legal considerations, again, based on the empirical investigation and the details about the technology being developed. A rigorous value investigation depends upon good empirical investigation. It can also result in the need for further empirical investigation about possible solutions to problems or opportunities for greater benefits. VSD is therefore not strictly linear with respect to empirical, value, and technical investigations: they often are, and should be, iterative and intertwined.
Having representatives from key stakeholder groups, and representing diverse perspectives, participate in value analysis and ethical investigation is crucial for identifying the full range of value and ethics considerations. It is also important for legitimizing the process, as well as for avoiding myopia (i.e., assuming one’s own limited perspective is the perspective) as well as caricaturing and tokenizing stakeholders’ values. In this way, VSD requires collaboration beyond technologists and openness to the perspectives of others.
Value investigation informs a number of fundamental technological design questions. First and foremost: what is the overall goal of the technology? The goal cannot be framed technologically – we do not build technology for the sake of technology itself. Rather, goals must be framed socially and ecologically – what is the work that you want the technology to do out in the world? A closely related question is developing metrics to assess the success or failure of your technology that are aligned with stakeholder values. These questions, as well as the questions involving stakeholders and values, are discussed at greater length a bit later.
The third and final investigation in VSD is technical investigation. The purpose of technical investigation is to take the goals, values, and metrics identified through the value investigation, and incorporate them into software in an empirically informed way. This is the step where the computer and data scientists take center stage (with a cameo by cybersecurity experts for good measure). The following list summarizes key aspects of technical investigations and key questions that they aim to answer.
As noted above, technical investigation is about architecting systems that embody the goals, values, and metrics identified during the second investigation. This involves creating prototypes and testing them with a variety of stakeholders to see if their key values are being met, and if key concerns are being avoided.
Note that “prototypes” refer to both software artifacts (e.g., an alpha or beta software meant for testing and feedback) as well as associated policy and process documents (e.g., a draft privacy policy or set of community guidelines). This combination of technical interventions and social interventions is a hallmark of VSD: it asks designers to expand the scope of the potential solution space beyond just software. This expansion is powerful, as it enables to designers to tackle more complex ethical problems. For example, suppose that the task is to develop a system that will automate a factory, resulting in job losses. This technical intervention (the automation) could be paired with a social intervention (e.g., a skill retraining program) to help workers transition into new roles, rather than just letting them go. Sometimes VSD requires integrated technical design and techno-social system solutions.
The technical investigation stage is where value tensions must be resolved. Many of the values identified during the value investigation may be shared by all stakeholders; in these cases, the designers job is simply to uphold these values in their designs. However, in other cases values may be in tension; it may not be possible to maximally support every value and address every concern. Returning to our example of content moderation, the value of free expression may be in tension with human welfare and respect, since the latter require tighter moderation policies. Resolving value tensions often requires creative designs that uphold both values; thoughtful and transparent compromises about which values supercede others; and integrating technical and social interventions.
Other key issues to address at this phase of development concern cybersecurity and privacy. Insecure systems will almost certainly violate the expectations and values of stakeholders (e.g., the value of trust), so developing secure software and implementing thorough operational security practices are critical. Privacy is also a key value held by many stakeholders that is implicated by many forms of modern technology, and must be carefully and conscientiously addressed.
With respect to systems that incorporate machine learning or artificial intelligence, there are a suite of concerns driven by the potential for these systems to learn humans biases directly from biased datasets. Thorough audits of datasets and trained models are necessary to preemptively identify these biases, as well as tuning new models so that they do not exhibit these characteristics.
In the next section, we present a concrete process for designing software that leverages VSD.