Computer Vision

Deep learning algorithms are rapidly changing the way in which audiovisual media can be produced. Synthetic audiovisual media generated with deep learning—often subsumed colloquially under the label “deepfakes”—have a number of impressive characteristics; they are increasingly trivial to produce, and can be indistinguishable from real sounds and images recorded with a sensor. Much attention has been dedicated to ethical concerns raised by this technological development. Here, I focus instead on a set of issues related to the notion of synthetic audiovisual (...) media, its place within a broader taxonomy of audiovisual media, and how deep learning techniques differ from more traditional approaches to media synthesis. After reviewing important etiological features of deep learning pipelines for media manipulation and generation, I argue that “deepfakes” and related synthetic media produced with such pipelines do not merely offer incremental improvements over previous methods, but challenge traditional taxonomical distinctions, and pave the way for genuinely novel kinds of audiovisual media. (shrink)

Is there a substantial difference between a portrait depicting the sitter’s face made by an artist and an image captured by a machine able to simulate the neuro-physiology of facial perception? Drawing on the later Wittgenstein, this paper answers this question by reference to the relation between seeing a visual pattern as (i) a series of shapes and colours, and (ii) a face with expressions. In the case of the artist, and not of the machine, the portrait’s creative process involves (...) the ability to see both aspects. From the perspective of the image’s viewer, the distinction is more difficult to draw. I address this difficulty by further distinguishing between two attributes of portraits: their representational accuracy, and their ability to convey the artist’s reflection on her experience of seeing the sitter face-to-face. While artificial intelligence can mimic this latter reflective ability, it cannot exactly reproduce it. (shrink)

Just as humans can draw conclusions responsibly or irresponsibly, so too can computers. Machine learning systems that have been trained on data sets that include irresponsible judgments are likely to yield irresponsible predictions as outputs. In this paper I focus on a particular kind of inference a computer system might make: identification of the intentions with which a person acted on the basis of photographic evidence. Such inferences are liable to be morally objectionable, because of a way in which they (...) are presumptuous. After elaborating this moral concern, I explore the possibility that carefully procuring the training data for image recognition systems could ensure that the systems avoid the problem. The lesson of this paper extends beyond just the particular case of image recognition systems and the challenge of responsibly identifying a person’s intentions. Reflection on this particular case demonstrates the importance (as well as the difficulty) of evaluating machine learning systems and their training data from the standpoint of moral considerations that are not encompassed by ordinary assessments of predictive accuracy. (shrink)

In this paper, building on these previous works, we propose to go deeper into the understanding of crowd behavior by proposing an approach which integrates ontologi- cal models of crowd behavior and dedicated computer vision algorithms, with the aim of recognizing some targeted complex events happening in the playground from the observation of the spectator crowd behavior. In order to do that, we first propose an ontology encoding available knowledge on spectator crowd behavior, built as a spe- cialization of the (...) DOLCE foundational ontology, which allows the representation of categories belonging both to the physical and to the social realms. We then propose a simplified and tractable version of such ontology in a new temporal extension of a description logic, which is used for temporally coupling events happening on the play- ground and spectator crowd behavior. At last, computer vision algorithms provide the input information concerning what is observed on the stands and ontological reasoning delivers the output necessary to perform complex event recognition. (shrink)