Stories in Pictures (and Non-Pictorial Objects): A Narratological and Cognitive Psychological Approach

Abstract

Narratological studies have frequently focused upon linguistic structures, considered to be paradigmatic cases of narrativity, while pictorial signs, such as icons and symbols or indices, have received comparably much less attention. In this paper, however, some basic and regularly occurring narrative aspects of pictures and non-pictorial objects will be presented and discussed.

Key Words

art history, cognitive psychology, narratology, pictures, prototypicality, schema theory

 
In this paper I intend to outline some basic and regularly occurring narrative aspects of pictures and non-pictorial objects. As a point of departure, influenced by approaches from cognitive psychology, such as the work of Roger Schank, I suggest that cognition basically consists of the storage and retrieval of action scripts or schemata, that is, narrative structures, that may occur on various levels of abstraction. These schemas incorporate generalized knowledge about event sequences, such as the order in which specific events will take place; causal, enabling, or conventionalized relations between these events, and what kind of events occur in certain action sequences. There also are scene schemas that are characterized by spatial rather than temporal relations. This means that we have mentally stored inventory information, that is, what kinds of objects normally appear in certain situations, as well as spatial-relation information, which concerns the usual spatial layout of a scene.

Through previous experiences we acquire a large quantity of culturally based event and scene stereotypes, along with idiosyncratic variations, either from our previously acquired, direct familiarity with instances of events, or through our acquaintance with written, oral, and, of course, pictorial descriptions of them, such as religious or mythological tales. They include settings, sub-goals, and actions in attempting to reach specific goals.

I claim that the production and comprehension of pictorial signs is frequently based on the existence and activation of such mentally stored action and scene schemas on the part of the beholders. Actually, even objects in general, whether artificial or natural, are capable of expressing or triggering such narrative structures, thus “telling us stories.” In this paper, I present some examples of pictures and non-pictorial objects where narrative structures become activated and, indeed, their recognizability and comprehensibility as such presuppose these structures.

1. The Narrative Qualities of Pictorial Works

Narration has frequently been associated with verbal discourse, whether in written or oral form, where events or situations are represented in a time sequence. Accordingly, theoretical discussions concerning narrativity have usually focused on literature and drama, as well as on film and television. However, the ability of static pictures to represent actions and to narrate stories has received much less attention in art theory contexts. On the other hand, the narrative aspects of visual art have constituted a prevalent focus of interest among art historians, though chiefly from a descriptive, interpretative, and historical point of view. Still, attempts to elucidate any deeper psychological and philosophical aspects involved in visual narrativity have usually occurred on a superficial level, consisting of scattered remarks, intuitively based hypotheses, or the like. Any continuous and systematic treatment of narrative and temporal imagery, compared to the vast quantity of discussion concerning the rendering of space and perspective, seems to be largely absent.[1]

This relative lack of theoretical interest is somewhat surprising, since visual narratives undoubtedly occur in most historical and cultural contexts. With regard to Western art, we find examples of pictorial story-telling at least as early as in ancient Egypt, Greece, and Rome, as well as in the Middle Ages and the Renaissance, and numerous examples from the Middle East or Asia could certainly be cited.[2]

Unfortunately, deeper theoretical reflections on this matter often occur only as scattered remarks even among art historians. Erwin Panofsky, one of the most influential art historians with outspoken theoretical concerns, may be credited with having elaborated the iconographical or iconological methods. According to Panofsky, a fruitful investi­gation of works of art should strive for an analysis of their meaning aspects, in contradistinction to their formal aspects. These aspects occur on several levels.[3] First, we have a pre-iconographic level, such as the depiction of human beings, animals, and natural or artificial objects. The identification of gestures, expressive qualities, and simple actions also belongs to this level. A second interpretative level is iconographical analysis, which consists in identifying the subject matter or theme of the art work. An iconographical interpretation demands an identification of the depicted agents as certain persons (for example, the Virgin Mary or Heracles) or personifi­cations with certain attributes, and would, if necessary, contain some reference to relevant myths or tales, that is, complex action sequences.[4] However, there is little analysis of the exact nature of such narratives, that is, the various means used by the artist in order to convey them, and the presuppositions needed on part of the beholder in order to understand them, in contrast to the rendering of space and perspective. It should be pointed out that Panofsky is no exception in that respect. Indeed, among art historians, as well as aestheticians, problems of narrativity in pictorial art have hardly received any continuous and thorough attention compared to those other issues.

To some extent this neglect is understandable. Usual conceptions of pictorial representation seem irreconcilable with the common sense idea of narration as being temporal and sequential, or, put in another way, as a “temporal program” explicitly manifested by a work. Paintings seem to present themselves as holistic and almost immediately graspable, while verbal narratives are viewed as linear, requiring a temporally successive perceptual process. Now, as the narratologist Gerald Prince has proposed, a minimal requirement for something to be a narrative consists of “the representation of at least two real or fictive events or situations in a time sequence, neither of which presupposes or entails the other.”[5]

Thus, not all modes of discourse may be properly regarded as narratives. For example, arguments are usually considered to be deductive or inductive forms of persuasion, relying on logic, although not necessarily strict syllogisms, in order to prove the validity of an idea or point of view. Expositions can be described as acts of expounding, setting forth, explaining, or conveying information, such as about a narrative’s plot, characters, setting, and theme). Descriptions, whether indefinite or definite, present the properties of things, verbally or visually (re-)presenting persons, places, events, or actions, as well as nonvisible or abstract states of affairs. And explanations, such as deductive-nomological or teleological ones, can be briefly defined as descriptive statements attempting to clarify the causes, contextual circumstances, and consequences of certain facts. None of these discursive modes has an internal time sequence seemingly required by narrative structures; they seem to be static or atemporal. Still, narratives may very well make use of, incorporate, or overlap with arguments, expositions, descriptions, or explanations.

Although Prince admits there are many different manifestations and varying degrees of narrativity[6], he adheres to a rather essentialist definition of the concept, where the necessary and perhaps even sufficient characteristic consists of the “event-sequence” criterion. Thus, according to Prince, a sentence such as “The water boiled then World War II started" would qualify as a minimal narrative.[7] However, to call such an extremely reduced event sequence a narrative seems to be rather counterintuitive.[8] As Noël Carroll has argued, such an example should instead be counted as a mere chronicle, where the crucial “narrative connection” is missing. Such a connection does not necessarily consist of strict causal entailments, rather, “[in] most narratives, the earlier events in a sequence of events underdetermine later events.”[9] Inspired by J. L. Mackie’s discussion of so-called INUS conditions, Carroll argues that a narrative connection occurs when there is “an insufficient but necessary part of a condition that itself is unnecessary but sufficient for an effect event.”[10]

An example of such INUS conditions would be the following sentence: “The thief enters the bank to rob it, but subsequently, as he exits, he is apprehended by the police.” Although the robbing of the bank is causally relevant, it does not causally determine the arrest. Apart from INUS conditions, according to Carroll, a narrative connection (1) also requires a perspicuously ordered temporal relation between the occurring events; (2) concerns the career of at least one unified subject (rather than just adding up disparate or disconnected subjects); and (3) is structured in a globally forward-looking manner, rather than being orientated “backwards.”[11] Other narratologists, such as Monika Fludernik, have attempted to delineate narratives from other forms of discourse by also stressing the representation of human protagonists, or at least anthropomorphic ones, such as speaking animals, performing goal-directed actions and being anchored in particular (existential) time-space settings.[12]

Even so, most discussions concerning narratives have focused on verbal and literary ones, while other narrative types, such as pictorial ones, have been treated quite casually. At first glance, such representations are most favorably manifested by “genuine” temporal arts, such as poetry, drama, literature in general, and motion pictures, which inherently have a sequential structure. Pictures, on the other hand, are inherently static and only capable of representing timeless situations or single, momentary instants. (Thus the concept “static picture” in itself would appear to be tautological.)

Accounts such as these have been put forward by, Lord Shaftesbury, James Harris, and Gotthold Ephraim Lessing.[13] In Discourse on Music, Painting and Poetry (1744), Harris distinguished between media such as music, which is concerned with motion and sound, and painting, which renders shapes and colors. Pictures, according to Harris, can “of necessity [only represent] a punctum temporis or instant.” Interestingly, though, he also admits that “in a Story well known the Spectator’s Memory will supply the previous and the subsequent… [This] cannot be done where such knowledge is wanting.” Indeed, he doubts whether the rendering of a historical situation in a painting would even be intelligible, “supposing history to have been silent and to have given no additional information.”[14]

A more well-known and much-debated account was put forward by Lessing in Laokoon: oder über die Grenzen der Malerei (1766), where he attempted to characterize the distinctive features of painting vs. poetry qua signs, claiming that the representation, or “imitation,” of actions does primarily (and best) occur in poetry.

Lessing claimed that painting essentially is an art of space concerned with the rendering of bodies, while poetry is an art of time, the latter being privileged in narrating actions, that is, the succession of events in time. But poetry cannot render actions without being “joined to certain agents. In so far as those agents are bodies or regarded as such, poetry describes also bodies, but only indirectly through actions.”[16] And “bodies…exist not only in space, but also in time… [P]ainting can imitate actions also, but only as they are suggested through forms.” That is, painting is capable of indicating actions, though only indirectly through suggestion, namely by preferably choosing the most pregnant, arrested movement in an imagined action sequence.

The representation/perception of actions in painting is thus not impossible per se, but it demands more effort, and is less “convenient” compared to poetry. Actually, the difference between painting and poetry is more a matter of degree than a matter of kind: poetry represents actions directly, painting only indirectly. Moreover, Lessing may very well be criticized for committing a naturalistic fallacy. From a factual description of the two genres as primarily spatial or temporal in their essential nature, he comes to the normative conclusion that these genres ought to be restricted to those natural, functional characteristics.[18] But is there any reason why we should adhere to such a rigorous normative position, just by referring to various degrees of convenience or ease?

Still, it seems quite possible that narratology’s primary concern with temporally extended arts, such as literature, movies, and so on, could have been influenced by similar essentialist lines of thought. We may ask, however, whether and to what extent conceptions of pictorial representation as basically static and non-temporal actually are tenable. It has frequently been admitted that the perception of pictures in itself is a temporal, successive process. In his essay “Time in the Plastic Arts” (1949), Etienne Souriau argued that the view that a pictorial work is seen “in its entirety in a single instant...is clearly false;” rather, viewing a picture, as with other visual works of art, involves “a period of contemplation wherein successive reactions take place.”[19]

This is not only the case when it comes to three-dimensional objects, such as inspecting a sculpture or walking through a Gothic cathedral; two-dimensional paintings also demand a similar effort. However, according to Souriau, the fictive time inherent in a pictorial representation “radiates...around the prerogative moment represented..., a structural center from which the mind moves backward to the past and forward to the future,” and thus, in this respect, his view bears a close similarity to Lessing’s.[20] Ernst Gombrich provided another example, maintaining that “...[t)he reading of a picture...happens in time, in fact it needs a very long time....We do it, it seems, more or less as we read a page, by scanning it with our eyes....We build it up in time and hold the bits and pieces we scan in readiness till they fall into place as an imaginable object or event, and it is this totality we perceive and check against the picture in front of us.”[21]

A number of experiments on eye movements and picture perception suggest that the perception of pictorial representations involves something like a temporally extended scanning activity and feature analysis. The Russian psychologist Alfred Yarbus, one of the pioneers in this field of research, studied the saccadic movements of beholders’ eyes when encountering different kinds of visual stimuli, such as photographs or paintings.[22] Eye movements do not occur arbitrarily but are a systematic scanning process, where the beholder fixes his attention on one feature at a time for a very brief period (about 300 msec.), and then moves on to focus on another feature. Now, it does not seem to be especially controversial to admit that temporal processes are involved in the perception of pictures, thereby repudiating that any instantaneous understanding of either medium or message is possible.

Still, one may argue that this kind of temporality is dependent on the viewer’s activities rather than on the object itself, which by nature is static and temporally “frozen.”[23] Such a narrow and essentialist view of the concept of pictorial representation may be questioned by pointing to a number of counter-examples. What about stage design or scene painting? In numerous cases such pictures are not static but make use of moveable parts, such as representations of clouds and waves, as well as various lighting effects, such as strokes of lightning, thus creating a changeable pictorial scene without even taking moveable subjects, such as actors, into consideration. We may also consider stained glass windows in Gothic cathedrals that change with the varying intensity of light filtered through them, or fountains or sculptural installations that make use of water effects. In the twentieth century, there are even further examples of non-static pictures or at least borderline cases, such as mobiles or op art-paintings, such as Bridget Riley’s “Crest” (1964).

2. Historical Examples of Pictorial Narratives

Within contemporary aesthetics, it is widely claimed that any attempts to define concepts such as art in essentialist terms by referring to necessary and jointly sufficient conditions, whether perceptual, functional or procedural, that they are supposed to possess are doomed to failure. Rather, we should think about this category as being like a family whose members resemble each other in some but not all commonly shared respects. This complicated network of similarities constituting the class of art works is, borrowing a Wittgensteinian term, called a family resemblance. This line of reasoning is quite familiar to those who are acquainted with contemporary aesthetics, especially analytic aesthetics. Moreover, numerous cognitive psychologists have followed Eleanor Rosch’s pioneering work by attempting to investigate, by means of quite strict experimental procedures, the nature and acquisition of categories in general, particularly taxonomic categories.[24] According to Rosch, the results obtained from these experiments support the assumption that categories, psychologically speaking, do not usually have clear-cut boundaries but possess a graded structure.[25] This means that there are certain category members that are experienced as cognitive reference points, or the clearest cases of category membership, while other members gradually deviate from them, although they still belong to the category in question. Put in another way, categories are formed around their most representative instances, which possess a prototypical character.

When it comes to narratives, we may also conceive of them as constituting a category with fuzzy boundaries; in this case it seems problematic to insist on a too rigid and essentialist view of their nature.[26] Narratives may be intertwined with descriptions, expositions, arguments, and explanations. Meaning-bearers may be more or less narrative, and narratives may be manifested in various genres, as those mentioned before. But if we admit the existence of temporal and narrative aspects in pictorial representations, the question still remains in which way(s) clear-cut (still) pictures, reliefs, or sculptures possess such features, and whether, and in which respects, some pictures might be regarded as more narrative than others. Let us take a closer look at some of the ways in which pictures seem to have a relatively straightforward narrative function (with temporal ingredients).[27]

First, we have numerous historical examples where static, monoscenic, and quite distinct pictures are linked in a narrative series having a fixed reading order, frequently horizontal or vertical. Modern instances of this kind of pictorial narration can be found in strip cartoons, but actually occur as early as in antiquity and the Middle Ages. Examples include scenes from the life of St. Ambrose on the back of the altar in S. Ambrigio, Milano, c. 850; the scenes from the Old and New Testaments on the bronze doors of the Hildesheim Cathedral (c. 1015, Illus. 1, below); Giotto’s Passion scenes in his frescoes in the Arena Chapel (c. 1306); Gaudenzio Ferrari’s Passion scenes in S M delle Grazie, Varallo (c. 1513); and William Hogarth’s series of moralizing engravings in the eighteenth century.

Illustration 1: Bronze doors from St. Mary’s Cathedral, Hildesheim (c.1015), showing the Creation of Adam and Eve and the Expulsion from the Garden of Eden (left), birth and Passion of Christ (right); vertical reading orders: left top-down; right down-top.

Second, and relatively often discussed by art historians, there are single pictures showing different events and persons in the same pictorial space. In these cases, sometimes called “continuous narratives” or cases of “simultaneous succession,” various phases in an event series are represented simultaneously.[28] Such forms of pictorial narration are also found throughout history, for example, the epic-documentary representation on the column of Trajan of the emperor’s war against the Dacians (c. 101 - 106 C.E.); Masaccio’s fresco “Tribute Money,” showing St. Peter three times in the same pictorial space (c. 1427); Fra Filippo Lippi’s depiction of the Banquet of Herod in the Cathedral of Prato (c. 1460s); Bernardino Luini’s Crucifixion in S. M. degli Angeli, Lugano (c. 1530).[29] “The Legend of the Relics of St. John the Baptist” by Geertgen tot Sint Jans (c. 1484, Illus. 2, below) is an especially interesting example. In the background we see the separate burials of the head and the body after the decapitation of the Baptist, believed to have occurred in the first half of the first century C.E. (Illus.2a). In the foreground is rendered the opening of the tomb and the burning of the limbs on the orders of the Roman Emperor Julian the Apostate, c. 362 C.E. (Illus. 2b, c); in the center is shown the rediscovery of the rescued remains in the thirteenth century (Illus. 2d and 2e). However, the last scene also includes a group portrait of the Knights of the Order of St. John Convent in Haarlem, thus relocating the scene in the late fifteenth century, when the relics were given to the Order by the Turkish sultan, the specific reason for commissioning this painting. The implied time span in this pictorial narrative is thus remarkably extended, stretching over a period of more than 1,000 years.

Illustration 2: “The Legend of the Relics of St. John the Baptist,” Geertgen tot Sint Jans, c. 1484 (Kunsthistorisches Museum, Vienna)

Illustration 2a

Illustration 2b

Illustration 2c

Illustration 2d

Illustration 2e

Of course, numerous examples may be found where these two forms of pictorial narration are intertwined, for example in Lorenzo Ghiberti’s reliefs on the Baptistery doors, the so-called “Porta del Paradiso,” in Florence, 1424-1452, showing ten separate, though narratively linked, scenes from the Old Testament. These scenes constitute a narrative series consisting of distinct pictures, beginning with Adam and Eve, then showing Cain and Abel, Noah, Abraham and Isaac, Joseph, Moses, Joshua, David, and Solomon. However, almost all of these reliefs are polyscenic or continuous narratives. In the picture showing Adam and Eve, for example, we can distinguish between various scenes in the same pictorial space: (i) the creation of Adam; (ii) the creation of Eve; (iii) the temptation and fall of Adam and Eve; and (iv) the expulsion from Paradise.

A third kind of pictorial narration is static pictures, which seem to have a less straightforward narrative function. Here a frozen scene in a tacit action sequence is shown from which what has preceded and will follow has to be inferred by the beholder. Lessing’s idea of the “pregnant moment” would very well fit into this category: only an arrested moment is directly represented, though it implies a wider, temporally extended action sequence. The Hellenistic sculpture group “Laocoön and his Two Sons” (first century C.E., Illus. 3), which Lessing himself discussed at length, is an example where the depicted scene refers to a series of mythological events. The “pregnant moment” rendered here consists of the death struggle between the priest Laocoön and his two sons with two snakes sent by Athena as

Illustration 3: "Laocoön and his Two Sons,” first century C.E.? (Vatican Museums, Rome)

© Michael Ranta; Courtesy of the Vatican Museums

punishment for Laocoön’s attempt to warn the Trojans from taking a wooden horse with Greek warriors hidden inside into the city. A beholder acquainted with the relevant narrative background might well see this sculpture as a significant or crucial moment within a narrative sequence stretching backwards in time as well as into the future in which Laocoön and his sons are killed and the Trojans are defeated by the Greeks.

Mythological, religious, political, and other broad narratives like this have frequently been rendered pictorially by visualizing significant segments of implied narrative structures. However, we may also think about static pictorial scenes that either simply refer to more common or even everyday action patterns that are more or less narratively indeterminate. In the next section, taking research in cognitive psychology into account, I shall discuss how and in what ways pictures may have narrative implications and give rise to the emergence of narrative mental representations in beholders.

3. Cognitive Interpretations of Objects and Events in Pictorial Works

Cognitive psychologists, such as Eleanor Rosch, mentioned earlier, have given considerable attention to the capacity of humans and other living creatures to categorize objects and events. The idea that this capacity is essential for organisms in order to survive and improve their living conditions seems unquestionable. The formation of categories enables us to apply previous experiences to new ones, to make inferences, to make predictions about the future, and to provide efficiency in communication, just to mention a few examples. However, important questions are how categories arise at all, that is, whether and to what extent they are the result of environmental features or constructive processes on the part of the categorizer, and how they are represented in consciousness. A significant characteristic of cognitive psychology, which clearly distinguishes it from traditional behaviorism, is the assumption that intelligent organisms are capable of constructing and manipulating mental representations.

A number of cognitive psychologists have argued that perceptual and cognitive activities are hierarchically structured. New information is compared with and assimilated into broader schemata or categories that are necessary for object recognition, explanations, predictions, and communicative activities. In other words, humans seem to store mental representations that have something like a type-character. These representations are abstractions stored in long-term memory with which external objects are compared. Common taxonomic categories are acquired after encountering several particular instances of the category in question, after which relevant characteristics are extracted and integrated into category knowledge.

Numerous studies in cognitive psychology indicate that category formation in general, whether furniture, fruit, birds, animals, and so on, may be explained as described above. It should also be emphasized that these studies are empirically based, making use of sophisticated and rigorous experimental and statistical methods, thus giving the hypotheses put forward additional strength compared to pure philosophical reflections.

Research in cognitive psychology suggests that events as well as objects may belong to more general categories, that of action schemas. For example, events such as buying a ticket or wearing a dark dress may belong to categories such as going to the cinema or going to a funeral, which may be further categorized as instances of an entertainment event, or an occasion for grief. Sequences of such stereotypical and categorizable actions are commonly called frames, scripts, or event schemas in cognitive psychology.[30] These schemas incorporate generalized knowledge about event sequences, such as the order in which specific events will take place; causal, enabling, or conventionalized relations between these events; and what kind of events occur at all in certain action sequences. Moreover, there are also scene schemas, which are characterized by spatial rather than temporal relations. For example, we have certain expectations of how the rooms, streets, and buildings appear where particular activities, such as going to a restaurant or going to a funeral, take place. Therefore, we have mentally stored inventory information, that is, what kinds of objects normally appear in such situations, and spatial-relation information concerning the usual spatial layout of a scene.[31]

A number of experimental studies have investigated the formation and structure of such action schemas or scripts. For example, Roger Schank and Robert Abelson proposed that our knowledge is usually organized around a large quantity of stereotypic situations consisting of more or less routine activities.[32] We acquire hundreds of such cultural stereotypes, along with idiosyncratic variations, through previous direct or indirect experiences. For example, a series of experiments by Gordon Bower, John Black, and Terrence Turner showed that people largely agreed on the nature of the characters, props, actions, and the order of actions in routine activities like eating in a restaurant and visiting a dentist. Moreover, when asked to recall texts narrating actions from a script, the subjects tended to confuse actions that were presented with unstated actions implied by the script. Subjects also tended to recall script actions in their familiar or canonical order; scrambled texts presenting script actions out of order were usually recalled according to the implicit underlying order.[33]

According to Schank, who extended his relatively early work on action schemas, intelligence consists to a considerable extent in the storage and retrieval of scripts, that is, generalized sets of expectations about what will happen in well-understood situations.[34] Moreover, such memory structures may occur on various levels of abstraction. In the lower levels there will be scenes, general structures that describe how and when a particular set of actions takes place, such as a doctor’s waiting room scene, reception scene, or surgery scene. Each scene defines a setting, a goal, and actions in attempting to reach a specific goal. Scenes can point to scripts that provide the details concerning stereotypical actions that take place within a scene. They are then organized into wider “memory organization packages” (MOPs), which are directed towards the achievement of a major goal.[35] Several MOPs may be active at one time and may reflect the physical, social, and personal aspects of a certain activity. Thus, as Schank suggests, a visit to a dentist will activate at least three MOPs: (1) M-HEALTH PROTECTION (the personal aspects of keeping fit); (2) M-PROFESSIONAL OFFICE VISIT (the physical activity of visiting the dentist; and (3) M-CONTRACT (the social contractual obligations, such as paying the dentist). Furthermore, MOPs may themselves be organized into higher-level structures, so-called meta-MOPs. For instance, the meta-MOP “mM-TRIP” can manage the stages in such a visit by activating MOPs such as M-AIRPLANE, M-HOTEL, and M-LEISURE. On a still higher level, there are “thematic organization packages” (TOPs), which allow us to be reminded of abstract principles or context-independent information that creates relationships between various contexts or MOPs.[36]

The knowledge of scripts, MOPs, meta-MOPs, and TOPs may be more or less idiosyncratic or historically-socially context-bound. It is hardly controversial to suspect, as Schank also claims, that the identity of cultures and sub-cultures is substantially based on the sharing of such low- and high-level narrative structures.[37] Furthermore, such culturally shared stories occur frequently in highly abbreviated forms or “gists.” People often do not remember specific narrations of stories, but rather gists. When a certain index reminds us of a possible gist, it might then be expandable into a full-fledged narrative.

Now with regard to pictorial art and other kinds of pictorial material, the rendered content more or less corresponds to and may be assimilated by narrative mental representations and expectations that are often shared by a relatively large group of beholders. For example, the art historian Michael Baxandall claimed convincingly that artists have usually adapted their work to the general cognitive demands and presuppositions of the intended beholders.[38] Although Baxandall focused chiefly on strategies for pictorial representation used in fifteenth-century Italian painting, it is possible to interpret his claim as suggesting a more general point. The production of visual works of art is influenced by the demands and needs of a certain public. The artist responds to these demands and offers opportunities for the beholder to apply the background experience of his or her “way of life,” in this case including the knowledge of biblical stories, as well as artistic conventions. The beholder interprets a work of art according to acquired category systems and habits that the work has been adapted to. In other words, pictorial representations trigger the retrieval of mentally stored, more or less well-known stories, and the beholders fill in the narrative gaps in the pictorial material with the necessary connecting details.

Storytelling in and by pictures is frequently based on the existence and activation of such mentally stored action and scene schemas of the beholders. These mental schemas are usually constituted out of earlier experiences of action series and events, either from the beholders’ previously acquired, direct familiarity with them, or from the beholders’ acquaintance with written, oral, and pictorial descriptions of certain events, such as religious or mythological tales. Further, pictorial narration consists of representing more or less significant components of action sequences familiar to the beholders, sometimes only by rendering a specific, arrested moment, which can then activate a wider, mentally imagined event schema. Moreover, narrative and temporal aspects in pictorial representations may also occur, for example, in implicit renderings of the cyclic processes of nature and the seasons; of humans’ or other organisms’ ontogenetic and phylogenetic development; and of cultural and historic situations as they relate to other contexts or even the present, that is, the context in which the picture has been created.

Narratively indeterminate pictures usually trigger efforts in their beholders to give them a more definite narrative structure or lead to the creation of narrative hypotheses. One example is Edward Hopper’s painting “Automat,” (Illus. 4, below), which shows a well-dressed woman sitting at a restaurant table. She is wearing makeup, perhaps indicating that she is on her way to or from work or another social occasion where personal appearance is important. She has removed only one glove, which may indicate that she is distracted or in a hurry or simply that she has just come in from outside and has not yet warmed up. Moreover, the woman is warmly dressed; thus it could be late-autumn or winter. Is it late at night, early in the morning, or early in the evening at a time of the year when days are short? Is she coming from or going to work, or has she arranged a rendezvous? And how should we interpret the general atmosphere of emptiness, loneliness, and her downcast eyes? Has anything severe happened in her life? Apart from these questions motivated by our efforts to give the painting a narrative fixation, general going-to-a-restaurant-MOPs are immediately activated, which on a more basic level give this picture a narrative framing.

Illustration 4: Edward Hopper, “Automat,” 1927 (Des Moines Art Center, Des Moines. Des Moines Art Center Permanent Collections. Purchased with funds from the Edmundson Art Foundation, Inc., 1958.2.)

Even nonfigurative pictures and objects may give rise to the emergence of narrative structuring processes. Does Piet Mondrian’s painting “Composition with Yellow” (Illus. 5d, below), an apparently completely static and atemporal picture, suggest any kind(s) of narrative(s)? Of course it does.

First, someone acquainted with Mondrian’s work in general might easily see this painting as a part, perhaps even as some kind of end result, of his “ontogenetic” artistic development, stretching from his early, relatively naturalistic landscape paintings to visual configurations extremely reduced to vertical and horizontal lines and primary colors (Illus. 5a-d, below).

Illustration 5a: Piet Mondrian, “Avond (Evening): The Red Tree,” 1908-10 (Gemeentemuseum, The Hague)

Illustration 5b: Piet Mondrian, “The Grey Tree,” 1911 (Gemeentemuseum, The Hague. Mondrian images © 2011 Mondrian/Holtzman Trust c/o HCR International Washington DC)

Illustration 5c: Piet Mondrian, “Composition no. II,” 1913 (Kroeller-Mueller Museum, Otterlo)

Illustration 5d: Piet Mondrian, “Composition with Yellow,” 1930 (Kunstsammlung Nordrhein-Westfalen, Düsseldorf)

Second, and on a perhaps more basic level, while currently many beholders would agree with their classification as “art,” their status as such was far from accepted at the time of their production in 1930. (His works were actually considered to be degenerate art, Entartete Kunst, in the Third Reich.) Attempts to define art, either in essentialist terms or as a family resemblance concept, have been a standing dispute within philosophical aesthetics. Now, as Noël Carroll has suggested, art has indeed a necessary and thus essential condition for its existence, namely its historical dimension with regard to its production as well as its reception and evaluation.[39] The reception of art on the part of the audience is guided by traditions of interpreting and appreciating art. Such traditions, or the knowledge of historical antecedents, provide the means for orientation towards contemporary art. Historically, as Carroll further claims, preceding art activities and present ones have a narrative connection. When it comes to historical narratives, the incorporated events are usually situated within an explanatory pattern that gives them significance by delineating their causal roles and teleological contributions to certain goals or outcomes.[40] According to Carroll, art historical narratives show a similar pattern. Some historical narratives function as identifying narratives; that is, they are used to establish the art status of contested or disputed works. The beginning of these narratives includes a description of a set of historical circumstances, previous art practices, whose status is generally undisputed. This background introduces a context that is adequate or sufficient for making the further development plausible and narratively intelligible. So the very fact that Mondrian’s painting is classified and classifiable qua art could be regarded as implying narrative presuppositions.[41]

But what about non-pictorial objects? Can they tell or imply stories? As Schank also claims, physical objects can certainly remind us of event structures.[42] Tools and household objects, for example, indicate their functional and goal-directed characteristics and trigger script-based memory structures. They function as perceptual clues that remind us of possible and actual uses of them in various event structures, and sometimes also imply narratives. (The notions of “events structures” and “narratives” should not necessarily be conflated.) For example, a hat such as the characteristic bicorne hat worn by Napoleon, which I recently saw at an exhibition, may give rise to the formation or retrieval of narratives. Even natural objects, such as plants or rocks, may be perceived as constituents of the narrative structures of seasonal or geological changes. In general, our knowledge and perception of the world is permeated by more or less full-fledged narratives that are necessary for our ability to make the world comprehensible, to manipulate it, to see causal relationships, and to prognosticate possible changes. Indeed, conceptions of theory-neutral observations, that is, somewhat story-neutral ones, presently have very few adherents within the philosophies of science and epistemology. The question is not whether almost anything tells, or can tell, a story; the question is how much it does so and how explicit this story-telling is.

When it comes to pictorial material, then, I argue that narrativity and at least implied temporality are more than just contingent or accidental aspects in pictorial representations. In many cases they constitute a basic characteristic and perhaps are even a presupposition in order to comprehend and appreciate them, especially when it comes to pictorial works of art. Pictorial material is frequently and intentionally produced in order to trigger stories or at least to give rise to narrative hypothesizing. However, these assumptions need a far more detailed elaboration, which unfortunately goes beyond the scope of this paper.

Michael Ranta

Michael.Ranta@semiotik.lu.se

Michael Ranta holds a Ph.D. in the History of Art from Stockholm University, Sweden, and is a research fellow at CCS (Centre for Cognitive Semiotics) at Lund University. He has done research in cognitive psychology, art history, and aesthetics, and has written on aesthetic and art historical issues, as well as art criticism.

Published on August 11, 2011.

Endnotes

This article is a revised version of a paper given at the XVIIIth International Congress of Aesthetics, “Diversities in Aesthetics,” Beijing, China, August 9-13, 2010. I would like to thank two anonymous reviewers for valuable comments made on an earlier version of this article.

1. Cf. Götz Pochat, Bild-Zeit - Zeitgestalt und Erzählstruktur in der bildenden Kunst von den Anfängen bis zur frühen Neuzeit (Wien/Köln/Weimar: Böhlau Verlag, 1996), p. 7. “While there have been numerous studies in art history concerned with the concept of space and the rendering of spatial depth, starting with Panofsky [1924],... treatises on the character of ‘time’ and its outcome in the visual arts do not reveal the same continuity and impact, although certainly some contributions related to the subject have been published.” (my trans.)

2. The examples chosen for this article have been taken from Western art history. Undoubtedly, a global art historical overview (or at least a discussion of non-Western examples) would have been advantageous but would have needed a more extensive account beyond the limits of this paper. Moreover, the issues discussed here are not primarily historical ones. Rather, I wanted to point to some questions (and possible answers) of a more principal or theoretical nature concerning the possibilities of pictorial narrativity. Fundamental principles of narrativity as well as of cognitive processes, which are the central issues to be discussed in the present context, are usually not considered to be culture-specific or context-bound per se (although their manifestations may vary, of course). However, as to non-Western examples of pictorial narrativity, we could, for example, think of pictorial works from Japan, such as the Tamamushi Shrine (c. 650, showing the Boddhisatva in a simultaneous succession narrative) or the E-ingakyo (or The Illustrated Sutra of Cause and Effect, c. 735, a hand scroll showing a narrative series of distinct pictures about the life of the Buddha). In Persian art, one could mention the illustrated version of Tutinama ("Tales of a Parrot"), a 14th-century series of 52 stories, containing 250 miniature paintings. In the art of India, we could consider the highly narrative Mughal paintings (16th-19th centuries). Many further examples could, of course, be cited.

3. See e.g. Erwin Panofsky, Studies in Iconology (New York: Harper Torchbooks, 1939/1962); Erwin Panofsky, “Ikonographie und Ikonologie” (1939/1955); reprinted in e.g. Bildende Kunst als Zeichensystem 1 - Ikonographie und Ikonologie, ed. Ekkehard Kaemmerling (Köln: DuMont Buchverlag, 1979/1987), pp. 207-225.

4. A third, iconological type of interpretation would treat the artwork as symptomatic of a cultural climate or world view and formulate statements sug­gested by the work in this respect. Cf. also my article “Implied World Views in Pictures: Reflections from a Cognitive Psychological and Anthropological Point of View”, Contemporary Aesthetics, 5 (2007).

5. Gerald Prince, Narratology--The Form and Functioning of Narrative (Berlin/New York/Amsterdam: Mouton Publishers,1982).

6. Ibid., p. 145f. For example, narratives “presenting relatively many time sequences should have more narrativity than…[those] presenting relatively few.…” (Ibid., p. 146).

7. Ibid., p. 145.

8. Cf. Lars-Åke Skalin, “Centres and Borders: On Defining Narrativity and Narratology,” in Borderliners – Searching the Boundaries of Narrativity and Narratology, ed. Per Krogh Hansen (Copenhagen: Medusa, 2009), pp. 19-75; ref. on p. 26ff.

9. Noël Carroll, Beyond Aesthetics – Philosophical Essays, (Cambridge: Cambridge University Press, 2001), p. 122.

10. Ibid., p. 409.

11. Ibid., p. 126.

12. Monika Fludernik, An Introduction to Narratology (London/New York: Routledge, 2009), p. 6.

13. Comparable ideas have also been proposed by Jean-Baptiste Dubos, Denis Diderot, and especially Edmund Burke in his essay A Philosophical Enquiry into the Origin of Our Ideas of the Sublime and Beautiful (1757), which explicitly influenced Lessing’s lines of thought. Cf. William Guild Howard, “Burke among the Forerunners of Lessing,” PMLA, 22 (1907), 608-32. However, as the present article is not primarily intended to provide a historical (or global) survey concerning theoretical positions about possible temporal aspects in the pictorial arts (see also endnote 2), the following discussion will chiefly focus on Lessing’s view, which in its relative clarity could be used as an exemplary standpoint on the possibility of pictorial narrativity.

14. Quoted in Ernst H. Gombrich, “Moment and Movement in Art,” Journal of the Warburg and Courtauld Institutes, 27 (1964), 293-306, ref. on p. 294.

15. Gotthold Ephraim Lessing, Laocoon: An Essay on the Limits of Painting and Poetry, trans. Ellen Frothingham (New York: Noonday Press, 1957), p. 91.

16. Ibid., p. 92.

17. Loc. cit.

18. Cf. also the discussion in W. J. T. Mitchell, Iconology - Image, Text, Ideology, (Chicago/London: University of Chicago Press, 1986), pp. 94-115.

19. Etienne Souriau, “Time in the Plastic Arts,” Journal of Aesthetics and Art Criticism, 7 (1949), 294-307, ref. on p. 294.

20. Ibid., p. 301.

21. Gombrich, p. 301 ff.

22. Alfred L. Yarbus, Eye Movements and Vision (New York: Plenum Press,1967). Cf. also Robert L. Solso, Cognition and the Visual Arts (Cambridge, MA: MIT Press, 1994), pp.134–138.

23. Cf. Mitchell, p. 100.

24. See Eleanor Rosch, “Cognitive Representations of Semantic Categories,” Journal of Experimental Psychology: General, 104 (1975), 192-233; Eleanor Rosch, “Categorization,” Encyclopedia of Human Behavior, 1, (Academic Press,1994, pp. 513- 523; Eleanor Rosch & Carolyn B. Mervis, “Family Resemblances: Studies in the Internal Structure of Categories,” Cognitive Psychology, 7 (1975); Eleanor Rosch & Barbara B. Lloyd (eds.), Cognition and Categorization, (Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers, 1978).

25. It should be pointed out, though, that there may be categories that actually reflect an all-or-none rule; that is, some entities belong, formally speaking, to the category in question in strict essentialist terms, while others do not. For example, the category odd number includes any number whatsoever that produces a remainder of 1 when divided by 2. All category members satisfy the rule equally. Still, despite the existence of exact formal criteria for category membership, it may be claimed that such a category has a graded structure, psychologically and cognitively speaking, because of the efficiency with which people establish membership of certain numbers, or from the fact that they regard some numbers as more typical than others (say, 3 compared to 1057). Cf. Lawrence W. Barsalou, “Deriving Categories to Achieve Goals,” in The Psychology of Learning and Motivation -- Advances in Research and Theory, ed. Gordon H. Bower, 27 (San Diego: Academic Press, 1991), pp. 1-64, ref. on p. 8.

26. Cf. also the discussion in Skalin, op. cit.

27. Cf. also Göran Sonesson, “Mute Narratives--New Issues in the Study of Pictorial Texts,” Interart Poetics--Essays on the Interrelations of the Arts and the Media, eds. Ulla-Britt Lagerroth, Hans Lund, & Erik Hedling (Amsterdam: Rodopi, 1997), pp. 243- 251, ref. on p. 244 ff.

    Although the following discussion focuses on the narrative function(s) of pictures, this by no means excludes other possible meaning functions. As an anonymous reviewer of an earlier draft of this article has remarked correctly, “…the Laocoön famously serves as the epitome of the visual representation of human agony as well as representing the Homeric narrative described by the author. A scene of the crucifixion of Jesus evokes and depends not only on the narrative of the Passion but can serve as the focus of both private devotion and liturgical celebration extra-narratively.” I could certainly not agree more. Pictorial representations may, of course, be used in a number of ways related to extra-narrative and extra-semiotic functions, such as religious, political, aesthetic, status-maximizing, or otherwise more or less pragmatic ones. And seen purely as representations, we might say that a picture P can function as a meaning be­arer in at least one or several of the following ways:

    1. P represents O, where O could stand for
       • a) one or several singular, real objects or subjects (like the mountain of Sainte-Victoire or Napoleon)
       • b) one or several general, real objects or subjects (like an apple or a woman)
       • c) one or several singular, fictional objects or subjects (like the Holy Grail or Zeus)
       • d) one or several general, fictional objects or subjects (like a halo or an angel).
    2. P expresses E, where E could be regarded as
       • a) emotional properties inherent in P (such as gaiety, melancholy, aggressiveness, or serenity)
       • b) emotional states attributed to the artist (e.g. at the moment of creation, or his usual state of mind)
       • c) emotional states arising (non-contingently) in the mind of the beholder.
    3. P has a sense SE. Here, the term ‘sense’ is supposed to refer to the occur­rence of certain features or attributes being included in or constituting the de­piction of O (i.e. modes of depicting O), for example
       • a) Napoleon as a child or as an emperor
       • b) Zeus as aggressive or as contemplative
       • c) Napoleon as a configuration of lines, patches, or rough brush strokes.
    4. P suggests ST. ST is supposed to refer to statements which P may imply, ex­press or suggest, perhaps partly as the result of a beholder’s background knowledge (e.g. relating to the context of creation, artistic traditions, the artist’s probable meaning-intentions). These statements may be descriptive or norma­tive, and they may refer to the world, society, the artist’s mental state, human actions, God, and so on.
    5. P symbolizes SY. In this case external “clues of connection" between the depicted objects and, for example, (i) religious, mythological, philosophical, or metaphysical ideas, or (ii) persons, groups, national, geographical, or cultural areas that are required in order to recognize the symbolic content. These clues may be symbolic dictionaries such as Cesare Ripa’s “Iconologia” (1593), but also religious, mythological, or literary texts in general. Moreover, acquaintance with actual political, religious, or historical events seems also sometimes to be necessary. Thus P could symbolize SY (by representing O), for example, as follows:
       • a) O = dove; SY = Holy Ghost
       • b) O = Pharaoh Narmer hitting another person; SY = Upper Egypt’s victory over Lower Egypt about 3000 B.C.E.
       • c) O = bees; SY = Pope Urban VIII (Barberini).

    See also Michael Ranta, Mimesis as the Representation of Types–The Historical and Psychological Basis of an Aesthetic Idea (Stockholm: Stockholm University, 2000), p. 31 ff.

28. For an early work on this matter, see Sven Rosén, Succession i simultana bilder--Stilkritiska studier i antikens och renässansens konst (Lund: Gleerupska universitetsbokhandeln, 1912).

29. For a detailed analysis of the relief on the Column of Trajan, see Richard Brilliant, Visual Narratives--Storytelling in Etruscan and Roman Art (Ithaca/London: Cornell University Press, 1984), pp. 90-123.

30. See Jean Matter Mandler, Stories, Scripts, and Scenes: Aspects of Schema Theory (London/Hillsdale, N.J.: Lawrence Erlbaum Associates, Publishers, 1984). In cognitive psychology, a number of terms have been employed to refer to mental representations of essentially complex phenomena. Apart from schemata and scripts, psychologists have also made use of terms such as mental models, causal mental models, which imply explanations and justifications, frames, situation models, episodic models, and so forth. Several of these concepts seem to have the same core set of attributes, though perhaps a basic distinction can be made between (i) representations of pre-existing generic knowledge, and (ii) specific representations which are constructed at the time of use. For a discussion and comparison of the meaning of these terms, see William F. Brewer, “Schemas versus Mental Models in Human Memory,” in Modelling Cognition, ed. P. Morris (Chichester, N.Y.: John Wiley & Sons Ltd., 1987), pp. 187-197.

31. See Mandler, pp. 13-17.

32. Roger C. Schank and Robert P. Abelson, Scripts, Plans, Goals and Understanding: An Inquiry into Human Knowledge Structure (Hillsdale, N.J.: Lawrence Erlbaum Associates, 1977).

33. Gordon H. Bower, John B. Black, & Terence J. Turner, “Scripts in Memory for Text,” Cognitive Psychology, 11 (1979), 177-220.

34. Roger C. Schank, Tell Me a Story: Narrative and Intelligence (Evanston, IL: Northwestern University Press, 1995); Roger C. Schank, Dynamic Memory Revisited (Cambridge: Cambridge University Press, 1999).

35. Schank. (1999), pp. 123-136.

36. Ibid., pp. 137-154.

37. Schank (1995), pp. 189-218.

38. Michael Baxandall, Painting and Experience in Fifteenth Century Italy: A Primer in the Social History of Pictorial Style (Oxford and New York: Oxford University Press, 1972/1988).

39. See Carroll, pp. 79-81.

40. George Wilson, “Narrative,” in The Oxford Handbook of Aesthetics, ed. Jerold Levinson (Oxford: Oxford University Press, 2003), pp. 392-406, ref. on p. 394.

41. It may be questioned, though, whether the classification of an object as a work of art is entirely dependent on its narrative-historical linkage or if, instead, we should also appeal to other kinds of conditions that are perhaps even necessary and jointly sufficient for something to qualify as an art work, if such conditions can be specified at all. In the present context, however, my concern is not to discuss any possible definitions of art but rather to point to the fact that at least sometimes art historical narratives contribute to the establishment, fixation, or enhancement of an object’s art status. Nelson Goodman’s suggested replacement of the traditional question “What is Art?” with “When is Art?” is still far from unconvincing. See Ways of Worldmaking (Indianapolis: Hackett Publishing Company, 1978), pp. 57-70. See also my discussion in “Categorization Research and the Concept of Art: An Empirical and Psychological Approach,” Nordisk estetisk tidskrift/ Journal of Nordic Aesthetics, 25-26 (2002).

42. Schank (1999), e.g., p. 22.