Vividdata Visualization

Communication in/through/with Data Visualization: A Case for the Message

Generally speaking, a statement like “Here, let me make this more complicated for you” isn’t an easy sell. But, in the case of data visualization, there’s a strong argument for making our understanding of how a visualization supports effective communications, much more complicated. This article takes a high-level view of computer science writing in the data visualization field as to what and how data visualizations are believed to communicate. It supports the belief that approaches taken to date in that research are overly simple in concept or narrow in scope, and as a result, some ‘unexplained variance’ in related communication processes and effects remains unexplored.

Human communications — especially when related to the often complex topics that are tackled using data visualizations — are complex and multi-dimensional, with dimensions that are interdependent (i.e., not orthogonal). Instead, an empirical and social-psychologically based message-effects approach/framework is a more comprehensive and effective model in understanding how to support effective communications. It allows us to understand, measure, analyze and even visualize the interpretations and effects that data visualizations have on the people using them.


Imagine… a world where political leaders make perfectly transparent public policy decisions, explicitly and demonstrably based on empirical evidence.

Or one where corporations are fully accountable because executives have a high-resolution, real-time understanding of their global operations.

It would also be a world of secure and data-governed visualized information broadly available to customers, clients, employees, suppliers, distributors, Board members, shareholders, stakeholders, and regulators alike.

In this world, even the scientific evidence on environment and habitat management would be irrefutably irrefutable.

Now imagine that data visualization technology1 is central to the delivery of these significant capabilities. But why should this all hinge on visualization, you ask? Data visualization technologies — by enabling people to see patterns in the complexity of a visualization — engage viewers and facilitate better understanding of issues which, for example, may reveal:

  • complexities of our modern economic systems and markets
  • interrelatedness of urban issues and social risks
  • fine detail in ocean temperatures allowing better extreme-weather risk prediction
  • where tax dollars are coming from and how they’re being spent, etcetera.

Even to contemplate so ambitious a future, we need a comfortable understanding of how data visualizations work, their characteristics, and what it would feel like for customers, communities or global citizens to engage in using them.

Over the relatively short, recent history of data visualization, one group has led the field in accomplishing the greatest amount of scientific R&D, expanding knowledge and capabilities. The IEEE computer science community has significantly advanced data visualization both as a field and as an industry (in academic, private sector and open source organizations) since the 1990s.

By the 2000s, IEEE leaders recognized that the technology ought, fundamentally, to enable (1) analysis of data and (2) communication of the visualization results[3]. There are many facets to IEEE data visualization research initiatives, but let’s focus on a subgroup that has sought to understand how communications might be enhanced in visualizations.

The central importance of ‘communication’ in data visualization research is supported by the mere prevalence of that word in Cook and Thomas (2005) Illuminating the Path[19], still an influential research-agenda-setting volume for visual analytics. Visual analytics, in the post 9/11 context of risk and national security, became an important sub-theme in the broader data visualization field.

An interesting body of research and writing (mainly IEEE in origin, dating from 2001-2016) unquestionably deals with various communicative aspects of data visualization. These include conceptual writings and articles reporting empirical study results. In these works, we can observe the core thinking about what and how data visualizations communicate, around these themes:

Storytelling in visualizations

  • Gershon and Ward (2001)[7] – visualizations communicate/convey: information
  • Wojtkowski and Wojtkowski (2002)[23] – visualizations communicate: n/a
  • Kosara and Mackinlay (2013)[13] – visualizations communicate: data, information
  • Figueiras (2014)[5] – visualizations communicate/convey: information
  • Boy et al. (2015)[2] – visualizations communicate: n/a
  • Rodríguez et al. (2015)[16] – visualizations communicate: n/a or unclear

Communication-minded visualizations

  • Viégas and Wattenberg (2006)[22] – visualizations communicate: discoveries, findings, insights

Metaphor / visual metaphor in visualizations

  • Ziemkiewicz and Kosara (2008)[24] – visualizations communicate: n/a
  • Ziemkiewicz and Kosara (2009)[25] – visualizations communicate: n/a

Rhetoric / visualization rhetoric

  • Hullman and Diakopoulos (2011)[9] – visualizations communicate: intended message, layered meaning (in a semiotic framework following Bertin, discussed below)

Narrative / visual narrative or narrative visualizations

  • Segel and Heer (2010)[17] – visualizations communicate: information
  • Dove & Jones (2012)[4] – visualizations communicate: insights (via ‘shared discovery experience’)
  • Hullman et al. (2013)[10] – visualizations communicate/convey: sequences and transitions
  • Figueiras (2014)[6] – visualizations communicate: information

Persuasion / persuasive qualities of visualizations

  • Pandry et al (2014)[14] – no mention of communications

Presentation in visualizations

  • Kosara (2016)[12] – visualizations communicate: knowledge

While this article is by no means a full review of this body of writing, several high-level observations are clear upon initial reading.

This research is well rooted in perceptual and cognitive psychologies as well as Human Computer Interaction (HCI). It tends to be ‘relatively light’ on a priori theoretical understanding of communication processes and effects that may be active in the creation or consumption of a data visualization2. It has virtually no crossover benefit from academic communications theory and research. Instead this IEEE research is most likely to cite other IEEE research to support its understanding of communications.

Some of this work explicitly draws on Bertin’s 1960s semiological analysis[1] as a putative theoretical framework to understanding how data might be communicated through cartographical maps and, by extension, through other types of visualizations.

The main, critical problem is that Bertin’s work is not a complete semiology of visualization, because it doesn’t consider all of the relevant and important ‘representations’ involved when a person creates or uses a data visualization3. The missing representations — particularly the one about how the data are understood to represent the measurement of people or things — are fundamentally important to understand if/what/how visualizations communicate effectively.

Many of these empirical studies have relied on deductive reasoning first it’s used to notice the occurrence of a genre of visualizations that seem to invoke communications (for instance, metaphor) then coding judges use deduction to determine potential sub-categories of those visualizations, and finally it’s used to categorize examples (among a set of collected visualizations) into those sub-categories. Typically, the sub-categories are tested against each other using an experimental design where the measured outcomes are generally data/information recall/memory accuracy or speed.

Often, it seems that the latent goal of this type of work is to discover some as-yet untapped cognitive process (similar to Treisman’s work and ideas on pre-attentive processing[20]) related to visual cognition, which (if invoked from within a visualization) would engage those cognitive processes to support, ostensibly, the effective communication of data or information from a visualization to a viewer.

A general implication of deductive reasoning is that the communication qualities or processes (e.g., rhetoric, metaphor etc.) are found only where they are sought, i.e., amid instances of visualizations. As a result of this unit of analysis/observation, it’s understandable that communication is implicitly thought of as being something intrinsic to a visualization instance, rather than (in a larger context) a communicative device intentionally used by one person to communicate a message to one or more other people (or, equally, to themselves at some point in the future).

This IEEE communications research, in recent years, has guided some of the features and functionality that currently appear in data visualization software and related services. These include notably: storytelling, presentation, publishing to groups and collaboration4. Yet since they are based on a relatively simple conceptualization of communication processes and effects, they might not be sufficient to ensure effectiveness let alone measure the evidence and outcomes of effective communications.

In contrast, a different group working in data visualization comprise those who work in and essentially have developed the ‘practice’ of visualization. These are the practitioners that include: data visualization designers, data journalists, data activists and advocates, members of data-focussed interest groups, as well as the content-builders and strategic-thinkers at visualization firms, among others. This group certainly has considered and understands the role of messages in visualizations: messages in the form of stories or arguments that might be explicit, or implicit, in widely circulated data visualizations.

The notion that visualizations communicate anything more complex than data or information  (such as a message) is generally seen by data visualization R&D communities as being more about the subjective ‘stage-craft’ of the visualization practice, which is, therefore, comfortably out of their scope.

This visualization-as-craft perspective seems generally supported by various data visualization industry pundits and analysts who comment on communications in visualization, based on various approaches, frameworks or underlying principles. But often the guidance doesn’t advance scientific knowledge much beyond providing some (unquestionably useful) tips and tricks for creating better or ‘effective’ visualizations.

Perhaps this whole thought ecosystem is using an overly simple view of human communications when it comes to data visualization.

There are empirical evidence-driven ways to understand and support higher level communication processes and effects. This simplicity seems oddly ironic given that data visualization technologies are ground-breaking in their ability to handle complexity in data and analysis. Yet the technologies are diametrically simple in terms of understanding what supports effectiveness in human communications processes and effects.

It would be beneficial to consider and conceptualize — more broadly and comprehensively — how data visualizations communicate. While this would require a more complex communications model, such complexity is very well supported by current visualization core capabilities.

A More-Complex Approach to Communications

What do we mean by a more complex approach to communications? The line of argument could go in various directions at this point. This is because human communications — as a domain of human behavior, of knowledge or of academic research — is inherently multi-dimensional.

For example, we can imagine that a single communication event (even a single tweet) may have political implications, effects on public health and safety, and potential influence on the economy or markets; drive changes in organizations’ strategies; and even impact how people talk to each other at the water cooler.

To reflect this multi-dimensionality, communication studies (as an overall discipline) generally cleave into distinct sub-disciplines5 (only one of which is semiotics) that address in different ways, the various facets of the communications in question.

The particular sub-discipline in which I was trained (and which seems highly relevant to data visualization today) is that of empirical media and message effects. Essentially this is the application of social psychology and social cognition to consider the impacts of communication messages on people’s attitudes, behaviors or cognitions.

In this perspective, a visualization does not necessarily have to communicate just data or just information, but it may communicate concurrently/simultaneously on multiple levels, at least (but not restricted to):

  1. data (and metadata)
  2. information
  3. message
  4. argument
  5. debate

Further, it’s reasonable to propose that these levels are in a hierarchal relationship, where the ability to communicate at each level is based on the strength and veracity of the underlying levels that necessarily support it. Note that if a particular communication process or effect (e.g., persuasion) operates on one level, it does not necessarily follow that an equal or equivalent process/effect is present at a different level, as this would be an ecological fallacy.

Obviously, not every visualization will have enough content to constitute an entire argument or debate on a topic, yet I contend that every visualization contains a message. That message may be explicit or implicit; denoted or connoted; or have different levels of issue salience and importance (across topics or individuals) which are cognitively processed centrally or peripherally, and against which there may or may not be active message counter-arguing.

The visualization message is subjectively contextualized in terms of either the person who created the visualization or the person viewing or using the visualization (i.e., denoted vs connoted). The range of messages will likely be an amalgam of interpretation, commentary, opinions and beliefs. This is likely what Hullman’s research team observed in their blog-comment data[11].

On reading this argument, some in the scientific visualization subfield will counter-argue that a pure scientific visualization does not contain any message but only purely objective data with its correspondingly unbiased information. This is unlikely. There is always a main takeaway, even if it’s, “Hey, this research colleague produced this strong visualization and all possible questions seem anticipated and responses have been clearly documented.” A visualization’s message can be as much about the creator of the visualization (positively or negatively) as about the data or observed patterns in them. Certainly, a research scientist’s professional reputation is of the utmost importance. That would be its message.

A key premise of message effects is that messages affect people by potentially modifying their (a) attitudes (b) behaviors or (c) cognitions (a distinction broadly used in empirical social-psychology). By the same token, a visualization message’s potential effects are also generally predicated on or mediated by the viewer’s prior (a) attitudes, (b) behaviors or (c) cognitions regarding various things, such as the topic of the visualization, or the person who created it, the research scientist who presented it, or the organization that sponsored it. This starts to cover some of the missing representations that were out of scope for Bertin.

A message-effects framework also means there are certain design and content informational requirements of data visualizations and visualization systems. It means that we need a supporting information exchange (essentially, a type of metadata) behind the visualization. Those creating visualizations and related systems need to provide the metadata that explicitly and fully confirms the validity and reliability of the visualization. In exchange, those viewing the visualizations are expected to provide direct, detailed feedback as to their response and interpretation of the visualization. This exchange of information is necessary for a message-effects framework to operate.

A message-effects framework for data visualization is empirical, measurable, recordable, governable and, perhaps not surprising to you by this point, able to be visualized.

There are different but equally valid ways to understand how and what data visualizations communicate, communications being multi-dimensional and multi-disciplinary. Different approaches will focus on different aspects or ‘variances’ in the communication. From my perspective the message-effects approach is a useful and valuable means to further build communication effectiveness into the practice and technologies of data visualization.

Any errors or omissions in this paper are entirely my own. Additional collegial perspectives are always welcome. If you find these ideas interesting please feel free to ask a question, start a conversation or even just say hi.

That term includes the software (in all forms) and related projection/sensation hardware/interfaces, with an emergent set of data-visualization practices, standards and related processes. Data visualization also comprises several academic and professional groups and sub-communities of practitioners, interest groups, pundits, customers and consumers.

Purchase et al. (2008)[15] discussed theory related to communication in data visualization. But the only approach explored is Information Theory, which is based on, and is not seemingly more enlightening than a mathematical theory of communications from the 1940s[18].

Vickers et al. (2013)[21] make further reference to semiotics and do not venture beyond the notion that a chart is only a representation of data.

This particular communicative process is also influenced by IEEE research e.g., Heer and Argawala (2008)[8] on collaboration.

E.g., there are distinct university programs or departments such as journalism, public opinion and political communications, health communications, crisis communications, science communications, interpersonal communications, organizational communications, semiotics/semiology, critical theory, cultural studies, intercultural communications, mass media studies, persuasion, message effects, etc.


Any errors or omissions in this paper are entirely my own. Additional collegial perspectives are always welcome. If you find these ideas interesting please feel free to ask a question, start a conversation or even just say hi.

1  That term includes the software (in all forms) and related projection/sensation hardware/interfaces, with an emergent set of data-visualization practices, standards and related processes. Data visualization also comprises several academic and professional groups and sub-communities of practitioners, interest groups, pundits, customers and consumers.

2  Purchase et al. (2008)[15] discussed theory related to communication in data visualization. But the only approach explored is Information Theory, which is based on, and is not seemingly more enlightening than a mathematical theory of communications from the 1940s[18].

3  Vickers et al. (2013)[21] make further reference to semiotics and do not venture beyond the notion that a chart is only a representation of data.

4  This particular communicative process is also influenced by IEEE research e.g., Heer and Argawala (2008)[8] on collaboration.

5  E.g., there are distinct university programs or departments such as journalism, public opinion and political communications, health communications, crisis communications, science communications, interpersonal communications, organizational communications, semiotics/semiology, critical theory, cultural studies, intercultural communications, mass media studies, persuasion, message effects, etc.

References

[1] Bertin, Jacques with Marc Barbut, Serge Bonin, Guy Arbellot and others (1973). Sémiologie graphique: Les diagrammes, les réseaux, les cartes (2ième édition). Les re-impressions Des Éditions de l’École des Hautes Études en Sciences Sociales, Paris.

[2] Boy, Jeremy. Francoise Detienne and Jean-Daniel Fekete (2015). “Storytelling in Information Visualizations: Does it Engage Users to Explore Data?” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1449-1458).

[3] Card, Stuart K, Jock Mackinlay and Ben Shneiderman (1999). “Information Visualization” (Ch. 1) in Readings in Information Visualization: Using Vision to Think, (pp. 1-34). Morgan Kaufmann Publishers Inc. San Francisco.

[4] Dove, Graham and Sara Jones (2012). “Narrative Visualization: Sharing Insights into Complex Data” paper at the Interfaces and Human Computer Interaction (IHCI 2012), Lisbon, Portugal.

[5] Figueiras, Ana (2014). “How to tell stories using visualization” paper at IEEE International Conference on Information Visualisation, 2014.

[6] Figueiras, Ana (2014) “Narrative visualization: A case study of how to incorporate narrative elements in existing visualizations” paper at IEEE International Conference on Information Visualisation, 2014.

[7] Gershon, Nahum and Ward Page (2001). “What storytelling can do for information visualization” in Communications of the ACM (Vol 44, No 8, pp. 31-37).

[8] Heer, Jeffrey and Maneesh Agrawala (2008). “Design considerations for collaborative visual analytics” in Information visualization (Vol. 7, No. 1, pp. 49-62).

[9] Hullman, Jessica and Nicholas Diakopoulos (2011). “Visualization Rhetoric: Framing Effects in Narrative Visualization” Transactions on Visualization and Computer Graphics (Vol. 17, No. 12, pp. 2231– 2240).

[10] Hullman, Jessica, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher and Eytan Adar (2013). “A deeper understanding of sequence in narrative visualization” in IEEE Transactions on Visualization and Computer Graphics (Vol 19, Issue 12).

[11] Hullman Jesica, Nicholas Diakopoulos, Elaheh Momeni, Eytan Adar (2015). “Content, context, and critique: Commenting on a data visualization blog” in Communications of the ACM.

[12] Kosara, Robert (2016). “Presentation-Oriented Visualization Techniques” in Computer Graphics and Applications (Vol. 36, No. 1, pp. 80–85).

[13] Kosara, Robert and Jock Mackinlay (2013). “Storytelling: The next step for visualization” in Computer (Vol 46, No. 5, pp. 44-50).

[14] Pandey, Anshul Vikram, Anjali Manivannan, Oded Nov, Margaret Satterthwaite and Enrico Bertini (2014). “The persuasive power of data visualization” in IEEE Transactions on Visualization and Computer Graphics (Vol. 20, No. 12, pp. 2211-2220).

[15] Purchase, Helen C., Natalia Andrienko, T.J. Jankun-Kelly and Matthew Ward (2008). “Theoretical foundations of information visualization” in Information Visualization (pp. 46-64), Andreas Kerren, John T. Stasko, Jean-Daniel Fekete, Chris North (Eds). Springer Berlin Heidelberg.

[16] Rodríguez, María Teresa, Sérgio Nunes, and Tiago Devezas (2015). “Telling Stories with Data Visualization” in Proceedings of the 2015 Workshop on Narrative & Hypertext. ACM.

[17] Segel, Edward and Jeffrey Heer (2010). “Narrative Visualization: Telling Stories with Data” in Transactions on Visualization and Computer Graphics (Vol. 16, No. 6, pp. 1139–1148).

[18] Shannon, Claude E. (1948). “A mathematical theory of communication” (parts 1 and 2) in Bell Systems Technical Journal (vol. 27, pp. 379-423, 623-656).

[19] Thomas, James J. and Kristin A Cook (Eds.) (2005). Illuminating the Path: the Research and Development Agenda for Visual Analytics. National Visualization and Analytics Center (USA).

[20] Treisman, Anne (1985). “Preattentive processing in vision” in Computer Vision, Graphics, and Image Processing (Vol 31, Issue 2, pp. 156-177).

[21] Vickers, Paul, Joe Faith and Nick Rossiter (2012). “Understanding Visualization: A Formal Approach Using Category Theory and Semiotics” in IEEE Transactions on Visualization and Computer Graphics (Vol.19,  Issue 6).

[22] Viégas, Fernanda B. and Martin Wattenberg (2006). “Communication-minded visualization: A call to action” in IBM Systems Journal (Vol. 45, No. 4, p. 801).

[23] Wojtkowski, Wita and W. Gregory Wojtkowski (2002). “Storytelling: its role in information visualization” in European Systems Science Congress.

[24] Ziemkiewicz, Caroline and Robert Kosara (2008). “The shaping of information by visual metaphors” in IEEE Transactions on Visualization and Computer Graphics  (Vol. 14, No. 6, pp. 1269–1276).

[25] Ziemkiewicz, Caroline and Robert Kosara (2009). “Preconceptions and individual differences in understanding visual metaphors” paper at EUROVIS, 2009.