Offline Impact Indicators Glossary
September 18, 2014
The wealth of digital audience data available sometimes has overshadowed the value of systematically capturing the types offline indicators journalists and other content creators have long been capturing anecdotally. Organizations often focus on impact such as policy changes, mentions in other media, and behavioral and attitudinal changes in individuals. However, systems to categorically collect offline changes associated with content, and a shared language that organizations can use for comparing and learning about offline impact, have been lacking.
The goal for this project is threefold. First, media organizations have expressed the need and desire for a shared language of media impact. Second, with a common language and understanding of how impact is defined, organizations will be able to deepen their understanding of the impact process. Finally, many foundations that fund media projects support a deeper conceptualization of impact that goes beyond online metrics and incorporates change in the real world.
This document presents a draft glossary of offline impact indicators and is part of an ongoing process to develop a taxonomy of media impact.[*] The USC Annenberg School for Communication and Journalism’s Norman Lear Center Media Impact Project commissioned this first draft to complement the online analytics and metrics included in the framing of the media impact project measurement system.
Elements of this glossary were articulated during an August workshop facilitated by The Center for Investigative Reporting and the Media Impact Project with participation from the Columbia University Tow Center for Digital Journalism’s NewsLynx project, the Harmony Institute, Univision, Chalkbeat, WisconsinWatch.org, the Solutions Journalism Network, the City University of New York Graduate School of Journalism, The Aspen Institute and Al-Jazeera America. This glossary builds upon CIR’s earlier work on measuring media impact, as well as the research methodologies presented in CIR’s white paper analyzing the impact of one of its multiplatform projects.
The next stage of the process will include gathering feedback from media organizations to refine the glossary and begin identifying the relationships among outcomes.
The wealth of digital audience data available sometimes has overshadowed the value of systematically capturing the types offline indicators journalists and other content creators have long been capturing anecdotally. Organizations often focus on impact such as policy changes, mentions in other media, and behavioral and attitudinal changes in individuals. However, systems to categorically collect offline changes associated with content, and a shared language that organizations can use for comparing and learning about offline impact, have been lacking.
The goal for this project is threefold. First, media organizations have expressed the need and desire for a shared language of media impact. Second, with a common language and understanding of how impact is defined, organizations will be able to deepen their understanding of the impact process. Finally, many foundations that fund media projects support a deeper conceptualization of impact that goes beyond online metrics and incorporates change in the real world.
This document presents a draft glossary of offline impact indicators and is part of an ongoing process to develop a taxonomy of media impact.[*] The USC Annenberg School for Communication and Journalism’s Norman Lear Center Media Impact Project commissioned this first draft to complement the online analytics and metrics included in the framing of the media impact project measurement system.
Elements of this glossary were articulated during an August workshop facilitated by The Center for Investigative Reporting and the Media Impact Project with participation from the Columbia University Tow Center for Digital Journalism’s NewsLynx project, the Harmony Institute, Univision, Chalkbeat, WisconsinWatch.org, the Solutions Journalism Network, the City University of New York Graduate School of Journalism, The Aspen Institute and Al-Jazeera America. This glossary builds upon CIR’s earlier work on measuring media impact, as well as the research methodologies presented in CIR’s white paper analyzing the impact of one of its multiplatform projects.
The next stage of the process will include gathering feedback from media organizations to refine the glossary and begin identifying the relationships among outcomes.
Format
1) Category: The category is one of three overarching types of offline impact indicators (micro, meso or macro).
a) Indicator type: The indicator type is a general type of offline indicator, of which there are 10.
i) Subtype: A subtype is a more specific type of the indicator, but still general. Indicators and/or methodological suggestions for measurement follow the subtype, where appropriate.
(1) Specific case: The specific case is an example of the subtype. There are potentially infinite specific cases affiliated with any subtype.
Example
2) Fruit: A usually sweet food that grows on a tree or bush.
a) Tree fruit: Fruit that grows on a tree.
i) Stone fruit: Fruit that grows on a tree and has a stony pit.
(1) Peach: A round, sweet fruit with white or yellow flesh, soft yellow or pink skin and a large, hard seed at the center.
Offline Impact Indicators Glossary
1) Micro: Changes that happen at the level of an individual as a result of his or her consumption of media.
a) Awareness (individual): Through exposure to content, an individual’s level of awareness of an issue changes. Awareness is
typically measured through surveys. Other indicators include but are not limited to:
i) Direct audience feedback: A consumer of content responds to the journalist, producer, etc., directly, stating his or her
awareness of an issue increased. Type of feedback. Frequency
(1) Letters to the editor: A consumer of content writes a formal letter to the editor in response to content. Frequency.
(2) Request more information: A consumer of content makes a request to the content producer for more information on
an issue. Frequency.
(3) Researcher contacts journalist: An individual interested in carrying out research on an issue contacts the content
producer. Frequency.
(4) Discussing with friend/family: A consumer of content relays information to others. Survey.
b) Attitude (individual): Through exposure to content, a content consumer’s attitude on an issue changes. For example, a content
consumer may feel more/less empowered, more/less tolerant, etc. Changes in attitude are typically measured through survey research.
i) Attitude change: Consumer of content reports his or her attitude shifted on an issue. Survey.
(1) Direct audience feedback: A consumer of content responds to the journalist, producer, etc., directly, stating his or her
attitude shifted with respect to an issue. Content analysis.
c) Behavior (individual): Through exposure to content, the behavior of a consumer changes. Changes in behavior can be measured through survey research. Behavior change, because it occurs in the real world and often in public ways, often can be measured through aggregate indicators. Behavior changes are nearly limitless, but the categories here are those that multiple content producers have suggested are relevant to their organizations and/or work.
i) Change in consumption/lifestyle: A consumer of content changes the ways in which he or she allocates resources in the
marketplace of goods and services. Survey. Aggregate analytics (i.e.: sales analytics).
ii) Civic engagement: A consumer of content engages in civic life in a new or more robust manner. Survey.
(1) Voting behavior: A consumer of content changes voting behavior, exemplified by voting or not voting or by voting differently than in the past. Survey. Voting data.
(2) Civic participation: A consumer of content participates in civic life in a new way. Survey.
(3) Join political party/movement: A consumer of content joins a political party or movement. Survey. Statistics reported by political parties/movements.
(4) Volunteer: A consumer of content volunteers. Survey. Statistics reported by organizations.
(5) Online action, protest: A consumer of content takes an online protest action.[†] Survey. News analysis. Analytics.
iii) Lawsuit filed: A consumer of content files a lawsuit. Lawsuit records.
iv) More resources to organizations (membership, fundraising, etc.): A consumer of content contributes resources to the
content producer or to a nonprofit or advocacy group addressing the issue. Records.
2) Meso: Meso-level outcomes are those changes that happen at the level of a group as a result of its consumption of media.
a) Awareness (group): Through exposure to content, whether primary or secondary, a pre-existing group’s level of awareness of an issue changes.
i) Discourse shift: Group discourse shifts in relation to content. Content analysis (topic modeling).
b) Attitude (group): Through exposure to content, whether primary or secondary, a pre-existing group’s attitude on an issue
changes. For example, the group may feel more/less empowered, more/less tolerant, etc.
i) Discourse shift: Group attitude shifts in relation to content. Content analysis (topic modeling and sentiment analysis).
c) Collective action: Groups or networks of individuals (pre-existing or newly formed) respond to content and act in a concerted
fashion.
i) Protest: Individuals act together and engage in contentious politics[‡] in order to disrupt. Surveys. News analysis.
d) Civic engagement: Consumers of content engage in civic life in a new or more robust manner. Survey. News analysis.
i) Influencing political agenda: Group/network publicly pressures elected officials, related to content, in an effort to influence political agenda. Content analysis.
ii) Community meeting: Individuals composing a community (geographic, of interest, etc.) gather to discuss and/or take action based on content. News analysis. Self-reported by organizations.
iii) Event held by advocacy organization: Advocacy organization convenes members around content. News analysis. Self- reported by organization.
iv) Lawsuit filed: A group/network files a lawsuit pursuant to content. Lawsuit analysis. News analysis.
e) Social capital: Social capital, characterized by trust among members of a society or community, is increased (or potentially decreased) through a group’s exposure to content.
i) New coalitions: Individuals not previously connected come together to form groups/networks for taking action. Surveys. Content analysis (of press releases, for example).
(1) Community screening: Individuals come together to watch and discuss a piece of content. News analysis. Self-reported by organizations.
ii) Strengthened coalitions: Pre-existing groups strengthen internal social capital (trust) through exposure to content.
(1) Story shared by advocacy organization: Advocacy organization shares content with its members. Content analysis.
iii) Institutional shift: Nongovernmental organization or advocacy group makes institutional change in response to content to better serve members and/or clients and increase trust.
f) Amplification: Other media responding to content increases the reach of content, amplifying its message and giving it an
extended life. There is significant evidence that extended coverage of issues in media can influence political agendas and public
opinion. The following are common ways media interact with others’ content that multiple content producers suggested were relevant
to their organizations and/or work.
i) Increased media coverage: Media amplify the content produced by other media.
(1) Data used by other organizations: Media use data made available by other media organizations. News analysis.
Analytics.
(2) Related story: Media write information published by other media organizations. News analysis. Content analysis.
(3) Story localized: Local media produce content based on another media organization’s story or data, adding more
information or context to make it relevant for their local audience. News analysis. Content analysis.
(4) Collaboration: Media request to work with organization on similar stories. Surveys. Self-reported.
(5) Editorial: Editorial based on content published. Newspaper archives.
g) Content producer change: Media organization responds to its own content production and the outside world’s response to
content. This is a long-term and iterative process.
i) Increased prestige: High-quality content and the impact it initiates results in greater prestige for the organization.
(1) Awards: Media organization receives awards for content. Frequency.
(2) Access to sources: Media organization and its reporters perceive change in access to sources. Self-reported. Surveys.
(3) Job opportunities: Media organization’s reporters perceive increased job opportunities both within their organization and with other organizations. Self-reported. Surveys.
ii) Changed priorities: High-quality content and the impact it initiates results in changed priorities within the organization.
(1) Organization covers topic more: The organization chooses to cover an issue area more/less. Self-reported. Content
analysis.
3) Macro: Macro outcomes are those changes that happen at the level of an institution or power holder, resulting in structural changes.
a) Agenda setting: Producing and distributing content about an issue can result in a new or altered political agenda. The power of
media to set the political agenda has long been debated and studied, and the following are key actions that link content to political
agendas.
i) Public officials respond: A public official responds to content. The response can include rejecting an issue, embracing an
issue and/or a shift in the framing of an issue.
(1) Elected official refers to reporting: Elected officials referring to content bring the issue to the fore and indicate that it
is an issue being placed on the political agenda. Content analysis.
(2) Story cited by decision maker: Decision-maker publicly cites content. Content analysis.
(3) Call for investigation: Power holder calls for an investigation as a result of content. Content analysis.
(4) Hearing: An official hearing is called about an issue presented by media. Content analysis. News analysis.
(5) Letter to the editor: Letters to the editor by public officials on an issue presented by media. Frequency. Content analysis.
b) Power holder acts: Power holders, including but not limited to elected officials, heads of corporations and rule-makers
(individuals or groups), respond to content by taking action. Possible actions are nearly limitless, but the categories here are those that
multiple content producers suggested were relevant to their organizations and/or work.
i) Institutional action: An institution takes an action in response to content.
(1) Investigation by official entity: A government investigation into an issue presented in content. Content analysis. News
analysis.
(2) Sanctions: Power holder imposes sanctions as a result of content.
(3) Institution stops/starts releasing data: In response to content, an institution changes its practices in making data
accessible. Policy analysis.
(4) Firing/resignation: An institution fires an individual or forces him or her to resign in response to reporting. News
analysis. Content analysis (of press releases, for example).
(5) Fine/penalty: Institution levies a fine or penalty in response to content. News analysis.
ii) Legal system response
(1) Criminal charges: A power holder brings criminal charges against someone as a result of content. News analysis.
(2) Reversal of legal decision: Judge(s) reverses a legal decision based on content. News analysis. Legal decision analysis.
iii) Legislative response
(1) Law changed: A law is changed in response to content. Policy analysis. Interviews.
(2) Law proposed: A new law is proposed in response to content. Policy analysis. Interviews.
(3) Law passed: A law is passed in response to content. Policy analysis. Interviews.
(4) Blocked policy: A proposed policy is blocked in response to content. News analysis. Content analysis. Interviews.
iv) Executive response
(1) Rule-making/changing: An institution changes or makes rule(s) in response to content. News analysis. Policy analysis.
(2) Blocked policy: A proposed policy is blocked in response to content. News analysis. Content analysis. Interviews.
(3) Law proposed: A new law is proposed in response to content. Policy analysis. Interviews.
[*] This document was drafted by Lindsay Green-Barber, ACLS public fellow and media impact analyst at The Center for Investigative Reporting, with contributions and edits from Cole Goins, CIR’s distribution and engagement manager.
[†] While joining an online protest is not an offline action, we have included it here because it represents an action that an individual actively carries out in an online space not associated with the media organization or a social network. Also, it more clearly indicates an individual’s attitude or opinion on an issue than the ambiguous analytics of page views, “likes,” and “shares,” none of which make the content consumer’s intentions known.
[‡] Contentious politics is the use of disruptive strategies to pressure officials to make change.