IMPACT IN ACTION
AS IS THE CASE WITH MANY COMMONLY USED NEWSROOM METRICS PACKAGES such as Chartbeat (chartbeat.com) and Parse.ly (www.parsely.com), is your focus on tracking audience reach and engagement? Or do you want a full-fledged evaluation process, complete with audience surveys, influence analysis, content analysis and media analytics? What do the journalists think about that
Flip this booklet for a nuts-and-bolts guide to how a pair of newsrooms are assessing their work in terms of outputs, outcomes and impact.
And if you’re curious to find more case studies of journalism impact, visit the Media Impact Funders’ Assessing Impact of Media (AIM) resources:
bit.ly/AIM_journalism
Flip this booklet for a nuts-and-bolts guide to how a pair of newsrooms are assessing their work in terms of outputs, outcomes and impact.
And if you’re curious to find more case studies of journalism impact, visit the Media Impact Funders’ Assessing Impact of Media (AIM) resources:
bit.ly/AIM_journalism
WRAP UP DO's AND DON'T's
READY TO GET STARTED ON WORKING WITH NEWS GRANTEES TO ASSESS IMPACT? Here’s a quick cheat sheet:
Do define how the media project you are seeking to fund relates to the goals of your foundation. What is your own theory of change, and how might that translate into concrete activities in the newsroom?
Do have a frank conversation with your grantee about where to draw lines that will protect the project’s editorial independence (see questions from The Prenups above as a discussion-starter).
Do prepare to be transparent, and answer hard questions about the outcomes you seek.
Do celebrate both successes and productive failures. The impact evaluation process should be informative and strategic, not punitive.
Don’t expect grantees to conduct rigorous evaluation without providing funds and related support for it.
Don’t confuse outputs with outcomes—the number of stories published does not equal changed minds, habits and fields.
Don’t neglect the long tail of impact—build in checkpoints at various stages of the project rather than a single evaluation at the end.
Don’t forget that the outlet may be answering to multiple funders, as well as audiences, board members and stakeholders. Temper your reporting expectations accordingly.
Do define how the media project you are seeking to fund relates to the goals of your foundation. What is your own theory of change, and how might that translate into concrete activities in the newsroom?
Do have a frank conversation with your grantee about where to draw lines that will protect the project’s editorial independence (see questions from The Prenups above as a discussion-starter).
Do prepare to be transparent, and answer hard questions about the outcomes you seek.
Do celebrate both successes and productive failures. The impact evaluation process should be informative and strategic, not punitive.
Don’t expect grantees to conduct rigorous evaluation without providing funds and related support for it.
Don’t confuse outputs with outcomes—the number of stories published does not equal changed minds, habits and fields.
Don’t neglect the long tail of impact—build in checkpoints at various stages of the project rather than a single evaluation at the end.
Don’t forget that the outlet may be answering to multiple funders, as well as audiences, board members and stakeholders. Temper your reporting expectations accordingly.
FOOTNOTES
1. Robert G. Kaiser, “The Bad News about the News,” October 16, 2014: http://www.brookings.edu/research/essays/2014/bad-news#
2. Eric Newton, “How Philanthropy Can Help Create Informed and Engaged Communities,” April 12, 2011: http://www.huffingtonpost.com/eric-newton/philanthropy-informed-engaged-communities_b_847807.html
3. Jesse Holcomb and Amy Mitchell, The Revenue Picture for American Journalism and How It Is Changing, March 26, 2014: http://www.journalism.org/2014/03/26/the-revenue-picture-for-american-journalism-and-how-it-is-changing/
4. Pew Research Center, State of the News Media 2014, March 26, 2014, http://www.journalism.org/packages/state-of-the-news-media-2014/
5. Learning for Action, Deepening Engagement for Lasting Impact, October 2013: http://www.learningforaction.com/wp/wp-content/uploads/2014/08/Media-Measurement-Framework_Final_08_01_14.pdf
6. Lindsay Green-Barber, “3 Things CIR Learned From Analyzing the Impact of Rape in the Fields,” August 18, 2014: http://www.revealnews.org/article-legacy/3-things-cir-learned-from-analyzing-the-impact-of-rape-in-the-fields/
7. Sally Farhat Kassab, “What 30 Foundations Have Learned from Media Projects They Support,” February 5, 2015: http://www.skollfoundation.org/what-30-foundations-have-learned-from-media-projects-they-support/
8. Emily Verellen, From Distribution to Audience Engagement — Social Change Through Film, August 2010: http://www.thefledglingfund.org/wp-content/uploads/2012/08/From-Distribution-to-Audience-Engagment.pdf
9. Josh Stearns, “Journalism’s Theory of Change: From Community Engagement to Civic Action,” August 4, 2014: http://localnewslab.org/2014/08/04/journalisms-theory-of-change-from-community-engagement-to-civic-action/
10. Richard C. Harwood, Assessing Community Information Needs, October 17, 2011: http://www.knightfoundation.org/publications/assessing-community-information-needs
11. Arabella Advisors, Knight News Challenge 2010-2011: What We Learned, August 26, 2014, http://knightfoundation.org.proxo.club/publications/knight-news-challenge-2010-11-what-we-learned
12. Melody Kramer, “Building Smart Newsroom Tools,” June 5, 2014: https://source.opennews.org/en-US/learning/building-smart-newsroom-tools/
13. Melody Kramer, “Social Sandbox Mega Recap Post,” January 6, 2015: http://www.melodyjk.com/social-sandbox-mega-recap-post/
14. Bob Steele, “The Ethics of Civic Journalism: Independence as the Guide,” August 25, 2002: http://www.poynter.org/uncategorized/2128/the-ethics-of-civic-journalism-independence-as-the-guide/
15. Gretchen A. Peck, “Trust in Media,” October 6, 2014: www.editorandpublisher.com/Features/Article/Trust-in-Media#sthash.xPMMeYEo.dpuf
16. Tom Rosenstiel, “Why ‘Be Transparent’ Has Replaced ‘Act Independently’ as a Guiding Journalism Principle,” November 24, 2014, http://www.poynter.org/news/media-innovation/223657/why-be-transparent-is-fnow-a-better-ethical-principle-than-act-independently/
17. Charles Lewis and Hilary Niles, “The Art, Science, and Mystery of Nonprofit News Assessment,” July 10, 2013, http://investigativereportingworkshop.org/ilab/story/measuring-impact/
18. Active Voice, The Prenups, 2009, http://www.theprenups.org/
19. Richard J. Tofel, Non-Profit Journalism: Issues Around Impact, 2013, https://s3.amazonaws.com/propublica/assets/about/LFA_ProPublica-white-paper_2.1.pdf
20. See an example of their 2013 report: www.propublica.org/about/impact-in-2013-from-our-annual-report
2. Eric Newton, “How Philanthropy Can Help Create Informed and Engaged Communities,” April 12, 2011: http://www.huffingtonpost.com/eric-newton/philanthropy-informed-engaged-communities_b_847807.html
3. Jesse Holcomb and Amy Mitchell, The Revenue Picture for American Journalism and How It Is Changing, March 26, 2014: http://www.journalism.org/2014/03/26/the-revenue-picture-for-american-journalism-and-how-it-is-changing/
4. Pew Research Center, State of the News Media 2014, March 26, 2014, http://www.journalism.org/packages/state-of-the-news-media-2014/
5. Learning for Action, Deepening Engagement for Lasting Impact, October 2013: http://www.learningforaction.com/wp/wp-content/uploads/2014/08/Media-Measurement-Framework_Final_08_01_14.pdf
6. Lindsay Green-Barber, “3 Things CIR Learned From Analyzing the Impact of Rape in the Fields,” August 18, 2014: http://www.revealnews.org/article-legacy/3-things-cir-learned-from-analyzing-the-impact-of-rape-in-the-fields/
7. Sally Farhat Kassab, “What 30 Foundations Have Learned from Media Projects They Support,” February 5, 2015: http://www.skollfoundation.org/what-30-foundations-have-learned-from-media-projects-they-support/
8. Emily Verellen, From Distribution to Audience Engagement — Social Change Through Film, August 2010: http://www.thefledglingfund.org/wp-content/uploads/2012/08/From-Distribution-to-Audience-Engagment.pdf
9. Josh Stearns, “Journalism’s Theory of Change: From Community Engagement to Civic Action,” August 4, 2014: http://localnewslab.org/2014/08/04/journalisms-theory-of-change-from-community-engagement-to-civic-action/
10. Richard C. Harwood, Assessing Community Information Needs, October 17, 2011: http://www.knightfoundation.org/publications/assessing-community-information-needs
11. Arabella Advisors, Knight News Challenge 2010-2011: What We Learned, August 26, 2014, http://knightfoundation.org.proxo.club/publications/knight-news-challenge-2010-11-what-we-learned
12. Melody Kramer, “Building Smart Newsroom Tools,” June 5, 2014: https://source.opennews.org/en-US/learning/building-smart-newsroom-tools/
13. Melody Kramer, “Social Sandbox Mega Recap Post,” January 6, 2015: http://www.melodyjk.com/social-sandbox-mega-recap-post/
14. Bob Steele, “The Ethics of Civic Journalism: Independence as the Guide,” August 25, 2002: http://www.poynter.org/uncategorized/2128/the-ethics-of-civic-journalism-independence-as-the-guide/
15. Gretchen A. Peck, “Trust in Media,” October 6, 2014: www.editorandpublisher.com/Features/Article/Trust-in-Media#sthash.xPMMeYEo.dpuf
16. Tom Rosenstiel, “Why ‘Be Transparent’ Has Replaced ‘Act Independently’ as a Guiding Journalism Principle,” November 24, 2014, http://www.poynter.org/news/media-innovation/223657/why-be-transparent-is-fnow-a-better-ethical-principle-than-act-independently/
17. Charles Lewis and Hilary Niles, “The Art, Science, and Mystery of Nonprofit News Assessment,” July 10, 2013, http://investigativereportingworkshop.org/ilab/story/measuring-impact/
18. Active Voice, The Prenups, 2009, http://www.theprenups.org/
19. Richard J. Tofel, Non-Profit Journalism: Issues Around Impact, 2013, https://s3.amazonaws.com/propublica/assets/about/LFA_ProPublica-white-paper_2.1.pdf
20. See an example of their 2013 report: www.propublica.org/about/impact-in-2013-from-our-annual-report