MEDIA IMPACT PROJECT
  • ABOUT
    • MISSION
    • OUR TEAM
    • WHAT WE DO
    • FELLOWS & PARTNERS
  • PROJECTS
    • OVERVIEW
    • AFRICA NARRATIVE
    • ACTION CAMPAIGNS
    • CHARITABLE GIVING IN THE MEDIA
    • CLIMATE & SUSTAINABILITY RESEARCH PROJECTS
    • HEALTH EQUITY
    • FILM/TV DIPLOMACY
    • IDEOLOGY & ENTERTAINMENT
    • IMMIGRATION ON TV
    • JOURNALISM STUDIES >
      • VIRTUAL REALITY
    • POVERTY NARRATIVES
    • THE SOCIAL DILEMMA IMPACT STUDY
  • PUBLICATIONS
    • OVERVIEW
    • Are You What You Watch?
    • Africa in the Media
    • CASE STUDIES & TIPSHEETS
    • Charitable Giving in the Media
    • IMMIGRATION ON TV
    • METRICS GUIDES FOR JOURNALISTS
    • POVERTY IN POP CULTURE
    • VIRTUAL REALITY
  • NEWS & EVENTS
  • BLOG
  • CONTACT
  • ABOUT
    • MISSION
    • OUR TEAM
    • WHAT WE DO
    • FELLOWS & PARTNERS
  • PROJECTS
    • OVERVIEW
    • AFRICA NARRATIVE
    • ACTION CAMPAIGNS
    • CHARITABLE GIVING IN THE MEDIA
    • CLIMATE & SUSTAINABILITY RESEARCH PROJECTS
    • HEALTH EQUITY
    • FILM/TV DIPLOMACY
    • IDEOLOGY & ENTERTAINMENT
    • IMMIGRATION ON TV
    • JOURNALISM STUDIES >
      • VIRTUAL REALITY
    • POVERTY NARRATIVES
    • THE SOCIAL DILEMMA IMPACT STUDY
  • PUBLICATIONS
    • OVERVIEW
    • Are You What You Watch?
    • Africa in the Media
    • CASE STUDIES & TIPSHEETS
    • Charitable Giving in the Media
    • IMMIGRATION ON TV
    • METRICS GUIDES FOR JOURNALISTS
    • POVERTY IN POP CULTURE
    • VIRTUAL REALITY
  • NEWS & EVENTS
  • BLOG
  • CONTACT

Public Diplomacy Evaluation: An Annotated Bibliography


This annotated bibliography is designed as a resource for students and researchers working on topics of Public Diplomacy (PD), especially those interested in the field of PD evaluation. Its point of departure is Professor Robert Banks’ resource guide on PD evaluation techniques, published in 2011 by the USC Center for Public Diplomacy. This bibliography, therefore, flags research articles published after 2011, especially those that address PD evaluation in the context of a changing global media landscape. Rather than take a comprehensive approach, this bibliography highlights some of the more innovative works of scholarship that address five sub-topics in PD: soft power, public diplomacy 2.0, networks, non-state actors, and critical perspectives in PD.
 
USC Center for Public Diplomacy Resources
 
Banks, Robert. “A Resource Guide to Public Diplomacy Evaluation.” CPD Perspectives on Public Diplomacy 19 (2011).
 
Banks’ Resource Guide is the most comprehensive overview of PD evaluation written to-date. Drawing from academic, government, and non-profit sources, Banks outlines the entire life-cycle of a PD evaluation project from design to administration to interpretation. Along the way, Banks notes the tradeoffs inherent in any evaluation of PD and describes best practices for conducting a systematic, rigorous, and independent PD evaluation.

The centerpiece of the document is a 94-page bibliography which catalogues hundreds of publications, websites, organizations, and professional networks relevant to PD evaluation. The resource guide is organized by subtopic (e.g., use of polling, exchange-programs, agenda-setting, New Media) and by actor (e.g., NGO’s, government evaluators). Each resource or publication is accompanied by a short summary highlighting key findings and main arguments.

Since the guide was compiled in 2011, it does not include much of the recent scholarship on social media and the role of transnational technology platforms on PD evaluation. Also, Banks errs on the side of providing actionable advice, rather than of mapping the key theoretical fault lines in the field of PD scholarship (see Pamment’s critique below) That said, the core principles and practices that Banks describes are relevant regardless of media and would be especially helpful for those in strategic planning, goal setting, or evaluation design within the PD field.
 
Soft Power
 
Hayden, Craig. The Rhetoric of Soft Power: Public Diplomacy in Global Contexts. Lexington Books, 2012.
 
Appeals to soft power pervade the literature on public diplomacy. To better understand how these rhetorical appeals function in practice, Hayden uses a comparative analysis of the policy rhetoric surrounding the PD efforts of four countries: Japan, Venezuela, China, and the United States. Each of these case studies is interested in how states deploy concepts of soft power within practical policy discourses. His approach, therefore, defines soft power in terms of “what works” from the point of view of the diplomats, organizations, and policymakers who lead PD efforts in the international sphere.

Among Hayden’s most helpful contributions is repeated calls for conceptual clarity when understanding soft power in practice (see Chapter 2). While Harvard professor Joe Nye famously coined the term “soft power” in the early 1990s to better describe the behavior of actors in the field of international relations, the phrase has evolved into a much vaguer term that nevertheless provides a “compelling justificatory logic” for a wide arrange of media-based, foreign policy initiatives.

Hayden is highly attuned to the unstated assumptions and inconsistencies that are the product of this historical evolution. For example, public policy makers characterize soft power, variously, as an asset, a tool, and an outcome which leads to ambiguities in the most appropriate metrics for assessing the use of soft power. These ambiguities complicate the ability of scholars and other researchers to effectively assess soft-power in the field.

While not a practical “how-to” guide, Hayden’s work is relevant to the work of public diplomacy professionals. He notes repeatedly that PD programs employ a “tacit logic” about the sources, techniques, and impacts of soft power. Making these assumptions more explicit should help practitioners bring greater analytical rigor to their work while, in turn, helping evaluators create more targeted frameworks and metrics.
 
Pamment, James. “Articulating Influence: Toward a Research Agenda for Interpreting the Evaluation of Soft Power, Public Diplomacy and Nation Brands.” Public Relations Review 40, no. 1 (2014): 50–59.
 
Like Hayden, Pammet addresses the gap between how soft power is defined by academic researchers and how it manifests in PD practice. Pammet critiques the assumptions that many evaluators and scholars make about universally-applicable “best practices” based on idealized conceptions of PD.

In particular, Pammet takes aim at Banks’ “positivistic, explanatory modes of analysis” arguing that Banks’ approach presupposes that PD is an exercise in rational-decision making used to reach objective and stable set of goals. Holistic and interpretive approaches, in contrast, help explain how PD evaluation is employed in practice by real-world governments and organizations. For Pammet, PD evaluation – like PD more broadly – is less the product of rational actors engaged in strategic decision making than the product of a complex negotiation between individuals, organizations, and the power structures these actors find themselves embedded in.

Pammet also notes that organizations commonly set PD goals (and, by extension, evaluation frameworks), they will often look inward toward their own activities, resources, and outcomes rather than looking outward to their target audience and the broader range of PD tools available to them to influence that audience. In other words, organizations are often using PD to demonstrate that their program “works” (what Pammet calls the problem of influence) rather than as a tool to achieve a set of predetermined and “objective” foreign policy goals.

Those who have worked on PD evaluation in the field will most likely recognize dynamics Pammet describes, particularly the problem of influence, which creates a gap between theoretically rigorous evaluation models and evaluations as implemented in practice. Rather than providing guidance on how to minimize or even eliminate this gap, Pammet urges scholars and other researchers to accept the existence of this “disjunction” or, better yet, describe it more explicitly within their work so that we can better understand the contextual factors that shape the real-world practice of PD.
 
Public Diplomacy 2.0
 
Cull, Nicholas J. “The Long Road to Public Diplomacy 2.0: The Internet in US Public Diplomacy.” International Studies Review 15, no. 1 (2013): 123–139.
 
Cull uses historical and institutional analysis to describe the rise of “Public Diplomacy 2.0” from the mid-2000s to the present in the American foreign policy establishment. Cull presents PD 2.0 as an admittedly vague but still useful label to describe three inter-related phenomena: 1. the ability of technology to promote the creation of online communities and social networks 2. the increased prevalence of user-generated content and 3. the perceptual shift that technology should be arranged horizontally, through networks, rather than vertically, through institutional hierarchies.

In addition, Cull lays out a detailed institutional history of public diplomacy efforts within the U.S. Department of State. He is careful to note differences between traditional PD efforts (to broadcast information to conduct advocacy) versus PD 2.0 (to facilitate exchange). Perhaps the most relevant section for those engaged in PD evaluation is the conclusion, in which Cull points out the opportunities but also the limitations of PD, especially practiced by a traditional state actor.
 
Khatib, Lina, William Dutton, and Michael Thelwall. “Public Diplomacy 2.0: A Case Study of the US Digital Outreach Team.” The Middle East Journal 66, no. 3 (2012): 453–472.
 
This case study provides an example of American PD practices targeting Middle Eastern audiences. Specifically, it describes the work of the State Department’s Digital Outreach Team during the Obama Administration that seeks to promote engagement through social media, particularly with audiences from the Middle East and Central Asia. American diplomats, working in Arabic, Urdu, and Farsi engaged directly with Internet users on blogs, message boards, and social media sites. The researchers used content and sentiment analyses to unearth the themes and attitudes that recurred in the work of the US Digital Outreach team. 
The case study raises questions about the efficacy of these PD efforts; however, the study was not designed as a definitive assessment of the program and only examined a very limited sample of online interactions. More importantly, the study provides a helpful discussion of challenges and limits of PD evaluation, especially when dealing with a relatively new government program.
 
Networks
 
Zaharna, Rhonda, “Network Purpose, Network Design” in Rhonda S. Zaharna, Amelia Arsenault, and Ali Fisher. Eds. Relational, Networked and Collaborative Approaches to Public Diplomacy: The Connective Mindshift. Routledge, 2014.
 
Discussions of networks are unavoidable when analyzing public diplomacy, particularly in the current, global media landscape. Indeed, some public diplomacy programs (especially exchange and capacity-building initiatives) are designed to foster the creation of inter-personal networks across borders. Zaharna’s book chapter takes a closer look at the relationship between the structure and goals of a network. She argues that purpose and structure must be aligned for networks to function effectively.

Zaharna’s intervention is particularly relevant for evaluators working on programs that seek to strengthen or galvanize global networks of professionals across borders. Her work suggests that evaluation should include an assessment of the structure as well as the process of networks in PD to fully understand their effectiveness.
 
Park, Se Jung, and Yon Soo Lim. “Information Networks and Social Media Use in Public Diplomacy: A Comparative Analysis of South Korea and Japan.” Asian Journal of Communication 24, no. 1 (January 2, 2014): 79–98. doi:10.1080/01292986.2013.851724.
 
This study examines the comparative social media strategies of South Korean and Japanese diplomats. While many studies of PD practice examine network structure in theoretical terms, this study employs network analysis and content analysis to evaluate the relative efficacy of public-sector PD outreach programs on social media. The study provides an example of a promising methodology for PD evaluation.       
 
Non-State Actors
 
La-Porte, María Teresa. “The Legitimacy and Effectiveness of Non-State Actors and the Public Diplomacy Concept,” 2015. http://dadun.unav.edu/handle/10171/38773.
 
La-Porte moves the focus away from the traditional subject of diplomacy research – the behavior of states and their governments – and instead highlights the role of civil-society organizations and other non-state actors working within the global public sphere. La-Porte observes that civil-society organizations face two main challenges in their work: legitimacy and effectiveness. Interestingly, whereas state actors gain legitimacy through democratic practices, the rule of law and other factors distinct from their effectiveness, civil-society actors gain their legitimacy, in large part, through their effectiveness (which presents an interesting challenge for evaluators). Nevertheless, La-Porte points out that civil-society groups are “pioneers” in the exercise of PD, and calls for increased scholarly attention in this area.
 
Bayles, Martha. Through a Screen Darkly: Popular Culture, Public Diplomacy, and America’s Image Abroad. New Haven: Yale University Press, 2014.
 
Bayles examines another, arguably even more influential non-state actor engaged in PD practice: private corporations, especially American media corporations that have enormous influence in shaping the overall global media landscape. She argues that the United States’ public diplomacy practices cannot be separated from the role of Hollywood and the private-sector entertainment industry; even when the US government has retreated from sponsoring public diplomacy efforts directly, the work of image-building to foreign publics has continued (or accelerated) under the aegis of Hollywood films and other American-produced media content. American government-sponsored efforts, therefore, now lag behind the much-more sophisticated and globalized de facto PD work led by private industry – work that potentially conflicts with the aims of American government policy and aims.

In addition, Bayless provides a very helpful summary of how American TV and film genres (e.g., the urban singles comedy, the blockbuster action film, the late-night talk show) compete with regional and foreign genres (e.g., the Middle Eastern Ramadan Series, the Latin American telenovela, Bollywood Romance) for the attention of foreign publics. This analysis should inform the work of PD practitioners looking to cut through this crowded forest of content to promote their own targeted messages, or, similarly, those looking to support the growth of local TV and film professionals working in emerging media markets.

Critical Perspectives
 
Comor, Edward, and Hamilton Bean. “America’s ‘engagement’ delusion: Critiquing a Public Diplomacy Consensus.” International Communication Gazette 74, no. 3 (2012): 203–220.
 
Comor and Bean challenge what they perceive as the accepted but naïve consensus about PD in the 21st century: that it encourages “engagement” and dialogue. They argue that, in practice, PD efforts on social media are only superficially concerned either with engaging foreign publics or promoting cross-cultural dialogue. Instead, PD’s real purpose is simple advocacy – to persuade foreign audiences to sympathize more with American policies. Comor and Bean argue that this asymmetry is both ethically and practically problematic (since the stated goals of PD are not aligned with actual practices).

The authors are particularly critical of the wholesale adoption of evaluation models from the field of Public Relations Research (e.g., the “Excellence Model” and “Brand Equity). Their proposal for a more ethically grounded process of evaluation serves as a useful tool for evaluators looking to promote a process of genuine deliberation or cross-border engagement.
 
Melissen, Jan. Beyond the New Public Diplomacy. Netherlands Institute of International Relations’ Clingendael’, 2011.
 
Melissen writes a thorough analysis of the critiques of PD as practiced by traditional state actors and provides responses to these critiques. Like some other scholars, Melissen calls for increased attention to non-state actors. Interestingly, Melissen identifies public-private partnerships as a particularly fertile area of study and potential source of best practices. Rather than abandoning the study of diplomacy itself as hopelessly old-fashioned, Melissen opens the door for further study of more “collaborative” models in which state and non-state actors partner in their efforts. Unfortunately, few scholars have followed up with careful empirical work of these partnerships, but this area of study will likely be increasingly relevant for PD evaluators in the future.
 
Other Resources
 
Blarel, Nicolas. “India’s Soft Power: From Potential to Reality?” India: The Next Superpower, 2012, 28–33.
 
Çevik, B. Senem. “Turkish Soap Opera Diplomacy: A Western Projection by a Muslim Source.” Exchange: The Journal of Public Diplomacy 5, no. 1 (January 1, 2014).
 
Cross, Mai’a K. Davis, and Jan Melissen. European Public Diplomacy: Soft Power at Work.
Springer, 2013.
 
Fullerton, Jami A., and Alice Kendrick. “Strategic Uses of Mediated Public Diplomacy: International Reaction to U.S. Tourism Advertising.” American Behavioral Scientist 57, no. 9 (September 1, 2013): 1332–49. doi:10.1177/0002764213487737.
 
Gregory, Bruce. “The Paradox of US Public Diplomacy: Its Rise and ‘Demise.’” Institute for Public Diplomacy and Global Communication. Washington, DC: George Washington University, 2014.
 
Hart, Justin. Empire of Ideas: The Origins of Public Diplomacy and the Transformation of U. S. Foreign Policy. OUP USA, 2013.
 
Jang, Gunjoo, and Won K. Paik. “Korean Wave as Tool for Korea’s New Cultural Diplomacy.” Advances in Applied Sociology 02, no. 03 (September 6, 2012): 196. doi:10.4236/aasoci.2012.23026.
 
Murphy, Sheila T., Heather J. Hether, Laurel J. Felt, and Sandra de Castro Buffington. “Public Diplomacy in Prime Time: Exploring the Potential of Entertainment Education in International Public Diplomacy.” American Journal of Media Psychology 5, no. 1–4 (2012): 5.
 
Pamment, James. “Digital Diplomacy as Transmedia Engagement: Aligning Theories of Participatory Culture with International Advocacy Campaigns.” New Media & Society 18, no. 9 (October 1, 2016): 2046–62. doi:10.1177/1461444815577792.
 
Rugh, W. Front Line Public Diplomacy: How US Embassies Communicate with Foreign Publics. Springer, 2014.
 
Zhang, Juyan. “A Strategic Issue Management (SIM) Approach to Social Media Use in Public Diplomacy.” American Behavioral Scientist 57, no. 9 (September 1, 2013): 1312–31. doi:10.1177/0002764213487734.
 
Zhong, Xin, and Jiayi Lu. “Public Diplomacy Meets Social Media: A Study of the U.S. Embassy’s Blogs and Micro-Blogs.” Public Relations Review 39, no. 5 (December 2013): 542–48. doi:10.1016/j.pubrev.2013.07.002.
The Norman Lear Center's Media Impact Project researches how entertainment and news influence our thoughts, attitudes, beliefs, knowledge and actions. We work with researchers, the film and TV industry, nonprofits, and news organizations, and share our research with the public. We are part of the University of Southern California's Annenberg School for Communication and Journalism.