An Indicator for Assessing the Relative Impact of Library Events

This article details one library’s attempt to create a simple assessment method for evaluating the relative engagement of program attendees across a variety of events. The indicator—a combination of perceived level of engagement and calculated level of certainty—can be used alongside other metrics to give a fuller view of the overall impact of library programming. By conducting this study, the authors created a method for quickly assessing and prioritizing the most and least impactful events within a particular set. quantitatively. This study outlines our attempt at creating an “apples to oranges” method of comparison across a wide range of library programs, providing a way to measure relative engagement across multiple events. This simple indicator—a combination of overall level of engagement with a level of certainty—can be used alongside other metrics to give a fuller view of the overall impact of library programming. ABSTRACT Public librarians are increasingly recognized as community partners who improve the reach of organizations focused in whole or in part on public health promotion. The capacity of librarians to support public health initiatives has previously been studied through case studies of particular communities. Few national studies have considered how and why public librarians are perceived as part of the public health infrastructure. This article analyzes data from interviews with 59 public library partners in 18 communities in 16 states across the United States. These interviews were collected as part of a larger study on how public librarians collaborate with partners to promote healthy eating and active living, or HEAL. Case study selection utilized a purposive sampling technique to recruit public libraries that self-identify as actively involved in public health initiatives. Representatives of those libraries introduced the research team to their community health partners. Findings indicate that in these communities, librarians are seen as trusted connectors, community experts, and as professionals that share goals with public health partners. Nevertheless, the strength of these partnerships is diminished by several factors. The discussion focuses on how a) increased knowledge and b) more strategic conversations on this topic, both within the public health and the public library sectors, could contribute to building better collaborations, locally, regionally, and nationally. Building and sustaining these collaborations could, in turn, help public librarians make more strategic and effective contributions to public health issues that appear both in their workplaces, and in their communities. and organize a downtown walking club to boost physical activity.”


KEYWORDS
Programming, Assessment, Visualizations, Outreach, Surveys I t is a well-worn trope within professional LIS literature that library outreach is difficult to assess. Like comparing apples to oranges, the variability of event inputs, outcomes, and measures of engagement make it seemingly impossible to evaluate the overall success of a library's outreach work. Authors such as Farrell and Mastel (2016); LeMire, Graves, Farrell, and Mastel (2018); and Diaz (2019) have organized and categorized various types of library outreach, thus mapping out the landscape, but a universal assessment method still eludes practitioners.
Simply put, the goal of library outreach is to create engagement with and within the library. Therein lies a substantial problem with assessing library outreach: the quality and character of engagement at one event may not be comparable to the quality and character of engagement at another event.
For the purposes of this study, the amount and quality of an individual's engagement during a library event does not matter as much as whether or not engagement is simply present. A positive, non-zero marker of engagement is sufficient for our purposes, thus making it possible to compare one event to another, quantitatively. This study outlines our attempt at creating an "apples to oranges" method of comparison across a wide range of library programs, providing a way to measure relative engagement across multiple events. This simple indicator-a combination of overall level of engagement with a level of certainty-can be used alongside other metrics to give a fuller view of the overall impact of library programming.
The William H. Hannon Library at Loyola Marymount University (LMU) serves a campus of 6,564 undergraduate students and 1,869 graduate students (as of 2020). LMU is a private Jesuit college in Los Angeles, California. On average, the library hosts between forty to fifty individual programs each year, including speaker events, tours, workshops, exhibitions, and other creative events. Our attendance at these events ranges from 5,000-5,500 students, staff, faculty, and campus guests each year. However, like many university libraries, the outreach team is small and has limited resources compared to other units within the library. Our department consists of three full-time librarians (the department head, a programming/exhibitions librarian, and a student engagement librarian), one full-time professional staff member (an event manager), and the equivalent of one part-time student employee (i.e., the combination of multiple student employees working a few hours each week). By conducting this study, we hope to create a method by which to quickly prioritize and weigh the most and least impactful programs in our repertoire.

Literature Review
The American Library Association (2014) conducted a multi-year, multi-part research project to document the characteristics, outcomes, and value of library public programs, and determined that public programming has become central to libraries' work and increasingly important. Moreover, discussion groups with library practitioners from a variety of library settings, including academic libraries, determined "evaluation" to be one of nine essential competencies for programming work. The white paper defines "evaluation" as " [working] toward using statistical and qualitative tools to measure program effectiveness and impact on all community audiences, including those that have historically been un-and underserved; and using this information to iteratively improve the development and delivery of programs." Some of the program evaluation characteristics include whether participants learn new knowledge, change their attitudes, or change their behaviors. However, of the fifty-eight ALA-accredited graduate programs evaluated in the study, none required coursework in library programming or evaluation.
The difficulty in evaluating and assessing library programming generally, or at a broader institutional level, is a recognized concern in LIS literature. As Farrell and Mastel (2016), Santiago, Vinson, Warren, and Lierman (2019), and Wainwright and Mitola (2019) point out, there is no one-size-fits-all method for either collecting or evaluating the overall impact of library programs. Farrell and Mastel's (2016) brief survey shows that librarians generally rely on only a few assessment methods for programming, even though they are familiar and comfortable with a broader range. They go on to categorize and define six types of outreach that are commonly used in libraries and recommend assessment strategies for each. Farrell and Mastel note that qualitative and quantitative assessment more often happens in the classroom, and less so for co-curricular library programs. Due to a variety of limiting factors (such as time, resources, and training) many librarians rely solely on head counts. The authors caution, however: "By only focusing on head counts we undermine our ability to accurately understand the qualitative and quantitative relevance of the assessments made when evaluating library outreach objectives and goals." Wainwright and Mitola (2019) outline various assessment measures, including surveys, whiteboard questions, post-reflections, and summary reports, to demonstrate qualitative methods that go beyond head counts to provide a more holistic perspective on their libraries' outreach efforts. However, their experience confirms what Farrell and Mastel discovered; namely, "[because] learning experiences [offered by academic libraries] can often be unique or serendipitous, measuring how these efforts are contributing to the library's teaching, learning, and research missions can be difficult." By using a variety of assessment methods, as evidenced by the two case studies described in their article, Wainwright and Mitola create assessment plans that are integrated with institutional goals and use mixed-methods approaches.
At the University of Houston, library staff created a team tasked with evaluating the return on investment for the libraries' outreach activities outside the classroom in relation to student success goals, as detailed in Santiago, Vinson, Warren, and Lierman (2019). By conducting an environmental scan, categorizing their programs, and reflecting upon various attributes (e.g. impact, purpose, partners), the task force was able to develop eleven recommendations for future outreach work. As the authors note, this type of top-down assessment of library programming had never been conducted before at their institution. However, the results could lead to significant improvements, such as "wiser allocation of resources, richer reporting and documentation, [...] and focusing on new outreach opportunities in high-impact areas." LeMire, Graves, Farrell, and Mastel (2018) conducted one of the most comprehensive surveys of academic library outreach, the SPEC Kit 361: Outreach and Engagement, in which they determined that "systematic outreach programs are still very much in their infancy and highly dependent on local organizational culture." Their survey found that libraries used a wide variety of assessment methods for programming, including headcounts, observations, peer and participant feedback, interviews, and focus groups. Most of the methods reported were fairly unobtrusive and easy to administer. Most importantly, the authors found that twenty-seven percent of respondents indicated that no one was responsible for overall program assessment.
Similarly, Meyers-Martin and Borchard (2015) conducted a meta-analysis of final exams week library outreach initiatives (e.g. therapy dogs, extended hours, arts and crafts, etc.), including the assessment methods used by libraries. While most libraries collected feedback from users in-person and tracked the number of attendees at these events, others also collected social media feedback, used questionnaires, and tracked the overall number of users in the library.
As noted by LeMire, Graves, Farrell, and Mastel (2018), most assessment methods used by librarians are "unobtrusive and easy to administer." However, some practitioners have attempted to use more complex methods. Strub and Laning (2016) outline a robust hierarchy of event evaluation methods to create a rubric that differentiates "how well" an event went with "what good" the event produced. "How well" examines the overall quality, as defined by success and efficiency, and measured by whether the event reached its target audience (e.g. number of attendees or market reach) and satisfaction or learning (e.g. content evaluation or space feedback). "What good" examines the impact, as defined by effectiveness and value, and measures factors such as whether learning occurred, behavior changed, or impact would be seen. The authors developed a question bank for all these levels of the rubric to be used as needed when assessing library programming.
German and LeMire (2018) also take a mixed-methods approach in their assessment of a major outreach event, Texas A&M University Libraries' annual open house. In addition to counting the number of attendees, the authors counted the number of visits to specific stations within the event, the number of give-away items taken by students, a poll of students' favorite station, a "one-word" assessment questionnaire, and a participant survey that collected both behavioral and attitudinal information. Chan and Kwok (2013) also used a mixed-methods approach in their assessment of an exhibition and three associated talks developed by technical services librarians at Hong Kong Baptist University Library. For each of the talks, librarians used questionnaires to collect feedback and an open comment sheet (i.e. a large sheet of paper) to collect remarks from visitors to the exhibition.
Surveys and questionnaires, like the ones used in this study, are a common assessment tool among outreach and programming librarians because of their ease of use. Jalongo and McDevitt (2015), in their study of the impact of using therapy dogs to help increase library usage, asked students "Would events with dogs influence your use of library resources, spaces and services in the future?" using a Likert scale. Similarly, Lannon and Harrison (2015) asked students to rank their level of stress before and after interacting with therapy dogs.
Both studies used open-ended questions to gather additional data. Preand post-surveying-like those above as well as Sclippa (2017) and Budzise-Weaver, Anders, and Bales (2020)-can provide "excellent insight," immediately showing what worked during a library event and what did not.
Surveys used by outreach librarians run the gamut between "quick" preand post-surveys and more robust questionnaires. Nicholas, Sterling, Davis, et al. (2015), in their study of the efficacy of a residence hall librarian program, employed a survey of library usage that included various multiple choice, ranking, binary, and open-ended questions. Oravet (2014), in assessing their library's "Human vs. Zombies" event, used a seventeen question survey intended to gather demographic information, information about previous library use, and assess whether students' future use and perception of the library would change as a result of the event.

Methodology
Between 2016 and 2020, we collected feedback at forty-four library events using brief, printed surveys that we handed out to every attendee. These surveys asked attendees to respond to three questions: (1) Why did you decide to attend today's event? (2) What did you learn from attending today's event? And (3) was there anything that surprised you and if so, what? Jackson (2019) outlines the intent and justification for using these three questions. A student assistant typed the handwritten forms into an online form which generated a spreadsheet of the 884 resulting responses. Additionally, we counted the number of attendees at each event. Using the number of attendees and number of feedback forms, we calculated a "response rate" for each event (number of feedback forms / number of attendees). This ratio will be used to determine a level of confidence in our data. For example, if half the attendees filled out a feedback form, then the confidence level for the feedback on that event would be fifty percent. An event in which all attendees filled out the forms would have a confidence level of one hundred percent. Relatively, we can be more confident in the perceived level of engagement (described below) for the latter event.
To determine the level of engagement (on the basis of perceived indicators of engagement in each feedback form), we needed to code each response. We used a binary yes/no code to determine if a response showed evidence of engagement. We decided that "engagement" would be determined by whether the feedback responses showed a change in behavior, attitude, or knowledge related to the goals of the event. Once again, we should emphasize that we did not rank the level or quality of engagement, as doing so would make it difficult to compare one event to another (note the "apples and oranges" problem described above). However, by using a binary yes/no coding system that could function without having to accord with the unique goals of each of forty-four events, we felt we could confidently compare different types of library programs.
We divided the spreadsheet of attendee responses into six sections and, following a norming exercise, randomly assigned each author (n=4) to code three of the six sections. The authors were grouped into pairs, and each pair compared their initial coding which found an intercoder agreement of between 89.8 percent and 97.5 percent. Each pair of authors then met to discuss the discrepancies in their initial coding until they reached consensus. Using the data from the coding exercise, we calculated an "engagement rate" for each event (percent of respondents who showed evidence of engagement).

Results
Most of the events fall into one of three categories: (1) Archives & Special Collections Exhibition Openings; (2) Faculty Pub Night; and (3) Other. Archives & Special Collections exhibition openings usually consisted of a lecture by one or two invited speakers, a talk by the exhibition curator, an opportunity for guests to explore the exhibition gallery, and catered food. Faculty Pub Night events usually consisted of a lecture by an invited faculty member and catered food (Hazlitt and Jackson, 2016). Other events included in the review set include: Women's Voices (featuring dramatic readings of famous historical figures); LMU Speaks (an autobiographical storytelling program); Careers in LIS (a panel discussion for graduating seniors); Luis Rodriguez (a panel discussion with a local poet); and Collaboration as Creative Synthesis (a panel discussion with a local artist). Figure 1 shows the relationship between engagement rates and response rates, with programs categorized by event type. Plots toward the right side of the graph had a higher response rate. Plots toward the top of the graph had a higher engagement rate. It should be noted that in the following figures, the y-axis is intentionally set to start at 0.65 (or, sixty-five percent engagement) to most effectively show the relative difference among various plot points. Thus, points near the bottom of the graph do not represent events with absolute low engagement but events with relative low engagement. It is important to note that all events plotted in these figures had moderate to high engagement, with more than sixty-five percent of attendees showing evidence of engagement.   The visualizations that follow (figures 2-4) show the same data, but with different factors emphasized graphically within the chart: by type of attendee, by attendance numbers, and by a combination of various factors (number of attendees, semester in which the event was hosted, and event category).

Discussion
It should be noted before we discuss these visualizations that one would not need to assess four years' worth of feedback forms to use this method. As noted in the introduction, we sought to create a simple method for quickly comparing the relative success of multiple events, even if those events had different expected outcomes. For example, to use this method, all one needs to do is (1) determine a simple means for assessing whether a program attendee was engaged and (2) determine how many attendees showed evidence of engagement. The threshold for what constitutes engagement in step #1 could vary from one event to the next, but for the purposes of this method, only the presence of engagement is necessary.
Instead of providing a more robust means of quantitative assessment, the visualizations above offer "food for thought." These rough sketches of library programming outcomes provide one lens, however hazy, through which to discuss the merits, problems, and impact of a large number of library events relative to each other. While it would be difficult to draw conclusions from the data with a high level of certainty, the visualizations offer an opportunity to generalize and inspire trains of thought that can inform future program development.
For example, events that fall in the upper right quadrant of the visualization can generally be said to be "highly successful" in that they show high levels of engagement with a high level of certainty. Examining the events that fall into this general area of the graph, we find a predominance of Faculty Pub Night programs, specifically those that focused on a science topic (Brain, Ford, Moffet, and Okada are all names of faculty in our School of Science & Engineering). What potential conclusions can we draw from this observation? While it was not within the scope or methodology of our study to determine why any one event was more successful than another, it is tempting to speculate. For one, we know from personal experience that science faculty frequently offer extra credit for their students to attend extra-curricular events (relatedly, the difficulty of science courses makes the offer of extra credit even more attractive). Second, the topics are highly specific (e.g. Okada spoke about the neural organization of language using functional neuroimaging). Perhaps the specificity of the topic attracted an audience that attended knowing full-well the subject matter to be covered. Applying the various assessment methods mentioned by Wainwright and Mitola (2019) could confirm the truth of these conjectures.
We also noticed that all Archives & Special Collections opening receptions, with the exception of one, have a response rate below fifty percent. Upon reflection, it became clear to us why. The typical structure of an Archives & Special Collections reception is that a series of speakers present on a topic related to the library's current gallery exhibition; following a question-andanswer period, attendees are then invited to leave the event space to enter the " With one exception, all events classified as "Other" ... ranked an engagement rate of over ninety percent. Events in this category include non-standard or ad-hoc programming. One possible reason for this high level of engagement is that the uniqueness of these programs offers an experience that is different enough from the library's regular programming to encourage a more enthusiastic response.

"
Apples and Oranges: An Indicator for Assessing the Relative Impact of Library Events, continued gallery and adjoining atrium to explore the exhibition, partake in food and drink, and mingle with other attendees. At Faculty Pub Night events, food is provided in advance and throughout the event, and we ask attendees to fill out the feedback forms while they are sitting and before they leave the event. We also encourage attendees at Archives & Special Collections receptions to fill out feedback forms, but at the moment just before they are invited to explore the exhibition (and the buffet). It is reasonable to conclude that many attendees skip the feedback forms altogether so they can partake in the food and gallery walk. Until reviewing the visualizations, this generalization was not obvious to us. Knowing this, we could change the program for future Archives & Special Collections receptions to accommodate more time for feedback forms, thus increasing the response rate and level of confidence in the engagement ranking.
One additional trend presents itself as worth noting. With one exception, all events classified as "Other" (i.e., not Faculty Pub Night or Archives & Special Collections receptions) ranked an engagement rate of over ninety percent. Events in this category include non-standard or ad-hoc programming. One possible reason for this high level of engagement is that the uniqueness of these programs offers an experience that is different enough from the library's regular programming to encourage a more enthusiastic response. Anecdotally, we know that many of our event guests are frequent attendees at other library events (e.g., library staff, faculty champions, student employees). However, without further analyzing and tracking individual attendance at multiple events, we cannot confirm this. It is also just a plausible that the uniqueness of the program attracted an audience wholly different from our usual patron. Once again, these visualizations offer directions for future assessment needs.
When the authors met to analyze the results, we noted the following additional observations: • Events with predominantly off-campus guests (labeled "Other") or audiences with no clear majority of attendees (between students, staff, and faculty) seem to have higher engagement rates. • Events with mostly faculty attendees seem to trend closer to the bottom left quadrant (thus, lower engagement and response rates). • No Archives & Special Collections reception had a one hundred percent engagement rate (although other events did). • All events with more than fifty-five attendees have response rates under fifty percent.
These observations, as well as others not noted in this paper, prompted a number of questions which will be used to further assess and improve library programming, including the following. To what extent does faculty involvement (i.e., their promotion and ability to bring a class) influence these results? What is it about each event that determines its response rate? What are the most important variables to capture in future assessment? One significant area for future research would be to build upon this model using more rigorous data analysis, such as regression analysis, to determine the certainty of the trends and conclusions drawn above. To make these types of analyses possible, future studies would need to improve the feedback rate of program attendees (e.g., requiring feedback during the event). A higher feedback rate would increase the reliability of the results and allow for more complex coding of the engagement level beyond a simple binary instrument. For example, future research could look for indicators of change in attitude, behavior, and knowledge separately. Additionally, future studies should also collect additional data to determine if other factors possibly contribute to engagement, such as: time of day, presence of food, various event formats (e.g. lecture, workshop), expenditures, and staffing.
Practitioners wishing to apply this method for prioritization and assessment can conduct a top-level review of all library programming as we have done, or it can be used in smaller circumstances, such as determining which of a handful of library outreach events needs additional improvement. This method could be employed to justify canceling a program.

Conclusion
In this article, we detailed the development of a convenient and useful indicator for quickly assessing the relative impact of a variety of library events, many of which vary greatly in their format, intent, and expected learning outcomes. Using a widely-used instrument (i.e., survey) and data that is regularly collected by many outreach and programming librarians, this methodology could easily be replicated and expanded by other practitioners. As we have shown, the visualization of these data offers food for thought over which outreach teams can reflect and ruminate to discover generalizations that can inform future outreach work.

University of North Carolina Greensboro
To cite this article: Lenstra, Noah, and Martha McGehee. 2022

How Public Health Partners Perceive Public Librarians in 18 US Communities
ABSTRACT Public librarians are increasingly recognized as community partners who improve the reach of organizations focused in whole or in part on public health promotion. The capacity of librarians to support public health initiatives has previously been studied through case studies of particular communities. Few national studies have considered how and why public librarians are perceived as part of the public health infrastructure. This article analyzes data from interviews with 59 public library partners in 18 communities in 16 states across the United States. These interviews were collected as part of a larger study on how public librarians collaborate with partners to promote healthy eating and active living, or HEAL. Case study selection utilized a purposive sampling technique to recruit public libraries that self-identify as actively involved in public health initiatives. Representatives of those libraries introduced the research team to their community health partners. Findings indicate that in these communities, librarians are seen as trusted connectors, community experts, and as professionals that share goals with public health partners. Nevertheless, the strength of these partnerships is diminished by several factors. The discussion focuses on how a) increased knowledge and b) more strategic conversations on this topic, both within the public health and the public library sectors, could contribute to building better collaborations, locally, regionally, and nationally. Building and sustaining these collaborations could, in turn, help public librarians make more strategic and effective contributions to public health issues that appear both in their workplaces, and in their communities.

KEYWORDS
Collective impact, community partnerships, health promotion, public libraries, qualitative research, community coalitions, health coalitions The public librarian may play any of several roles in a community-wide action system: information specialist, catalyst change agent, interpreter of community need, channel to community resources, expert in planning and group process. . . . The versatile librarian may exercise leadership and bring library resources and services to bear in a variety of ways -Margaret E. Monroe, a public librarian before becoming a professor of Library Science at the University of Wisconsin-Madison, Library Trends, 1976 I n 2017, the health-focused Robert Wood Johnson Foundation characterized "public libraries" as one facet of community-based "cultures of health," alongside "housing affordability, access to healthy foods, youth safety, residential segregation, early childhood education, complete street policies, and air quality" (Chandra et al. 2017). Despite being increasingly framed as part of our public health infrastructure, public libraries and public librarians are not widely studied as partners within the public health research literature. Within that literature, the topic of the perception of librarians among health partners remains unexplored.
Existing evidence suggests that health partners tend to focus more on the public library as a site than on public librarians as partners. For instance, within the sub-field of public health focused on prevention, or "intervening before [negative] health effects occur" (Centers for Disease Control & Prevention n.d., 1), public libraries have been studied as sites for Play Streets (Umstattd Meyer et al. 2019), healthy aging classes (Matz-Costa 2019, 1007-1016), and summer meal and nutrition programs (de la Cruz et al. 2020(de la Cruz et al. , 2179(de la Cruz et al. -2188. This literature tends to focus on the potential of the public library as a trusted community space, and not on public librarians as active community agents. This article aims to empirically understand how public librarians in particular communities are framed by the organizations that work with them to support public health. The focus of the partnerships studied is the promotion of what public health professionals call HEAL, or healthy eating and active living (Journal of Healthy Eating and Active Living, 2021). Results, derived from qualitative interviews with partners who have worked with public librarians in 18 communities across the country, illustrate some of the strengths, weaknesses, and opportunities associated with these partnerships. These case study results lead into a discussion of the further work needed to integrate the public library sector more fully into our understanding of public health infrastructure.

Literature Review
What Is Public Librarianship? Perceptions and Realities. Public libraries are dynamic, socially responsive institutions that change and evolve along with their communities. A study commissioned by the American Library Association found that over 20% of public libraries offered fitness and nutrition classes in 2014, primarily by leveraging community partnerships (Bertot et al. 2015, 270-289). As these public health partnerships have become more widespread, they have prompted public librarians to reassess what skills are critical to being a public librarian. The Public Library Association (2018) found that the second most needed job skill in the profession is how to be a "Community Liaison/ Partner." Public librarians increasingly work as community partners to address topics as diverse as homelessness (Terrile 2016, 133-146) the opioid crisis (Allen et al. 2019) early childhood development (Tilhou et al. 2021, 111-123), the reading gap (Pasini 2018), and adult education (Daurio 2010).
Although the idea of public librarians as community partners has received increased national attention over the last decade, it is not a new idea. In the 1960s and 1970s, work by scholars such as Margaret E. Monroe at the University of Wisconsin-Madison analyzed the various ways in which public librarians participate in community organizing efforts (Monroe 1976), finding that librarians across the country work creatively and nimbly alongside their partners.
Nevertheless, a gap in our knowledge centers around the perception of public librarians among actual and potential community partners. Scattered evidence suggests that public librarians are typically not considered as community partners on contemporary community concerns. Aldrich (2018) notes in her analysis of media representations of public librarianship that, "rarely does a writer miss the opportunity to speak to her own nostalgia about libraries, the printed word, and the quiet solitude of the libraries of her youth" (1). She argues these media messages make it difficult for librarians to be seen as community partners; she also points out that librarians struggle to embed community outreach and community partnerships into their work. Empirical work supports the idea that librarians are not always seen as community partners, even in core areas like literacy. In a study on adult literacy partnerships, Daurio (2010) concluded potential partners "did not see the library as a partner" (ii). This finding was confirmed in a recent study of library partnerships relating to the opioid crisis (Allen et al. 2019), wherein researchers found that potential partners did not think of librarians until librarians reached out to them. A report commissioned by the American Library Association found that most voters do not see public librarians as individuals who are well known in the community, knowledgeable about the community, or understand community needs and how to address them (OCLC and American Library Association, 2018, p.10). The literature suggests those working outside of libraries would generally tend not to see public librarians as community partners, unless librarians first suggest the idea to them.
Public librarians as HEAL partners. Despite the absence of a national conversation on public librarians as community partners, over the past decade an emerging research literature has highlighted how, in particular places, public librarians do work with partners to promote public health, including in the domain of healthy eating and active living.
A state-wide study in South Carolina found librarians there already doing initiatives "around healthy eating and active living and [wanting] to do more" with community partners (Draper 2021, 1). A state-wide study in California found that librarians there recognized a need for a summer meal programs, and were thus motivated to serve meals at libraries in collaboration with summer meal sponsors, such as school districts (de la Cruz et al. 2020).
Similar findings have emerged from studies of particular communities. An Appalachian Regional Commission (Cecil 2018) study highlights how in McCreary County, Kentucky, library director Kay Morrow "understands that the library is an important component of a community that can offer a lot more than books …. The library's meeting room serves as a place for healthy-cooking classes …. Always eager to make a better life for residents here, Morrow is spearheading efforts to rebuild the crumbling sidewalks downtown, secure more lighting at night, and organize a downtown walking club to boost physical activity." (Cecil 2018, 49) McGladrey, M., et al. (2019) examine the efficacy of a multisectoral approach to development of rural physical activity promotion coalition in Clinton County, Kentucky, concluding that public librarians are key participants in multi-sector efforts to increase physical activity in rural America. In Eastern North Carolina, Flaherty and Miller (2016) discussed how the Farmville Public Library director worked with a parks and recreation department and a university public health department to start circulating pedometers and to organize the town's first 5K fun run. In rural Oklahoma (Umstattd Meyer et al. 2019) and Columbus, Ohio (Adhikhari et al. 2021), two separate research teams independently found public librarians to be willing and eager participants in multi-sector efforts to bring Play Streets, temporary closures of streets for active play, to their respective communities. Bedard, Bremer, and Cairney, (2020, 101-117) recruited four public librarians in Southwestern Ontario to become trained Move 2 Learn program leaders, demonstrating "the feasibility of teaching staff without specialized training [i.e. librarians] in physical education to implement" (114) a physical literacy intervention. Also in Canada, kinesiologists made 90 pedometers available for circulation from five public libraries, finding libraries to be ideal sites for this form of physical activity promotion (Ryder et el. 2009, 588-596). Freedman and Nickell (2010) studied the impact of after-school nutrition

How Public Health Partners Perceive Public Librarians in 18 US Communities, continued
workshops in a public library. Sandha and Holben (2021) analyzed stakeholder perception of a summer meal partnership at a rural library in Mississippi. Together, these studies give us some glimpses into how those outside public librarianship frame librarians as health partners, but since the partnership itself was not a central focus in these studies we are left without any in-depth understanding of the perceptions of the partners working with the librarians.
This study seeks to apply this literature to assess how librarians are perceived by the organizations with which they work to advance HEAL outcomes: Research question: How do partners that work with or include libraries in HEAL initiatives frame libraries and/or librarians?

Methods
Case studies show how certain practices are developed in specific communities and, therefore, help elaborate theories related to those practices (Ospina et al. 2018). Qualitative case studies allow the study of research questions in depth, while leaving room for unexpected, interesting findings that can form the basis for concrete hypotheses to be tested in future research (Yin 2013). Case studies are especially useful when there is little existing research on a topic, as is the case here. Case study research has been successfully used in the public library research literature, most recently by Coleman, Connaway, and Morgan (2020) and by Norton, Stern, Meyers, and DeYoung (2021). The former studied how in eight communities, public librarians worked with others to respond to the opioid crisis. The latter studied how in 12 communities, public librarians support social wellbeing. The goals in these and other case studies are to identify and articulate practices and trends that can be further elaborated in subsequent studies.
Case study research has also been widely used in the field of public health, which has as one of its goals conducting "epidemiological surveillance," or "the systematic collection, analysis and dissemination of health data for the planning, implementation and evaluation of public health programmes" (Thacker, Parrish & Trowbridge 1988, 11). Over the last thirty years, public health researchers have recognized and struggled with the limitations of existing surveillance systems, leading to a call for more case study research on how cultures of health emerge from the ground up in particular places. Most notably, the Robert Wood Johnson Foundation funded a series of case studies on what they call sentinel communities, geographical communities selected not because they are normal, but because they may be unique, because they may offer researchers the opportunity to observe how a culture of health takes hold and evolves at the local level in a particular place (Chandra et al. 2017).
The broader study of which this article is a part has the goal of understanding "how, why, and with what impacts do public libraries collaborate with others to co-develop programming around healthy eating and active living?" (IMLS 2020). To answer that question, public libraries in 18 communities across the United States (Table 1) were purposively sampled to try to secure representation of an array of community types and regions. The purpose sampling of communities emerged in part through public librarians in these 18 communities self-identifying as communities involved in multi-sector HEAL promotion efforts through a call for participation circulated online in the Let's Move in Libraries newsletter in February 2020.

"
The literature suggests those working outside of libraries would generally tend not to see public librarians as community partners, unless librarians first suggest the idea to them.
The participating libraries are in 16 states, and serve a range of communities, with the largest library serving a population of 2,095,545 and the smallest serving a population of 12,960. Like libraries nation-wide (IMLS 2021), most of their funding comes from local governmental sources, with some exceptions, such as the McArthur Public Library, which as a 501(C)3 nonprofit receives large amount of revenue from donations, and Delaware's Laurel Public Library, which like other Delaware libraries, receives a substantial amount of revenue from the state government. The total revenue libraries have per capita also varies widely, with a high of $88 per person per year at Elgin, Illinois, and a low of $9 per person per year in rural Rutherford County, North Carolina. Per capita library funding serves as a barometer for both the political climate of a community and its relative affluence.
In these communities, the identification and recruitment of public library partners for interviews emerged through interviews with public librarians.
Librarians introduced the research team to their partners. The 59 partners interviewed (Table 2) represent a heterogeneous array of community partners -including local non-profits, public health departments, parks and recreation agencies, and K-12 schools -that work with public librarians in these communities. As with any case study research, these interviewees represent a small number of the potential respondents at their organizations, and therefore their experiences cannot be generalized as the experience of the entire organization. The research team did not construct a sample of potential partners to interview but instead interviewed partners through the case study process of identifying key stakeholders (Yin 2013).

How Public Health Partners Perceive Public Librarians in 18 US Communities, continued
The interview guide was developed from the Wilder Collaboration Factors Inventory, a widely used tool to understand how different sectors collaborate in communities (Perrault, et al. 2011). The guide was further developed based on the first author's previous work on this topic (Lenstra 2018, Lenstra and Carlos 2019, Lenstra and D'Arpa 2019, as well as with the input of the project's advisory board, which includes experts from both the public library sector and from the sectors that would engage in the interviews as partners (e.g. public health, parks & recreation).
The recorded interviews, which took place over Zoom in Fall 2020 and Spring 2021, were semi-structured and based around a series of prompts designed to elicit narratives about the development and utilization of public library partnerships, and of the roles of particular individuals, including the interviewee, in those partnerships. These methods received IRB approval from the UNCG Office of Research Integrity. The protection of stakeholder identities in case study research is a complicated process, particularly when communities are named (Yin 2013). Coleman, Connaway, and Morgan (2020) discuss these ethical dilemmas in their research on public librarians and the opioid crisis. All efforts have been made to protect the privacy of interviewees, but they were informed there is a risk of being identified. This study's IRB application was modeled on that used Coleman, Connaway, and Morgan (2020), and one member of their research team served on the advisory board of this project and provided input to this project's ethical framework (additional details in Allen et al, 2019, p. 25).
Data analysis drew upon the case study tradition of qualitative analysis (Yazan 2015). Transcripts were analyzed to develop case study narratives about how partnerships formed, impacts, and how they were sustained over time. Simultaneously, the P.I. and graduate student researchers used grounded theory techniques (Charmaz 2014) to extract themes that cut across the different conversations and cases. Table 3, below, which conceptually lays out the framework developed from this iterative coding process, emerged from four months of intensively moving across the three levels of analysis (interview quotation, thematic code, theoretical memo), until the research team came to a consensus about the nine themes that encompass the range of attitudes partners conveyed about their experiences collaborating with public librarians on public health initiatives. Each of these themes is illustrated below using a representative example from the different case studies.

Limitations
As with all case study research, this study does not claim to offer generalizable trends. At every level of sampling (community, partner organization, partner representative), purpose sampling techniques were deployed that undercut generalizability. It is impossible to extrapolate from a case, or from 18 cases, to make broad conclusions on a topic. Future research will need to do that extrapolation, and the discussion section concludes with a call for precisely that.

Findings
Across the interviews, libraries are seen as trusted connectors (Table 3). In some cases, though, the partnership is diminished because of weak ties to the institution. An opportunity identified is to cultivate more connections between public libraries and partners. Public libraries are seen as community experts. Weakening this perception is the idea that library partnerships are aberrant. An opportunity emerges to cultivate more awareness of transformations in public librarianship. Partners see librarians having shared goals with them. Weakening this perception is the fact that other librarians do not share those goals, with a related opportunity being to cultivate more HEAL champions within the library workforce.

Section 1: Connections
Trusted connector. Since 2009, the staff of the Laurel Public Library have worked to cultivate a reputation as a trusted community connector, with that work leading to transformations in partner perceptions. An early institutional partner was the University of Delaware Cooperative Extension. An Extension agent said that although he has worked in Laurel since the 1990s, he did not perceive the library as a connector until 2009. He now sees the library as: "Instigators. So basically I reached out to the library and said, 'Can we use you?'" As a result, the library became the host of the Extension's 4-H program, and as that relationship developed it led to the library and the Extension working together to transform the built environment in 2014 (Figure 1). Another of the library's long-term partners, a faith-based organization, remembered that: Table 3: Strength, weaknesses, and opportunities associated with public libraries as HEAL partners. Themes developed from qualitative analysis, see Methods, above.

How Public Health Partners Perceive Public Librarians in 18 US Communities, continued
"The first big thing that... we partnered with them to do [was] to put exercise stations in a local park down the street. They got the grant. They got the equipment shipped in. I put people together to get it done. And it still is used today. That was one of the first and biggest things we did together." Since 2014 the library has extended their connections, offering nutrition classes in partnership with the Delaware Food Bank (a SNAP-Ed implementing agency), becoming a summer feeding site in 2017, adding indoor exercise equipment and 2019, and during COVID-19 starting a Farm-to-Patron initiative where extra produce from surrounding farms is dropped off at the library for anyone to take.
Weakly connected to partner. Since 2006, staff of the Gail Borden Public Library in Elgin, Illinois, have participated in Activate Elgin, a city-wide initiative to engage all sectors of the community to provide opportunities to improve health, particularly around HEAL. One librarian had been the key liaison to Activate Elgin since 2009, and when she retired in summer 2020, the partnership was put into jeopardy. The combination of the COVID-19 Pandemic and the retirement of a key staff member illustrates weaknesses that can emerge when HEAL partnerships are dependent on particular individuals.
A community educator at a local hospital stated she was, "heartbroken when I heard that [the librarian] was leaving, because we have a super good relationship." At the time of the interview, she did not know if the library would appoint a new representative to Activate Elgin. She said that during the pandemic she has been thinking about, "how can we continue to work with the library? [For example] can I download or check out a DVD from the library that would lead me in yoga because I can't go in and see my yoga instructor? Can I go check out a cookbook that would have some healthier recipes? So what can we do? How can we partner together?" She said that she is unable to answer these questions because she no longer has a contact at the library. Having lost a key contact in the library, she feels the partnership has ground to a stand-still. The future of the library's role in Activate Elgin is uncertain.
Cultivate more connections. The McCracken County Public Library in Western Kentucky has been a key player in multi-sector coalitions organized by a local hospital and the United Way. As the library director became more involved in these coalitions, she sought to involve library staff at all levels. The leader of the Healthy Paducah community coalition said that as a result of her efforts the library is "so visible in the community." As much as possible, library staff spend time outside of the library, attending community meetings, doing programs at farmer's markets, and bicycling around town on their 'Brary (short for library) Bike.
This example illustrates how the library director empowered staff to cultivate connections with partners. A youth services librarian shared the story of how the library became a summer feeding site through her community connections: "[It] started with a conversation I had at the food bank, when I was volunteering there with the nutrition coordinator from the school. I was at the food bank because [a local nonprofit that] was bringing meals to the library parking lot. [The nonprofit] put out a call for volunteers, and since I knew him through his work in the library, when the call went out, I decided to volunteer." Throughout the interviews with librarians and partners of the McCracken County Library, stories like this one occurred again and again. Partnerships lead to partnerships, creating a dense weave of different institutions working together to address persistent community health issues. The leader of Healthy Paducah said they "would be lost without them [library staff]."

Section 2 : Community expertise
Librarians as community experts. In the sprawling jurisdiction of Harris County, Texas, staff from Harris County Public Health see the public library as their "go to partner" for everything from mosquito control and testing to childhood obesity prevention. This intergovernmental partnership began around 2005 with jointly hosted "kid dance parties.... We've had smoking cessation, we've had exercise, family nutrition, and it's just grown through the years," particularly once the library became a member of the health department's Healthy Living Matters coalition.
Three staff from Harris County Public Health were interviewed. In 2015 they started working on creating Mobile Health Villages that include free check-ups alongside fun activities like active play stations and farmers' markets. From the beginning, library staff were involved in planning: "I had met with the library early on. We started partnering with Harris County Public Library because we felt they had tremendous reach into the community. All you have to do is look at their branches to know what the needs are in that community. That was one of the reasons we wanted to work with them. And they've been such a good partner [with the Mobile Health Villages] since then. They make it easy, and we've established so many different kinds of partnerships on so many different levels [with them]." Throughout the interview, they identify libraries as valuable partners because of their expertise on community needs. Health department staff later stated that library staff are "in touch with the community, integrated with target communities, they know how to connect with everyone in the community," and "the community that we're trying to target already perceives libraries as much more of a resource than a place where you can get a book [and] not only is the library a resource for us, but we're a resource for the library." The shared goals at the heart of this partnership will be returned to later in this article.
Partner library seen as aberrant. In Clinton, Massachusetts, the library director has been an avid proponent of HEAL partnerships, even serving on a multisector HEAL committee convened by the Community Health Network of North Central Massachusetts. Nevertheless, partners tend to see their library partner as aberrant, an exception rather than the norm.
The local hospital started working with the library in 2017 to co-sponsor a Walk with a Doc(R) program. The library had a walking club, and the hospital added their program on top of that. Asked how that partnership became established, the hospital's community health specialist stated "we like to collaborate with non-traditional organizations that we wouldn't typically partner with in the community." The framing of the library as an organization a hospital typically would not work with recurred again and again throughout the conversation.

How Public Health Partners Perceive Public Librarians in 18 US Communities, continued
This attitude appeared in other interviews in this community. The food bank coordinator said her partnership with the library, focused around cooking classes, emerged "because [the library director is] so open to it. When I look at her, I really don't look at her as a librarian. I guess because I have a stereotype in my head about what that means. She actually is more of a community advocate, and she's kind of turned that whole position into that." The framing of "community advocate" and "librarian" as separate roles illustrates how partners, even as they work closely with librarians, see those partnerships as aberrant.
Cultivate awareness of public library transformations. When a new director of parks and recreation moved to Scotch Plains, New Jersey, the second person he met was the public library director. From that moment, the public library and parks and recreation department have worked closely together on everything from StoryWalk installations in parks to taster classes of recreation center offerings provided for free at the library. He stated, "at the end of the day they have resources I can't get," including their community expertise.
His awareness of the public library as a partner was not shared with his predecessor. According to the library director, there were no park-library partnerships until the new director came to town. His success, and his knowledge that not all parks and recreation personnel share his recognition of librarians as community experts, has led him to seek to inspire others. At the time of the interview, he was working: "With the New Jersey Recreation and Parks Association on a [continuing] education opportunity, 'Leverage the Library.' I have a whole outline for it. It's something that I've considered, how to work with your library: Obviously, you need to have trust. And, obviously, you need to understand that you're going to benefit as much as they're going to benefit. There are all kinds of ways to leverage and work with them and, and provide the programs and facilities that can benefit both [partners]."

Section 3 : Cultivating shared goals
Shared goals. In Anne Arundel County, Maryland, a community health educator who has worked with the health department since 2000 said that during that time she has always seen the library as "a spot to hold classes and meetings. It was a location to be at, rather than a deep, deep partnership." This transactional relationship evolved over time into a "deeper partnership. Connecting [with library staff] about how to work more together" which led to the realization that both partners have the fundamental goal of "better serving the community." The realization of shared goals emerged through a community coalition. The coalition was "key in opening up the connection between [library staff] and me. The [librarian] is an active participant in those meetings, and so I got to know what she's trying to accomplish, and then how she can help [meet our goals]. Being part of coalition meetings: That's something that libraries do, they are active participants, I really wanted to emphasize that." Her desire to emphasize librarians as active coalition partners emerges from her reflecting on the fact that earlier in her career she merely saw libraries as passive spaces.
Asked to give an example of what kinds of shared projects emerged through the coalition, she responded:

"
When I look at her, I really don't look at her as a librarian. I guess because I have a stereotype in my head about what that means. She actually is more of a community advocate, and she's kind of turned that whole position into that.
"They even helped us with some of our research: We did a food assessment and we utilized the library staff in designing this project. In the food pantry that we are working on, [we asked] 'How can we have a better volunteer system?' [The librarian said] she runs a volunteer system for the library. So we connected with her about how to develop that volunteer system for the food pantry. She's got great experience, and advised in an important way." By cultivating awareness of their shared goals, these partners work together to develop solutions.
Other libraries don't share goals. In Biddeford, Maine, the library works with the Coastal Healthy Communities Coalition, a SNAP-Ed implementing agency. A Nutrition Education Program Manager shared both her positive experiences working with the Biddeford library, and her struggles securing similar partnerships in other parts of her service area. She said the adult cooking programs she had at the library have "the most diverse class I've ever worked with. When it comes to age, race, ethnicity, gender, it was very diverse, which I think is a sign that they're doing something right [at the library]." Based on this success, the Nutrition Educator naturally sought out similar partnerships in other libraries, but has thus far been unsuccessful: Cultivate more champions within the library workforce. Before moving to Western Montana in 2005, the director of the Belgrade community library worked in the corporate sector, and there became passionate about workplace wellness, eventually becoming a part-time fitness instructor with training from the YMCA. As a library director, she has infused the principles of workplace wellness into her leadership, and in the process has cultivated champions of HEAL within her workforce. She said that workplace wellness is "part of how I live and work and breathe. It's a natural thing, a natural component of being a librarian." She empowers her staff to see health as a priority, for themselves, and for communities.
One of her initiatives has been to work with the town government to secure paid walking breaks not only for library staff, but for every employee of the town of Belgrade. For her, the library can not only be a space that cultivates wellness among library staff, but can also be a community hub for health and wellness. These efforts culminated in the library securing the title of Library Journal's Best Small Library in American in 2015. These efforts have led to the library being seen as a partner by everyone from the senior center to the regional hospital. By foregrounding the importance of workplace wellness, this library leader sets the stage for librarians to become champions of HEAL partnerships.

Discussion
Public librarians are increasingly recognized as community partners work with others in their communities to support public health (Allen et al. 2019), including around the promotion of healthy eating and active living (McGladrey 2019, 62-67). This study found that partners in these case study communities see librarians as individuals who help them increase their reach, while also creating opportunities for new voices to be heard in community planning.
By extending the lens beyond a single community or intervention (e.g. Bedard et all. 2020, 270-289;de la Cruz et al. 2020de la Cruz et al. , 2179de la Cruz et al. -2188, this study How Public Health Partners Perceive Public Librarians in 18 US Communities, continued broadens the national conversation about public librarians as partners in the public health infrastructure. Although much more is needed to understand this topic, this study has set the stage for future research on the unique roles of this poorly understood (Aldrich 2018), if ubiquitous (IMLS 2021), social infrastructure (Klinenberg 2018).
The idea of public librarians as community partners on heterogeneous community concerns has been part of the research literature since at least the 1970s (e.g. Monroe, 1976), and yet there is still much to learn about why in some cases librarians partner with others while others do not. This study shows how in some cases partners work well with some libraries but struggle to connect with others, in others librarians struggle to sustain partnerships across staff turnover, while in other cases strong leadership and investment in partnerships by library administrators support this practice.
This research could be extended by surveying the membership of national organizations that represent the professional interests of the local organizations interviewed in this project, such as the National Recreation & Park Association, Feeding America, the American Public Health Association, Partnership for a Healthier America, Alliance for a Healthier Generation, the National Association of County and City Health Officials, the Society for Public Health Education, the Farm to School Network, among others. Such a survey could use the perceptions identified in this study as a starting point for more systematically evaluating how public librarians are perceived by others working in communities across the country to promote healthy eating and active living. The research could be extended even further to more systematically understand how potential partners more generally perceive public librarians as community partners. Much work remains to be done, and this study does not claim to be the definitive research on this topic.

Implications
To ensure the power of public librarians is fully leveraged in multi-sector initiatives, it is important to understand the characteristics of successful partnerships, as well as what motivates partnerships. One promising practice is the identification and/or cultivation of health champions within the library workforce, as well as finding ways to more strategically educate those outside of librarianship to the reality of librarians as health partners. This work may require over-turning stereotypical ideas of libraries and librarians (OCLC and American Library Association, 2018) within the perceptual frameworks of partner organizations.
Beyond addressing perceptions of librarians, work could be done to better institutionalize "partnerships" as a core facet of public librarianship. Library leaders could share how they support partnerships at their libraries, as well as how they make investments of time and resources to enable library staff to participate in community coalitions and in other settings that would enable library staff to build relationships with others in their communities.
Within partner organizations, coalitions play a vital role in bringing librarians to the planning table. A concrete tactic would be to encourage anyone organizing or leading a health coalition anywhere in the country to, at the very least, reach out to their local public library to see if anyone on staff there may wish to attend a meeting, or join the coalition. Public librarians can also be on the lookout for such convenings. A convenient way to identify such health coalitions is through regular library participation in general community organizations --such as United Way, Chambers of Commerce, or the Rotarythat will typically include overlapping memberships with health coalitions.
More generally, this study suggests that a promising practice for public librarians is to simply talk more about public health. The results of this research suggest that the more public librarians talk about public health within their institutions and within their communities, the more potential partners see them as partners. The power of conversation is not to be under-estimated in terms of its capacity to change cultures of health.

COVID-19 Addendum
This study was conceived and proposed before the COVID-19 pandemic's arrival in North America. All the interviews were conducted during the pandemic. The fact that public and community health workers were willing to take time out of their efforts to combat the pandemic to talk about their experiences partnering with public librarians illustrates the critical nature of these partnerships to the work of public health, in both good times and bad.