21 An Example: Building a Summary

In this chapter, we will go through one technique—a reasonably useful one—for building a careful critical summary of several sets of research findings on a single question: “How much violence is in popular media?” The question of violence in media is both important and trivial. Trivial, because the summary of research for any policy question—or any discussion of any research question, for that matter—would go through a similar process of incorporating results into a description of what is known. Important, because the question determines which research articles are relevant to include and which are not.

To mimic the actual process, for each article, we will start with the notes taken from the methods and the findings of one of the research papers. Generally speaking, the introduction, the review of literature, and the discussion are not included in the summary.

(Note: The introduction generally establishes why a particular research paper is a) a contribution to an important practical problem, b) an important theoretical or methodological problem, or c) an intrinsically interesting problem. Since the starting point of the summary is “How much violence do popular media show?” the reason why each study selected is relevant to the summary is already implicitly given. The review of literature is essentially hearsay evidence,^[1] or what the author(s) of the article feels is important in setting up why they are doing the study. While I frequency use the literature review to look for additional papers to read, I generally go to the referred article itself before using any finding cited in a literature review. The discussion section is essentially where the researcher is reviewing the findings and/or extending the findings to other situations (either theoretical or practical). Since the nonrepetitive portion of the discussion section is essentially hypothetical, information from the discussion section is not valid as findings. Hence, the summary should be careful to summarize information about how the findings were developed (the methodology section) and what the findings are (the results or findings section).

Box 21.1 is detailed notes from “Trends of Sexual and Violent Content by Gender in Top-Grossing U.S. Films, 1950-2006,” a content analysis study of top grossing films (Bleakley, Jamieson, & Romer, 2012). The notes start with a check on the soundness of the study, move on to a discussion of study weaknesses, and then to the findings.

Discussion of the Study

Bleakley et al. selected samples from Variety’s annual list of top-selling movies. Variety uses data compiled by Nielsen EDI and Rentrak Theatrical figures, two of the most reliable sources of data for television viewing and box office (movie) revenue. Variety is the premier source of entertainment industry news. Industry leaders closely watch it, which suggests that errors in the lists would be noticed and corrected.

Further, since the researchers sampled the most popular movies, they also captured the movies with the widest audience attention, which, arguably, have the most general impact. (Films that appeal to smaller niche audiences are—by definition—less likely to be sampled.)

The first look at the research findings should be an answer to the main question of the research summary—in this case, is there violence in media? Looking specifically at Bleakley et al., the reader can say—with a fair degree of confidence—that yes, there is violence in top-grossing films and, in fact, most of the films studied were violent (89 percent). Including the words “top-grossing films” in the write-up also gives a fair clue to the careful reader on how the sampling limits findings (see Box 21.2).

The next step is to look for interesting details that show the scope of the violence, including the differences in other relevant factors, such as gender and time. (Both genders started and received violent acts, males slightly more often than women, and all violence increased over time.)

This particular study included other findings which were less relevant to the major question. Generally, only findings relevant to the major question of the summary are included.

Introducing New Material into a Research Summary

Each additional research study should introduce new findings, which will either reinforce, expand, or limit the scope of what can be said about a particular topic. The next study’s research question, “What is the level of violence in crime films set in the current world (i.e., excluding historical, science fiction, war, sports, and horror movies)?” is both clearly relevant to the policy question, but from a more limited population. Therefore, we automatically know that the second study will likely underestimate violence in media by not including several genres of film known for having violent content (e.g., horror films, war films) as well as missing violence in other types of films (e.g., romance films) that are more likely to include emotional violence. (Note: the study method in Box 22.3 only tracks physical violence; emotional violence is not included.) Since errors run in one direction—that is, underestimating violence—the careful reader can say that the overall violence is probably more prevalent than what the study suggests.

The Gosselt et al. study (2015) sampled a different population (coding synopses of crime films) than the Bleakley et al. study (top grossing films) (2012), but similarly to Bleakley et al., the synopses of films also likely underestimated the actual acts of violence, given that synopses writers are more likely to overlook a violent act that is not central to the plotline than to introduce violence that did not appear in the film. Again, a literature summary should include enough information that the careful reader can determine bias introduced in the sampling (See Box 21.4). This information is contained within the initial description of the study sample “an analysis of synopses of present-time crime genre movies.” The researchers obviously will not tell the reader that the population they studied is not the population that the reader is interested in; readers will have to do that work on their own.

In terms of what findings to include, the literature review should include a description of the level of violence found (the main answer to the literature review question) and additional specific information relevant to the main question. Any of the following would qualify as fleshing out the main question:

Gun violence was, by far, the most common type of violence (54 percent) and guns the most lethal (67 percent were fatal shootings).

Both men and women were aggressors and victims, but men were far more likely to be the victim (79 percent) and the aggressor (80 percent). Women were aggressors or victims 13.1 percent of the time for each category.

More recent films were less likely (slightly) to have nonlethal violence.

A rewritten literature review (see Box 21.4) combining both studies should first state the new basic answer to the guiding question, and then point out interesting supporting details from both studies. (The words retained from the description of the study in Box 21.2 are in black, and additional information is in green.)

The findings reported should support or qualify the major claim of the topic sentence and not wander off onto other topics. The exact findings used for supporting details will probably vary from writer to writer. For example, Box 21.3 included evidence that supports the claim that movies show a consistently high level of violence, but did not discuss guns or blades versus physical contact (hands, feet etc.). That information would have been fine to include because it helped characterize the violence, but not absolutely necessary. Including that information, however, would be absolutely necessary if the main question for the policy analyst was, “What is the level gun violence in media?”

Looking broadly at the main question of violence in media, the two studies discussed so far only cover one medium (movies) and one particular type of violence (physical). As such, the studies’ findings are an incomplete answer. The two additional studies broaden the scope to include other types of media (music videos and TV reality shows) and other types of violence (emotional/psychological violence).

Introduction of a Third Study into a Summary of Research Literature on Violence in Media

The difficulty of adding the findings for the study on media violence, is how to handle questionable findings.  For the first time, the study is both old and problematic. First, the authors do not discuss how television channels select the music videos they show, so the direction of any sampling bias on the part of the TV producers is unclear. Second, the data was collected in 1997. There is no substantive reason to believe that music videos are the same now as they were then. Each of the television channels listed in the methods section of the research paper, “Violence in Music Videos,” have substantively changed their programming, and most music videos are on distribution channels (e.g., YouTube) that have far weaker content restrictions than mainstream television. So, what to do? Should you throw the data out because it is flawed? Well, no. It is not flawed enough that the research produced has no value. The study data shows that in 1997, music video violence on three channels (that were the major music video channels at the time the research was done) was relatively low. This is a valid finding for over twenty years ago—even though it does not demonstrate the incidence of violence in current music videos. A good literature review must include all sound relevant research studies, whether the studies agree with the majority of the rest of the findings or not.

Turning to the specific findings, the music video findings significantly weaken the claims that can be made about violence in media. Instead of saying that there is a uniformly “high level of violence,” the overall assessment will need to be softened to accurately reflect the additional knowledge (see Box 21.6, in red) that one study showed some (but not much) violence. The writer also included the year of the study, so that a careful reader will know how dated the research is.

Introduction of a Fourth Study into a Summary of Research Literature on Violence in Media

The Wilson, Robinson, and Callister (2012) study on the television reality show, Survivor, extended the definition of violence to include emotional aggression. This study examined multiple episodes of a single television reality show and found multiple instances of verbal, deceit, and indirect aggression (defined as acts that take “place behind the victim’s back”) (Wilson, Robinson, and Callister 2012).

The addition of this study means that the topic sentence would need to include both physical and emotional violence, and specific findings about the prevalence and kind of antisocial violence would need to be added in the evidence section (see Box 21.8; the additional changes to the literature review are in brown).

Finally, the writer should end the summary with any overall assessment that the writer feels is necessary to highlight for the reader, including important limitations of the research findings. For example, the research studies examined do not include all media. In particular, they do not cover digital media—YouTube, Instagram, or Snapchat. This means that the studies have ignored important distribution channels that systematically have fewer restrictions than traditional media. Since it is probable that the content of less regulated media is more violent, the lack of digital media studies is an important limitation that would bias the overall results. Overall, the literature review should give the reader an accurate sense of the answer to the major question, “Research consistently found violence in all media studied,” and enough evidence that the reader would agree with the topic sentence based on the evidence alone (Box 21.8) without actually reading the topic sentence. There is no specific, hard-and-fast rule as to which exact topic sentence the writer can choose to develop and what evidence the writer chooses to include, but there are some shared guidelines for what is acceptable in the topic sentence and what is not. No one, that is, no one, who is a rational reader can finish all four studies and say that the media showed no violent content. A writer who suggests no violent content would be delusional or lying. A writer who suggests that movies only showed limited violence would be misrepresenting the level and seriousness of lethal violence in the movies. Anyone in this culture who does not accept that lethal violence is severe is fundamentally detached from shared social reality.

Writers, however, have some choice on secondary research themes, such as:

The interaction between violence and gender (covered in most studies),

Whether or not the violence shown has consequences (an important factor in how likely viewers will model behavior they see in media and covered in some studies), and

The interactions between violence and race (an important issue in social justice and covered in most studies).

Any or all of these themes would be acceptable subcomponents of the larger theme of violence in media.

The research-by-research rewrite of a research summary is not necessarily the only method to use to construct a summary, given that people have different writing strategies. However, all summaries should have the following structure and inclusions:

1. 1. For each sound research paper, the findings of the paper relevant to the main question should be presented.
   2. The literature summary should start with a topic sentence that answers your main question: What did the research papers find?
   3. The rest of the paragraph should include pertinent details from individual research papers. The summary should include enough pertinent details that the reader would be able to develop a topic sentence strikingly similar to the actual topic sentence just from the details alone (without reading the topic sentence).
   4. The summary should include shadings and findings that qualify and/or contradict the main summary sentence (topic sentence) of a summary paragraph or paragraphs.
   5. The careful reader should be able to pick out where the population in each study differs from the population of interest in the summary. (This information, while important, is the most likely to be missed in extant summaries.)
   6. The paragraph(s) should end with a description of where the evidence is incomplete or missing.