A Systematic Review of Unpublished and Non-Refereed Music Therapy Mental Health Papers Found via Google Scholar
As Google Scholar searches yield unpublished papers, it may inadvertently impact the perception of the music therapy literature for clinicians, researchers, and service users. Therefore, the purpose of this systematic review was to identify and analyze the current literature comprised of unpublished and non-refereed papers regarding music therapy and mental health from January 2000 to September 2017 located via Google Scholar. After establishing inclusion and exclusion criteria, papers were identified using a variety of combinations of music therapy and mental health keywords. Twenty-one papers met inclusion criteria. Sixteen papers were master’s theses and five were doctoral dissertations. Almost half of the papers (n = 8) involved adolescents with mental health diagnoses. Although not all papers contained data, more papers contained qualitative data (n = 10) than quantitative data (n = 3). The unpublished music therapy and mental health literature may represent a valuable resource for guiding clinical practice and research. As the majority of authors were affiliated with universities outside the United States, perhaps there is greater interest in mental health outside the United States. It is concerning that many identified papers required additional login credentials. Implications for clinical practice, limitations, and suggestions for future research are provided.
Date received: 14 December 2017
Date accepted: 6 June 2018
Publication date: 1 July 2018
Google Scholar constitutes an accessible, powerful, and free resource for expediently reviewing scholarly literature. However, since Google Scholar also includes unpublished and non-refereed papers in its searches (Google Scholar, 2017a, 2017b), many unpublished and non-refereed music therapy papers are retrieved when using Google Scholar to search the literature. Although unpublished and non-refereed papers do not appear in other search indexes, some of these papers located via Google Scholar are available to the public. Additionally, as many people – including clinicians and service use seekers – may not have access to university libraries and sophisticated search engines, they may rely upon Google Scholar as it is a free resource. Thus, if Google Scholar searches result in unpublished literature that has not been subjected to a blinded peer review process, it may influence people’s decisions when using the literature to make informed clinical choices.
Although evidence-based practice emphasizes using the best research evidence available for making clinical decisions (Aarons, Hurlburt, & Horwitz, 2011; Rubin, 2013), clinicians attempting to use evidence-based practice may not have access to university libraries. These clinicians may be forced to rely upon Google Scholar and their conclusions about the status of the literature may be impacted by unpublished and non-refereed work. Anecdotally, as it seems that Google Scholar is widely used and contains many unpublished and non-refereed papers, we were interested in systematically reviewing unpublished music therapy mental health papers found via Google Scholar. Therefore, the primary purpose of this systematic review was to identify and analyze the current literature comprised of unpublished and non-refereed papers regarding music therapy and mental health located via Google Scholar. A secondary purpose was to collate a synthesized resource to provide access to these unpublished papers.
Google Scholar (http://www.scholar.google.com) is an extension of the Google search engine. It was launched in November 2004 (Vine, 2006) with the purpose of providing a simple, efficient, and speedy method for searching scholarly literature (Google Scholar, 2017c). Google Scholar encompasses a wide variety of research including journal articles, conference papers, books, theses and dissertations, court opinions, and patents (Google Scholar, 2017a). As Google Scholar’s search index contains work from licensed journals, many links will provide the abstract but require additional login credentials or a fee to view the full text of an article (Vine, 2006). However, Google Scholar’s search index also locates literature from open-access sources, allowing the user to view the full-text article for no charge. As Google Scholar is free and many service users and clinicians not associated with universities do not have access to academic search engines, Google Scholar represents an accessible and powerful search engine for many people.
Acknowledging that a great deal of scholarly research involves building on what academics have already discovered and contributed (Google Scholar, 2017d), Google Scholar searches both published and unpublished literature. Literature may be unpublished due to publication bias, wherein only studies with favorable and significant results are published. Publication bias is a problematic phenomenon that occurs throughout many fields (Dwan et al., 2008; Joober, Schmitz, Annable, & Boksa, 2012; Niemeyer, Musch, & Pietrowsky, 2013; Van Lent, Overbeke, & Out, 2013) and may be resultant of researchers not wanting to publish negative results, the source of funding for the study, or journals rejecting articles that do not contain significant favorable results. Whatever the reason may be, publication bias is a precarious reality as scientific information and knowledge is withheld from the public. Publication bias has many consequences, including but not limited to: Impacting a consumer’s health due to lack of knowledge about an intervention or treatment, influencing a clinician’s practice techniques, and exhausting a researcher’s time and resources replicating a study that has already been conducted but not published (Joober et al., 2012; Turner, 2013).
Publication bias also impacts evidence-based practice and future research projects. As the current published literature ultimately impacts the direction of future research, without the knowledge of unpublished articles, researchers are only able to base their clinical practice, hypotheses, and proposed studies on a certain set of results. While publication bias is largely perceived as a problem, Google Scholar can be conceptualized as a potential solution because it includes many unpublished and non-refereed papers in its searches. To the best of the authors’ knowledge, there is no music therapy literature concerning publication bias or Google Scholar.
Although mental health remains one of the largest categories of clinical populations with which music therapists work in the United States (AMTA, 2016) and worldwide, the peer reviewed music therapy mental health literature base remains relatively small (Silverman, 2008, 2010, 2015). In a descriptive analysis of music therapists working in mental health settings in the United States, over 50 percent of respondents indicated they had a graduate degree (Silverman, 2007), which would likely indicate they received research training in order to complete their degree programs. In the same study, 84% of participants noted they conducted research. Therefore, if 84% of participants reported that they conducted research and over 50% of participants had graduate degrees that likely included research training, where is this mental health research? If such a large number of music therapists are conducting mental health research while the peer reviewed literature base remains relatively small (Silverman, 2008, 2010, 2015), perhaps much of it is not being published in refereed journals. These papers would be helpful for clinicians and scholars and might be accessible via Google Scholar. Due to lack of mental health and music therapy research, unpublished research would certainly be valuable to better understand processes and products of music therapy within mental health settings. Perhaps searching for unpublished music therapy mental health research via Google Scholar would constitute a way to access the unpublished literature.
Sole reliance upon published literature ultimately impacts the direction of future research. Additionally, without the knowledge of unpublished papers, people are only able to base their clinical practice and research investigations on a certain set of results. To the best of the authors’ knowledge, there is no article addressing the problematic nature of publication bias within the music therapy and mental health literature. The current systematic review represents an initial attempt to locate and analyze unpublished mental health contributions to the literature, increase accessibility to the unpublished research, and call attention to the publication bias surrounding music therapy and mental health. Therefore, the primary purpose of this systematic review was to identify and analyze the current literature comprised of unpublished papers regarding music therapy and mental health located via Google Scholar. A secondary purpose was to collate a synthesized resource to provide access to these unpublished papers. Research questions were as follows:
We identified papers through the advanced search function in Google Scholar using a variety of combinations of music therapy and mental health keywords between March 1 and September 30, 2017. Keywords used included: music therapy, mental health, mental disorder, mental illness, mentally ill, psychiatric, psychosis, bipolar, depression, schizophrenia, schizoaffective, suicidal ideation, suicide, borderline personality disorder, and personality disorder.
Inclusion and Exclusion Criteria
We included papers if they met the following criteria:
Written in English.
Located via Google Scholar.
Not published in a refereed journal.
Accessible without requesting permission from an additional source.
Uploaded between January 2000 and September 2017.
We excluded papers if participants were in a forensic setting, if the population was older adults, or if they were published in languages other than English.
Data Collection and Extraction Process
We created a table for data extraction for papers meeting inclusion criteria. Data extracted and summarized for each study included: (1) Author and year; (2) Institutional Affiliation; (3) Purpose of research (i.e., thesis, dissertation, or non-required research); (4) Author credential (i.e., if the author was a music therapist); (5) clinical population and sample size; (6) type of study and design; (7) findings; and (8) the link to the unpublished paper. Throughout this process, we met frequently and discussed any discrepancies between our interpretations of the extracted data. In the case of the type of study/design, we categorized papers by the study design identified by the author in each paper.
Search and Inclusion Results
The search yielded a total of 550 possible papers. After screening the abstract and determining if the paper was published in a refereed journal, 65 unpublished papers were identified. We then read each paper to determine if it met inclusion criteria. Nine papers met most inclusion criteria but required additional login credentials for access and were therefore not included in the analysis. Eight of these nine papers were written by authors affiliated with universities within the United States. To ensure that each paper had not been published, we also emailed authors in an attempt to confirm their papers had not been published. A total of 21 papers met all inclusion criteria. Results of the search are depicted in Figure 1.
Twenty-one papers met our inclusion criteria. We provide a summary of the results below and extracted data are depicted in Table 1.
The majority of the authors (n = 15) were affiliated with universities outside of the United States, with New Zealand (n = 7) and Finland (n = 3) representing the top two countries.
Of the 21 papers that met inclusion criteria, 16 of the papers were master’s theses and five were doctoral dissertations.
Of the 21 papers meeting inclusion criteria, 19 were authored by music therapists. Two projects were conducted at Ball State University for Masters of Science degrees. Within these papers, we were unable to determine if the authors were Board-Certified Music Therapists. Also, both of these were prospective studies and datum was not collected.
Clinical Population, N
Almost half of the papers (n = 8) involved adolescents with mental health diagnoses. Authors conducted the research in various settings, including inpatient facilities, outpatient facilities, and transitional school settings. Concerning the number of participants in each study, studies varied considerably, and due to the diversity of paradigms and that some papers consisted of multiple studies, we decided calculating descriptive data would not be appropriate. However, we depicted these data in Table 1.
Type of Study/Design
We categorized two papers that were designed but not conducted as prospective studies. Although not all papers contained data, more papers contained qualitative data (n = 10) than quantitative data (n = 3). We categorized papers containing qualitative data as having various designs, including: exploratory (n = 3), grounded theory (n = 2), case study (n = 1), theoretical (n = 1), ethnography (n = 1), action research (n = 1) and a combination of three separate papers (n = 1). We also identified papers using quantitative data as a post-treatment only survey (n = 1), a concurrent schedule/alternating treatments single-subject design (n = 1), and a combination of three separate papers into a single paper (n = 1). Four papers contained both qualitative and quantitative data.1 Two papers did not contain either type of data and were categorized as a descriptive analysis of published music therapy papers and a developed resource for music therapists.
Due to the assortment of types of papers and research questions, findings are mixed and we were unable to synthesize them due to their diversity. However, findings are depicted in Table 1 and summarized briefly below. We also provided links to each of the papers that met our inclusion criteria.
While outcomes of the papers varied, authors reported music therapy as having positive effects on coping with stress (Salur, 2016), emotional regulation (Salur, 2016; Young, 2010), socialization and social skills (Morgan, 2007; Salur, 2016; Travis, 2003; Young, 2010), self-expression (Young, 2010), self-esteem (Karvonen, 2015), relaxation (Travis, 2003), and attendance rates (Davis, 2008). Authors also reported that music therapy assisted in decreasing depressive symptoms (Punkanen, 2011; Salur, 2016), anxiety symptoms (Punkanen, 2011), and psychiatric symptoms (Morgan, 2007).
Other outcomes from the papers include identification of key factors that influence group dynamics (Beatson, 2012), a descriptive analysis of music therapy and mental health research published in the Journal of Music Therapy, Music Therapy Perspectives, and The Arts in Psychotherapy (Moore, 2015), a resource for music therapists working with individuals diagnosed with schizophrenia (Purvis, 2007), and a description of the relationship between music therapy practice and key competencies of a school curriculum (Halligan, 2012).
The purpose of this systematic review was to identify and analyze the current literature comprised of unpublished and non-refereed papers regarding music therapy and mental health located via Google Scholar. A total of 21 papers met our inclusion criteria.
Author and Year
Each paper had a different author and had been uploaded since 2003. This factor was limited by our search criteria (i.e., uploaded between 2000 and 2017) and the ability to publish papers online. For example, some papers might have met most of our inclusion criteria but may have been completed before online publication was common practice. The results were also limited to the English language as that is our primary language. International research teams conducting related investigations may be able to acquire and interpret papers meeting inclusion criteria that are published in other languages.
As the majority of the authors were affiliated with universities outside of the United States (n = 15), perhaps open-access graduate papers are promoted and accepted more widely throughout other countries than the United States. Additionally, it may be that more graduate music therapy students outside the United States are researching mental health and thus their papers met our inclusion criteria. However, nine papers met most inclusion criteria but were not included as they were located on websites that required additional login credentials for access. As eight of these papers were written by authors affiliated with universities within the United States, perhaps universities outside the United States are more apt to publish papers on open access websites.
Of the 21 papers that met inclusion criteria, 16 of the papers were master’s theses and five were doctoral dissertations. Reasons for these papers being unpublished are difficult to conjecture but may be resultant of authors experiencing fatigue from the research process, being rejected from a journal and not submitting the paper to another journal, or not believing their paper warrants publication due to quality, sample size, or non-significant results. From these data, however, it seems that music therapists are conducting research for the fulfillment of a graduate degree. It is interesting that no identified paper was conducted by a clinician who was not seeking a graduate degree. However, it may be that non-student or clinician researchers are publishing their research and these papers did therefore not meet our inclusion criteria and were therefore not included in our paper. These factors contribute to questions concerning incentive for conducting research as a clinician. Moreover, it should be noted that facilities tend to employ and pay music therapists to treat patients and conducting research is typically not a part of the job duties of a clinical music therapist.
Of the 21 papers meeting inclusion criteria, 19 were authored by music therapists. Two papers were conducted at Ball State University for Masters of Science degrees. Within these papers, we were unable to determine if the authors were Board-Certified Music Therapists. Also, both of these papers were prospective studies and datum was not collected. As Ball State University does not have a music therapy program, it may be that these authors were Board-Certified2 Music Therapists seeking a graduate degree in another field after completing an undergraduate degree in music therapy. However, as music-based intervention reporting guidelines (Robb, Burns, & Carpenter, 2010, 2011) were not followed by identifying the type of clinician or training they received, one might postulate that the authors were not Board-Certified music therapists. Perhaps not being Board-Certified Music Therapists prohibited data collection, as both of these were prospective studies and no datum was collected.
Clinical Population, N
Eight papers involved adolescents with mental health diagnoses. Authors conducted their research in various settings, including inpatient facilities, outpatient facilities, and transitional school settings. Due to the numerous designs, there was considerable variation in the number of research participants in the located papers meeting inclusion criteria, ranging from one to 109 research participants. As these papers were graduate theses and dissertations and likely not funded, it seems appropriate that the sample sizes were fairly small. Certainly, larger sample sizes are warranted, but these can be difficult to attain, especially when researchers are students who may have incentive to finish their project and start a paid music therapy clinical or faculty position.
Type of Study/design
Although not all papers contained data, more papers contained qualitative data (n = 10) than quantitative data (n = 3). These papers had a wide variety of study designs and are depicted in Table 1. Four studies contained both qualitative and quantitative data. However, data within these studies did not answer the same research question and were not mixed, thus not meeting criteria to be categorized as mixed-methods (Creswell & Plano Clark, 2011). Two papers did not contain either type of data and were categorized as a descriptive analysis of published music therapy papers and a developed resource for music therapists. Two papers that were designed but not conducted were categorized as prospective studies. From the considerable variance of paper types and designs, one might conclude that the researchers’ questions appropriately impacted the type of design and paper.
Due to the assortment of types of papers and research questions, findings are mixed and we felt it would be inappropriate to integrate, synthesize, or pool the data. Readers should consult Table 1 and the current authors caution against generalizing results. While outcomes of the papers varied due to the plethora of research questions, authors did report that music therapy had positive effects concerning a number of factors, including coping with stress (Salur, 2016), emotional regulation (Salur, 2016; Young, 2010), socialization and social skills (Morgan, 2007; Salur, 2016; Travis, 2003; Young, 2010), self-expression (Young, 2010), self-esteem (Karvonen, 2015), relaxation (Travis, 2003), and attendance rates (Davis, 2008). Concerning symptoms specific to mental health, authors noted that music therapy assisted in decreasing depression (Punkanen, 2011; Salur, 2016), anxiety (Punkanen, 2011), and psychiatric symptoms (Morgan, 2007).
We provided links to each of the papers that met our search criteria. We hope this access to information is useful for clinicians, scholars, service users, and information seekers.
Implications for Clinical Practice
Nine papers met inclusion criteria but required additional login credentials for access and were therefore not included in the analysis. Eight of these papers were written by authors affiliated with universities within the United States. Perhaps the lack of availability was a result of the capitalist orientation of the United States. Congruently, many journals publishing research require a subscription and people seeking information for health-related decision making often do not have access to the necessary information. In our opinion, scientific information is too frequently limited to only those who have access to the internet and expensive search engines. We believe limiting scientific results is dangerous and harmful to service users, clinicians, the public, and the music therapy profession. To us, this is a social injustice as only the privileged who have access to expensive library databases can attain relevant information. In an attempt to remedy this problematic occurrence, future researchers might consider publishing via open access journals and could include unpublished papers on open-access websites. Researchers might consider including open access fees in grant proposals to increase access to scientific information.
Reflections from the Authors
There is a plethora of reasons authors may have chosen to not publish their papers. First, and perhaps most obvious, the paper may not be of a high enough quality to warrant publication in a refereed venue. As non-significant findings are likely when conducting objectivist music therapy research with mental health service users (Silverman, 2008, 2010, 2015), publication bias may limit the number of quantitative studies published. This might also be influenced by small sample sizes in the theses and dissertations, which could impact statistical power and generalizability of results. Interpretivist research might be limited by a lack of consensus among participants, biases, or problems in the question, design, or interpretation. Moreover, authors might not even recognize that their papers are of publishable quality. Additionally, as all the identified papers were theses or dissertations, it would seem that at least some of the responsibility for encouraging publication falls upon the academic advisor. Although the submission process may seem daunting to beginning researchers, academic advisors might consider including information concerning the publication process – as well as the importance of publishing graduate work – in their coursework so students are adequately prepared to submit their manuscripts for publication consideration. However, academic advisors might encourage their advisees to publish, but the publication never comes to fruition. Rejection after an initial submission could deter researchers from integrating critical feedback and attempting another submission to a different journal.
As there is likely a dearth of additional unpublished music therapy research in existence but not necessary accessible online, perhaps posting unpublished papers online would be helpful to future researchers. However, without adequate peer review, less experienced readers and researchers risk of misinterpreting data and making false conclusions that could mislead eventual readers. Truly, these are complex and consequential issues, especially in the contemporary era of increased access to information via the internet.
One potential limitation of our study is the use of the advanced search function and the specification that the keywords needed to occur in the title of the article. With this specification, each search yielded around 100 articles or less. Without it, each search yielded anywhere between 40,000 and 70,000 articles depending on the combination of keywords. While the high number of results is an advantage of Google Scholar, the considerable difference between the two search options allows for potential unpublished articles to go undiscovered.
Another potential limitation is the use of keywords and titles of unpublished articles. For example, a citation notification was emailed to the second author of the current study containing the title of an article that, under further scrutiny, would have met all inclusion criteria. However, this article did not appear in the initial search results. As search engines such as Google Scholar continue to become increasingly sophisticated, it will be interesting to assess how access to knowledge and keywords impact the music therapy profession in the digital age.
Suggestions for Future Research
Future researchers could find and explore other open-access sources and university research databases to identify unpublished articles. Also, as nine papers met most inclusion criteria but required additional login credentials for access, future researchers could include unpublished articles that were not available on open-access websites. Future researchers might identify how many music therapists use Google Scholar to obtain scholarly output.
The unpublished music therapy and mental health literature is a valuable resource for guiding clinical practice and research. The purpose of this systematic review was to identify and analyze the current literature comprised of unpublished and non-refereed papers regarding music therapy and mental health located via Google Scholar. Twenty-one papers met inclusion criteria. Although not all papers contained data, 10 papers contained qualitative and three contained quantitative data. As the majority of authors were affiliated with universities outside the United States, perhaps there is greater interest in mental health outside the United States. It is concerning that many identified papers were located on websites that required additional login credentials. Future research concerning publication bias in music therapy is warranted.
 Since qualitative and quantitative data strands were not mixed and often did not answer the same research question, we did not categorize these papers as mixed-methods.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4-23, https://dx.doi.org/10.1007/s10488-010-0327-7.
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A., Cronin, E., & Williamson, P. R. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE, 3(8), e3081, http://dx.doi.org/10.1371/journal.pone.0003081.
Google Scholar. (2017a). Search tips: Content coverage. Retrieved from https://scholar.google.com/intl/en/scholar/help.html#coverage
Google Scholar. (2017b). Inclusion guidelines for webmasters. Retrieved from https://scholar.google.com/intl/en/scholar/inclusion.html#indexing
Google Scholar. (2017c). About. Retrieved from https://scholar.google.com/intl/en/scholar/about.html
Google Scholar. (2017d). Search tips: General questions. Retrieved from https://scholar.google.com/intl/en/scholar/help.html#general
Joober, R., Schmitz, N., Annable, L., & Boksa, P. (2012). Publication bias: What are the challenges and can they be overcome? Journal of Psychiatry & Neuroscience, 37(3), 149-152, http://doi.org/10.1503/jpn.120065.
Niemeyer, H., Musch, J., & Pietrowsky, R. (2013). Publication bias in meta-analyses of efficacy of psychotherapeutic interventions for depression. Journal of Consulting and Clinical Psychology, 81(1), 58-74, http://doi.org/10.1037/a0031152.
Robb, S. L., Burns, D. S., & Carpenter, J. S. (2010). Reporting guidelines for music-based interventions. Journal of Health Psychology, 20, 1-11, https://dx.doi.org/10.1177/1359105310374781.
Silverman, M. J. (2007). Evaluating current trends in psychiatric music therapy: A descriptive analysis. Journal of Music Therapy, 44, 388-414, https://doi.org/10.1093/jmt/44.4.388.
Silverman, M. J. (2008). Quantitative comparison of cognitive behavioral therapy and music therapy research: A methodological best-practice analysis to guide future investigation for adult psychiatric patients. Journal of Music Therapy, 45, 457-506, https://dx.doi.org/10.1093/jmt/45.4.457.
Silverman, M. J. (2010). Applying levels of evidence to the psychiatric music therapy literature base. The Arts in Psychotherapy, 37, 1-7, https://dx.doi.org/10.1016/j.aip.2009.11.005.
Turner, E. (2013). Publication bias, with a focus on psychiatry: Causes and solutions. CNS Drugs, 27(6), 457-468, http://doi.org/10.1007/s40263-013-0067-9.
Van Lent, M., Overbeke, J., & Out, H. J. (2013). Recommendations for a uniform assessment of publication bias related to funding source. BMC Medical Research Methodology, 13, 120, http://doi.org/10.1186/1471-2288-13-120.
Vine, R. (2006). Google scholar. Journal of the Medical Library Association, 94(1), 97-99, Retrieved from, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1324783/.