- Download PDF-
SEEFOR 6 (2): 237-247
Article ID: 69
Survey Research in the Forest Science Journals - Insights from Journal Editors
Mirjana Stevanov 1*, Zuzana Dobšinská 2, Silvija Krajter Ostoić 3
1 University of Novi Sad, Institute of Lowland Forestry and Environment, Antona Čehova 13, RS-21000 Novi Sad, Serbia
2 Technical University in Zvolen, Department of Economics and Management of Forestry, T. G. Masaryka 2117/24, SK-960 53 Zvolen, Slovakia
3 Croatian Forest Research Institute, Division for International Scientific Cooperation in Southeast Europe (EFISEE), Zajčeva 28, HR-10000 Zagreb, Croatia
STEVANOV M, DOBŠINSKÁ Z, KRAJTER OSTOIĆ S 2015 Survey Research in the Forest Science Journals - Insights from Journal Editors. South-east Eur for 6 (2): 237-247. DOI: http://dx.doi.org/10.15177/seefor.15-17
Received: 13 Mar 2015 / Accepted: 29 May 2015 / Publlished online: 15 Jun 2015
Background and Purpose: Survey research is one of the most commonly applied approaches in the social sciences. In the forest research it has been used for more than five decades. In spite of that or the fact that the amount of survey-based articles in the forest science journals has increased during the last decade, their share in all articles published in 20 forest science journals (9,372 articles, 2005-2014) is quite modest (3.2%). In our paper we look at the opinions and attitudes of forest science journal editors towards survey research, as their perspective might enlarge our understanding of the use of this approach in the field of forestry.
Materials and Methods: We selected 20 forest science journals - 15 from the SCI list and five non-SCI journals and contacted editors of these journals with the self-administered e-mail questionnaire. Data were collected in October 2014 and analyzed by descriptive statistics. The overall response rate was 75%. The assumptions for the study were based on the evidence addressing opinions and attitudes of journal editors from other research fields (finance) since no similar study was found in the field of forestry.
Results: The majority of editors reported the same review process for survey-based articles as for all others. In two journals, articles with the survey-based content are screened more rigorously and in two journals their publishing is generally discouraged. 40% of journal editors hold the view that no difference should be made between survey research and other types of original research, and another 40% think that survey research should in the first place play a complementary role. As the main strength of survey research editors see the possibility to obtain data unavailable from other sources. They perceive adverse selection and the difficulty to generalize results as the main weaknesses.
Conclusions: Editors of forest science journals have similar opinion on survey research as those from the field of finance. In both fields, survey-based articles typically undergo the same review process as all other original research articles. Journal editors were evenly split in their views if survey research should be considered equal or complementary to other types of original research. The two most commonly identified strengths and weaknesses differed just by the order.
Keywords: survey research, journal editors, forest science, forest policy
From all approaches the survey research is probably one of the most common ones that the social science employs in order to empirically disclose characteristics and interrelations of psychological and sociological variables. It is used to document existing conditions, the characteristics of a population, opinions or attitudes in a way that the data is collected from a sample of elements by the use of a questionnaire , , . The sample is typically drawn from a well defined study population and it is large enough to allow extensive statistical analyses, on the base of which generalizable statements are drawn about the object of the study , , .
Generalizability has become the most advantageous characteristic of the survey research, together with efficiency , , , and the ability of providing data inaccessible by other methods , , . On the other hand, surveys are one of the approaches that are used for revealing people’s opinions and attitudes, the main criticism of which refers to the respondents: they can do very different things than what they say they do – and even if they do what they say, their real reasons for doing things can be different from the ones they cite . Besides, once the survey work is underway, there is little one can do upon realizing that some crucial item was omitted from the questionnaire, or upon discovering that a question is ambiguous or is being misunderstood by respondents , . Finally, evidence shows that survey research calls for better implementation , , and a more adequate interpretation of results , , , .
In the forest research surveys have been used for more than fifty years. During the last decade 9,372 articles were published in 20 forest science journals, out of which 304 were survey-based, what makes the share of 3.2% . Compared to the articles using other methods inherent to life sciences, the modest contribution of survey-based articles may perhaps be explained by a still traditional research interest in the field of forestry. This means that many research topics and questions often do not take into account the human dimension, i.e. people’s attitudes, opinions and values. Furthermore, in the forest research social science methods are mainly used in the forest policy and forest governance, which are rather young disciplines and as such they are not frequently in the scope of traditional forestry journals. When looking at other research fields that also adopted survey from the social sciences and have been using it for many decades, then the field of management accounting counts for 30% and management information systems for 17% of survey-based articles respectively. Although this evidence dates back to 1993 and 2005, it shows the sparse contribution of survey-based articles in the forest science journals and indicates that survey research does not yet belong to the mainstream approaches in the forest research, although it has already been used for five decades . On the other hand, the number of survey-based articles published in forest science journals has been increasing over the last decade, which leads to the expectation that this trend will also continue also in the future .
In our paper we focus on the survey research from the viewpoint of science journal editors. Editors’ potential ability to provide insightful commentary about particular research and its role makes them very valuable respondents. Editors act as “channel members (and, some would argue, gatekeepers) who assemble product assortments (academic papers) from various producers (researchers) and deliver them to appraisers (reviewers) who deem them acceptable or unacceptable for public consumption (publication)” , and in that way they are obviously exposed to more papers in their respective disciplines than most other scholars . Besides, editors are assumed to be the leaders in their respective areas of research and their viewpoints are assumed to reflect a mainstream disciplinary bias in these areas .
The views of science journal editors have already been examined in several disciplines. For example, applied linguistics looked on how editors view the issue of nonnative speakers publishing in their journals, criminology and criminal justice examined editors’ opinion with respect to the use of qualitative and quantitative methods in published research and the medicine revealed editors’ views on the overall medical research publication or policies, practices and attitudes of medical journal editors . There are also discipline-independent studies including all journals of the one particular publisher, as that of the Wiley-Blackwell, looking for the editors’ views on publication ethics . With respect to the survey research it is the field of finance where editor views were examined , . However, no studies were undertaken within the forestry or natural resources research.
By assuming that in the process of publishing research results the science journal editors play a crucial role, , , , , we looked into what attitudes occur in the “population” of forest science journal editors with respect to survey research. While our analysis aimed to ascertain facts, not to test the theory, we used plain assumptions based on the already existing research results.
Our findings aim to broaden the understanding of survey research and its use in the field of forestry, complementing that way the already existing evidence on the amount, development, quality and maturity of survey research. All these different aspects together have the potential to provide critical insights into the nature, structure, and behavior of a research field in question , which is that of forest research in this particular case.
MATERIALS AND METHODS
For our analysis we first selected forest science journals and then approached their respective editors. By the journal selection we used the SCImago Journal & Country Rank (SJR), as this has already been a proven approach: the SJR web portal offers a list of peer-reviewed journals ranked according to the SJR-indicator1 from the category Q1 (best ranked) to the Q4. From the list of 150 journals (Subject area “All”, subject category “Forestry”, year 2013) we excluded: journals having too broad (e.g. Applied Geography) or too narrow focus (e.g. Forest pathology); journals dominated by natural-sciences (e.g. Trees – Structure and Function) or bio-technical journals (e.g. Tree Genetics and Genomes; Wood Science and Technology); book-type journals; journals with no current editor or defunct journals; and journals ceasing publication. From the remaining 58 journals we took a sample of five journals per quality groups Q1 to Q3, by trying to: maintain a sort of a geographical distribution (if that is possible in today’s international publishing); include both journals with male and female editors; have a full perspective on forests (e.g. journals addressing urban forests); forest journals specialized in publishing research based on the social-science approaches (e.g. forest policy) as well as those of a wider forestry scope. We focused on the categories Q1 to Q3 while assuming that apart from the SCI-journals (listed in Web of Science® Citation Index ExpandedTM, issue august 2014) the study would benefit from the inclusion of non-SCI journals as well (instead of Q4), i. e. “traditional” and open access journals outside the SCI-list . Those journals are peer-reviewed, national or international, they accept papers from all forestry disciplines (including social-science research), publish more than one issue per year (in one of the languages authors are fluent in), and have accessible online database.
Based on criteria mentioned above we took a sample of 20 journals, 15 from the SCI-list (categories Q1, Q2 and Q3) and the five “non-SCI” journals (Table 1). We analyzed all survey-based papers published in these journals, which we address elsewhere, and approached journal editors for finding out their opinion on survey research. The editors of the selected journals were identified by reviewing the journals’ website. The e-mails were sent to them in October 2014 containing a short explanation and the questionnaire. The self-administered questionnaire consisted of six closed-ended questions (see results section). In the case of two questions there was a possibility of multiple answers, and in case when none of the answers applied, the editors had a possibility to add their views and comments for each.
TABLE 1. Forest science journals included in the study, journal categories, responses from journal editors and respective response rates
* Q1, Q2 and Q3 categories from the SCImago Journal & Country Rank (see the text for explanation);
SCI - appearing in the Web of Science® Citation Index ExpandedTM, issue august 2014 ;
Non-SCI - not appearing in the mentioned list .
Since there is no previous similar research related to forestry, and since there is some in other disciplines, we applied the “earlier method to a different group of people” in order to learn whether in that case the conclusions derived from the earlier study apply equally well to our conclusions. By knowing that replication is more common in natural than in social sciences our analysis aimed to ascertain facts, not to test the theory, and so we used plain assumptions based on the already existing research results . These assumptions are as following:
- review process of survey-based articles is the same as for all other articles,
- editors have divided opinions on the role survey research should play in comparison to other types of original research,
- according to journal editors, there is no one major strength or weakness of survey research that obviously prevails over others.
Data were collected in October 2014 and inserted into the MS Excel database. We proceeded with the analysis with the help of descriptive statistics.
Due to the small sample size, the tests of statistical significance, determining whether genuine differences exist between the responses of editors of different journal categories, were not possible.
We asked editors of 20 forest science journals (11 male and 3 female editors) that are geographically distributed as shown in Table 1, to answer 6 questions from the questionnaire we sent them via e-mail. We got 15 answers, which makes an overall response rate of 75%, whereas response rates among different journal categories varied (Table 1). Most editors replied within the first three days (15 journals), one questionnaire was returned after one week and one after two weeks. Five editors did not respond, even though they were sent the reminder twice.
By answering the first question, if their particular journals have an established policy involving the publication of survey research, all journal editors selected “No”. It means that in their respective journals survey-based manuscripts go through the same procedure as the rest of the submitted articles.
While reporting that their journals do not have an established policy involving the publication of survey research, journal editors were asked to indicate the path that is followed when survey-based manuscripts are considered for publication. Table 2 shows that majority of editors (74%) follow the same review process for the survey-based articles as for all other articles, whereas in two journals, the publishing of manuscripts with the survey content is generally discouraged. One of these journals is from the Q2 category and one from the non-SCI category.
TABLE 2. The review process of survey-based papers (n=15, one answer possible)
On the question about the role survey research should play in the forestry literature 40% of editors hold the opinion of equal position, which means that no difference should be made between the survey research and other types of original research (Table 3). The same amount of them (40%) think that this role should be a complementary one, meaning that survey research should in the first place complement information and knowledge created through other types of original research. Although the small sample size does not allow generalization of results, foremost when different journal categories are concerned, it is worth to mention that the half of the editors from SCI-journals (Q1-Q3) think that survey research should play a complementary role, whereas the majority of editors form non-SCI journals hold the view of equal role of all manuscripts (Table 3). From two editors who hold the view that the role of survey research is limited when compared to other types of original research, one is from a SCI and other from a non-SCI journal. The editor who did not select any of offered answers wrote: “All research is considered on its individual merits. It is the value of the research rather than the method which is the key issue”.
TABLE 3. The role that survey research should play in the forestry literature (n=15; one answer possible)
* SCI - appearing in the Web of Science® Citation Index ExpandedTM, issue august 2014 ; Non-SCI - not appearing in the mentioned list .
The last two questions were about strengths and weaknesses of survey research. Answers were multiple so that their number does not match to the number of editors, which is 14 in this case. From Table 4 it is obvious that all journal editors consider survey research as having particular strengths. Their views about these strengths are rather dispersed, although one third of the editors think that survey results are valuable for suggesting “new avenues for future research” (Table 4) and another quarter of them thinks that surveys are valuable for gathering the “data unavailable from other sources” (Table 4). “Survey research can, at times, be the most appropriate approach to addressing a research issue”, was noted by one editor. Another editor added that survey data can, depending on the question, provide information on perceptions and preferences quite efficiently.
TABLE 4. Perceived strengths of a survey research (n=14, multiple answers possible)
The results summarized in the Table 5 transmit the message that editors do not see one major weakness of survey research as prevailing over others. By more than 20% of answers survey research is viewed as suffering from adverse selection problems and from the difficulty to get research results generalized in a required manner. Other potential weaknesses were more or less evenly distributed (Table 5). Editors also provided additional views (Table 5). One editor sees the way of interpreting the data as problematic, together with the poor description of a method: “All too often we find people interpret responses to questions as a measured fact and not as a perception. It is seldom found that perceptions as established through questionnaires are verified by some measurement and too often we find that the survey results are used to establish some quantitative result which is misleading and incorrect. (…) The other common problem is that the methods used are generally poorly described and thus not repeatable or testable. Generally, authors don’t even supply a list of the questions asked in the survey”. Indeed, another editor observed that survey methodology is often quite weakly implemented, making this approach “rather superficial” and unable to grasp the very essence of a problem.
TABLE 5. Perceived weaknesses of a survey research (n=14, multiple answers possible)
When our results are compared to that of finance, then one can recognize great similarities. Neither surveyed journal editors from finance (all 25 of them) nor from forest science report any difference when survey-based articles are submitted for publishing. In most cases, which means 81% in finance and 74% in forest science journals, the review process for survey-based articles follows the same procedure as all other articles. It is two editors from finance , the same as in forest journals (see above), that indicated a more rigorous screening of survey-based articles. In finance however none of the editors indicated that survey-based articles are discouraged of publishing.
The attitudes of surveyed finance and forest journal editors, regarding the role survey research should play in the literature, are also similar. The editors’ opinions are in the true sense evenly split when it comes to whether survey research should be considered equal or complementary to other types of research (Table 3 and Table 6 in appendix). The remaining results related to this question show that perception differs with respect to the journal category. Almost 70% of the “core” finance journal editors stated that survey research should play a complementary role (Table 6, appendix), whereas in the forest science 75% of non-SCI journal editors indicated that survey research should be considered equal to other types of original research (Table 3). It, however, needs to be underlined that the sample size is rather small and as such limits the generalization of results. Also, the small sample size has to be kept in mind when comparing the results among two research fields.
All responding journal editors agree that survey research has its strengths as well as weaknesses. According to finance journal editors the two highly ranked strengths are the production of data unavailable from other sources and the opinion that survey responses can suggest new avenues for future research (Table 7, appendix). Editors of forest science journals have very similar views that differ only by the ranking order (Table 4). The same applies to editors’ views toward the potential survey weaknesses as well. As seen by finance journal editors, those are the difficulty of generalizing the results and a non-response bias (Table 7, appendix), whereas by forest journal editors it is vice versa (Table 5).
The editors’ scepticism about the generalization of results might be related to the fact that survey typically asks for a collection of data from a fraction of the study population, so that conclusions can be “transferred” back to a target population only if sample respondents truly reflect this population , , . This should therefore clearly be demonstrated in the paper , . Also, probability sampling should be adequately accomplished or an otherwise selected method should be justified and reported in detail , , .
In the survey research, a non-response bias can occur if some respondents included in the sample do not respond so that the error is caused by the absence of respondents rather than by the collection of erroneous data, , , . This concern can among others be eliminated by looking for the ways of reducing existing non-response bias or by having that bias discussed and analysed . This is especially relevant in the forest research where not enough attention has been paid to it .
Essentially, actual strengths and weaknesses of any selected approach will depend upon research questions in place, the specifics of the research design and the nature of the knowledge being sought. That is because every approach has its advantages and disadvantages and selection depends on the set of cumulative factors . Another aspect is whether survey research is adequately implemented and whether the conclusions are properly drawn. The responsibility in this case lies with the researcher, using proper sampling methods and testing a non-response bias among others . Later in the review process the evaluators are those assessing the methodological rigor and the quality of a paper based on survey results.
In our paper we were interested into attitudes and opinions of forest science journal editors as related to survey research. We approached editors of 20 forest science journals with the self-administered e-mail questionnaire (75% response rate) and analyzed the obtained data with the help of descriptive statistics. While our analysis aimed to ascertain facts, not to test the theory, we used plain assumptions based on the already existing research:
The review process of survey-based articles is the same as for all other articles: indeed, according to editors’ responses, survey-based articles in the forest science journals typically undergo the same review process as all other original research articles. Two from 15 editors stated that this kind of articles are generally discouraged in their journals.
Journal editors have divided their opinions on the role survey research should play in comparison to other types of original research: indeed, opinions of responding editors were evenly split between equal (40%) and complementary role (40%).
According to journal editors, there is no one major strength or weakness of survey-based research that obviously prevails over others: indeed, editors’ views were rather dispersed. As the most commonly identified strength was the ability of survey research to suggest avenues for future research. Main weaknesses were seen in adverse selection problems and the difficulty to generalize results.
By comparing our results to those obtained from finance journal editors we could conclude that in both cases attitudes and opinions about survey research were very similar. We should however point out that our results are only indicative due to a small sample.
This research was realized partly by the project “Studying climate change and its influence on the environment: impacts, adaptation and mitigation” (43007) of the Serbian Ministry of Education and Science within its framework of integrated and interdisciplinary research for the period 2011-2014 and partly by the Slovak Research and Development Agency under the contract no. APVV-0057-11.
- FRANK A 2007 On the value of survey-based research in finance. Alternation 14 (1): 243-261
- VISSER PS, KROSNICK JA, LAVRAKAS PJ 2000 Survey research. In: Judd CM, Reis HT (eds) Research methods in social psychology. Cambridge University Press, New York, USA, pp 223-252
- BABBIE ER 2007 The practice of social research. Thomson Wadsworth, Belmont, CA, USA, 584 p
- GROVES RM, FOWLER FJ, COUPER MP, LEPKOWSKI JM, SINGER E, TOURANGEAU R 2009 Survey methodology. John Wiley and Sons, Hoboken, NJ, USA, 488 p
- GABLE GG 1994 Integrating case study and survey research methods: an example in information systems. Eur J Inform Syst 3 (2): 112-126. DOI: http://dx.doi.org/10.1057/ejis.1994.12
- BRYMAN A 2012 Social research methods, 4th edition. Oxford University Press, New York, NY, USA, 808 p
- FOWLER FJ 2014 Survey research methods, 5th edition. Sage publications, Thousand Oaks, CA, USA, 184 p
- VAN DER VAART T, VAN DONK DP 2008 A critical review of survey-based research in supply chain integration. Int J Prod Econ 111 (1): 42-55. DOI: http://dx.doi.org/10.1016/j.ijpe.2006.10.011
- BAKER HK, SINGELTON JC, VEIT ET 2011 Survey Research in Corporate Finance: Bridging the Gap Between Theory and Practice. Oxford University Press, New York, NY, USA, 464 p
- PINSONNEAULT A, KRAEMER KL 1993 Survey Research Methodology in Management Information Systems: An Assessment. J Manage Inform Syst 10 (2): 75-105
- GROVER V, LEE CC, DURAND D 1993 Analyzing methodological rigor of MIS survey research from 1980-1989. Inform Manage 24 (6): 305-317. DOI: http://dx.doi.org/10.1016/0378-7206(93)90028-R
- VAN DER STEDTE WA, YoOUNG MS, CHEN CX 2005 Assessing the quality of evidence in empirical management accounting research: the case of survey studies. Account Org Soc 30 (7-8): 655-684. DOI: http://dx.doi.org/10.1016/j.aos.2005.01.003
- KROTT M, SUDA M 2001 Befragung als Methode der Sozialforschung in der Forstwissenschaft (In German). Sauerländer, Frankfurt, Germany, 160 p
- HEEG B 1971 Meinungsbefragungen im Bereich der Forstpolitik- Kritische Anmerkungen. Forstarchiv (42): 217-223
- STEVANOV M, DOBSINSKA Z, SUROVY P 2015 Assessing survey-based research in forest science: turning lemons into lemonade? Forest Policy Econ. DOI: http://dx.doi.org/10.1016/j.forpol.2015.07.004
- MADDEN CS, EASLY RW, DUNN MG 1995 How Journal Editors View Replication Research. J Advertising 24 (4): 77-87. DOI: http://dx.doi.org/10.1080/00913367.1995.10673490
- FLOWERDEW J 2001 Attitudes of journal editors to nonnative speaker contributions. TESOL Quarter 35 (1): 121-150. DOI: http://dx.doi.org/10.2307/3587862
- BUCKLER K 2008 The quantitative/qualitative divide revisited: A study of published research, doctoral program curricula, and journal editor perceptions. Journal of Criminal Justice Education 19 (3): 383-403. DOI: http://dx.doi.org/10.1080/10511250802476210
- ETERNADI A, RAISZADEH F, ALAEDINNI F, AZIZI F 2004 Views of Iranian medical journal editors on medical research publication. Saudi Med J 25 (1): 29-33
- WILKES MS, KRAVITZ RL 1995 Policies, practices, and attitudes of North American medical journal editors. J Gen Intern Med 10 (8): 443-450. DOI: http://dx.doi.org/10.1007/BF02599916
- WAGER E, FIACK S, GRAF C, ROBINSON A, ROWLANDS I 2009 Science journal editors’ views on publication ethics: results of an international survey. J Med Ethics 35(6): 348-353. DOI: http://dx.doi.org/10.1136/jme.2008.028324
- BAKER HK, MUKHERJEE TK 2007 Survey research in finance: views from journal editors, International Journal of Managerial Finance 3 (1): 11-25. DOI: http://dx.doi.org/10.1108/17439130710721635
- SCIMAGO JOURNAL & COUNTRY RANK 2014. URL: http://www.scimagojr.com (31 October 2014)
- MURRAY T 2003 Blending Qualitative & Quantitative Research Methods in Theses and Dissertations. Corwin Press, A Sage Publications Company, Thousand Oaks, CA, USA, 256 p
- MALHOTRA MK, GROVER V 1998 An assessment of survey research in POM: from constructs to theory. J Oper Manag 16 (4): 407-425. DOI: http://dx.doi.org/10.1016/S0272-6963(98)00021-7
- DE VAUS DA 2002 Surveys in social research. Allen & Unwin, Crows Nest, NSW, Australia, 379 p
TABLE 6. Views of finance journal editors on the role survey research should play in the finance literature
TABLE 7. Perceived strengths and weaknesses of a survey research as seen by finance journal editors
© 2015 by the Croatian Forest Research Institute. This is an Open Access paper distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0).