There is a form of pecking-order for research journals in all disciplines and that includes psychology. To have an article published in Nature or Science signals something of the importance of one’s work. Researchers are attracted to publishing in the most prestigious journals for professional advancement. Virtually every journal has a surfeit of material submitted to it so very good material may some-times be rejected. The rejection rate of articles submitted for publication in journals is relatively high. In 2008, rejection rates varied from 35 per cent for Experimental and Clinical Pharmacology to 89 per cent for Teaching of Psychology with an average of 69 per cent across the non-divisional journals published by the leading American Psychological Association (American Psychological Association, 2009). Not unexpectedly, agreement between referees or reviewers about the quality of an article may be considerably less than perfect (Cicchetti, 1991). Quality, after all, is a matter of judgement. Authors may well find that an article rejected by one journal will be accepted by the next journal they approach.
The impact factor is a measure of the frequency with which the average article in a journal has been cited within a particular period. This may be regarded as a useful indicator of the quality of the journal. More prestigious journals should be more frequently cited than less pre-stigious ones. The Institute for Scientific Information which produces the Web of Science also publishes Journal Citation Reports annually in the summer following the year they cover. The impact factor of a particular journal may be found using these reports either online, on CD-ROM or on microfiche. The period looked at by the Journal Citation Reports is the two years prior to the year being considered (Institute for Scientific Information, 1994). For example, if the year being considered is 2011, the two years prior to that are 2009 to 2010. The impact factor of a journal in 2011 is the ratio of the number of times in 2011 that articles published in that journal in 2009 and 2010 were cited in that and other journals to the number of articles published in that journal in 2009 and 2010:
journal’s impact factor 2011 = citations in 2011 of articles published
in journal in 2009–2010
number of articles published in journal in 2009–2010
So, for example, if the total number of articles published in 2009 and 2010 was 200 and the number of citations of those articles in 2011 was 200, the impact factor is 1.00.
The impact factor excludes what are called self-citations where authors refer to their previous articles.
Taking into account the number of articles published in a particular period controls for the size of the journal.
If a journal publishes more articles than another journal, then that journal is more likely to be cited simply for that reason if all else is equal. This correction may not be neces-sary as it was found by Tomer (1986) that the corrected and the uncorrected impact factor correlates almost perfectly (0.97).
The impact factors for a selection of psychology journals for the years 2004 to 2008 are presented in Table 7.3.
The impact factor varies across years for a journal. For example, for the British Journal of Social Psychology it decreased from 1.99 in 2007 to 1.71 in 2008. It also differs between journals. For these journals in 2008, the highest impact factor is 5.04 for the Journal of Personality and Social Psychology and the lowest is 0.59 for The Journal of Psychology. An impact factor of about 1.00 means that the average article published in that journal was cited about once in the previous two years taking into account the number of articles published in that journal in those two years. The Web of Science includes only those journals that are considered to be the most important (Thomson Reuters, n.d.).
However, even the Institute for Scientific Information which introduced the impact factor measure says that the usefulness of a journal should not be judged only on its impact factor but also on the views of informed colleagues or peers (Institute for Scientific Information, 1994). The impact factor is likely to be affected by a number of variables such as the average number of references cited in a journal or the number of review articles that are published by a journal. The relationship between the citation count of a journal and the subjective judgement of its standing by psychologists has not been found to be strong.
For example, Buss and McDermot (1976) reported a rank-order correlation of 0.45 between the frequency of citations for 64 psychology journals in the period 1973–1975 and a five-point rating made of those journals
Î
by the chairs or heads of 48 psychology departments in the United States in an earlier study by Mace and Warner (1973). This relationship was stronger at 0.56 when it was restricted to the ten most highly cited journals. In other words, agreement was higher when the less highly cited journals were excluded. Rushton and Roediger (1978) found a Kendall’s tau correlation of 0.45 between the ratings of these journals by these departmental heads and their impact factor. Chairs of departments are an influ-ential group of people in that they are often responsible for selecting, giving tenure and promoting academic staff.
However, it is possible that nowadays chairs are more aware of the impact factor and so the relationship between the impact factor and the rating of the journal may be higher.
There appears not to be a strong relationship between the number of times a published paper is cited by other authors and either the quality or the impact of the paper as rated by about 380 current or former editors, associate editors and consulting editors of nine major psychology journals who had not published papers in those journals (Gottfredson, 1978). Because the distribution of the number
of citations was highly skewed with most articles not being cited, the logarithm of the citation number was taken. The correlation between this transformed number was 0.22 for the quality scale and 0.36 for the impact scale. The number of times a paper is cited is given by Web of Science just below the journal title as shown in Figure 7.5. If you select the number after Times Cited (provided that it is not zero), you will see details of the papers that have cited this reference.
The lack of agreement about the quality of published papers was dramatically illustrated in a study by Peters and Ceci (1982) in which 12 papers which had been published in highly regarded American psychology journals were resubmitted to them 18 to 32 months later using fictitious names and institutions. Of the 38 editors and reviewers who dealt with these papers, only three realised that they were resubmissions. Of the nine remaining papers, eight of these previously published papers were rejected largely on the grounds of having serious methodological flaws.
This finding emphasises the importance of the reader being able to evaluate the worth of a paper by themselves and not relying entirely on the judgements of others.
Table 7.3 Impact factors for some psychology journals for 2008 to 2004
Journal 2008 2007 2006 2005 2004
British Journal of Social Psychology 1.71 1.99 1.42 2.11 1.59
Journal of Personality and Social Psychology 5.04 4.51 4.22 4.21 3.63
The Journal of Psychology 0.59 0.54 0.59 0.53 0.42
Journal of Social and Personal Relationships 1.10 0.87 0.99 0.72 0.82
The Journal of Social Psychology 0.73 0.86 0.66 0.60 0.60
Personality and Social Psychology Bulletin 2.46 2.58 2.42 2.09 1.90
Social Psychology Quarterly 1.14 2.07 1.30 1.06 1.40
7.5 Personal bibliographic database software
There is much bibliographic database software which enables you to quickly store the details of references of interest to you from electronic databases such as Web of Science and PsycINFO. These include EndNote, RefMan, ProCite and RefWorks. If you look at the Web of Science screenshots in Figure 7.5 or 7.6 you will see that there is an option to Save to EndNote, RefMan and ProCite. In the PsycINFO screenshot in Figure 7.14 there is a RefWorks icon which if you select will permit you to save references to RefWorks. For example, we could save the details of the reference in Figure 7.14 by Singh and colleagues in RefWorks as shown in Figure 7.16. You can also use this kind of software to write out the references that you cite in your work in a particular style,
FIGURE 7.16 Full details of a reference in RefWorks (Source: RefWorks is a registered trademark of Elsevier B.V.)
FIGURE 7.17 6th edition APA publication style of a RefWorks reference (Source: RefWorks is a registered trademark of Elsevier B.V.)
such as that recommended by the American Psychological Association. For example, we could format the Singh reference in terms of 6th edition of the APA Publication manual as presented in Figure 7.17. This does not mean that we do not have to familiarise ourselves with the details of this particular style, as we still have to check whether the software and our use of it has presented the references in the appropriate style.
7.6 Conclusion
The development of any discipline is the collective effort of numerous researchers acting to a degree independently. It is necessary for researchers to communicate their findings and ideas in publications such as journal articles. Similarly, researchers need to be able
to access the work of other researchers in order to make an effective contribution to developing the field of research in question. Effectively searching the literature involves a number of skills. In this chapter we have concentrated on efficient searches of avail-able databases. Of course, professional researchers have a wider variety of information sources available. For example, they go to conferences and hear of new work there, they get sent copies of reports by colleagues doing research elsewhere, and they have an extensive network of contacts through which news of research elsewhere gets com-municated. Students have fewer options at first.
Searching the literature on a topic takes time and unless this is taken into account, students may have problems fitting it into their schedule. No one can expect that all materials will be available in their university or college library. There are ways of obtaining material which are increasingly dependent on the World Wide Web. If you are unfamiliar with the research on a particular topic, it may be helpful to find a recently published book which includes an introduction to that topic to give you some idea of what has been done and found. Electronic databases such as Web of Science and PsycINFO are a very convenient way to find out what has been published on a particular topic. These electronic databases provide short abstracts or summaries of publications which should give you a clearer idea of whether the publication is relevant to your needs.
You may not always realise the importance or relevance of a paper when you first come across it. Consequently, it may be better to make a note of a paper even if it does not appear immediately relevant to your needs. This is easily done with the copy and paste functions of the computer software you are using. You need to learn to judge the value of a research paper in terms of what has been done rather than simply accepting it as being important because it has been published or has been published in what is reputedly a good journal.
z The key to carrying out a successful literature research in any discipline lies in using the various available sources of information. Of these, modern research is most heavily dependent on the use of electronic databases such as Web of Science and PsycINFO. These are generally available in universities and elsewhere. Students will find them useful but often the materials available via their library catalogue will take priority because of their ease of availability.
z The major databases essentially consist of abstracts or summaries of research publications including both journal articles and books. An abstract gives a fairly detailed summary of the research article and is an intermediate step to help the reader decide whether or not the complete article or book is required. In addition, these databases frequently contain enough information to enable the author to be contacted – often this goes as far as including an e-mail address.
z Databases are not identical and one may supplement another. Furthermore, there may well be other sources of information that some researchers can profitably refer to. For example, the fields of biology, medicine, sociology and economics might provide essential information for researchers in some fields of psychology. Knowledge of these builds up with experience.
z Abstracts and other information may be copied and pasted on your computer. In this way it is possible to build up a record of the materials you feel will be useful to you.
z There are numerous ways of obtaining published research. The Internet and e-mail are increasingly rich sources. It can be surprisingly easy to get in touch with academics all over the world. Many are quite happy to send copies of their work either in the mail or electronically.
Key points
ACTIVITIES
1. If you do not already know this, find out what electronic databases are available in your library and how to access them.
The best way of checking how good a system is and how to use it is to try it out on a topic that you are familiar with. It should produce information you are already aware of. If it does not do this, then you can try to find out how to locate this information in this system. Try out a few systems to see which suits your purposes best.
2. Many university libraries provide training in the use of their resources and systems. These are an excellent way of quickly learning about the local situation. Enquire at your library and sign up for the most promising. Afterwards try to turn your effort into better grades by conducting a more thorough search as preparation for your essays, practical reports and projects. Using information sources effectively is a valuable skill, and should be recognised and rewarded.