7 Sternberg papers: 351 references, 161 self-citations

Robert Sternberg, editor-in-chief of Perspectives on Psychological Science (PoPS), published 7 papers in PoPS in the last 2 years. The papers contain 351 references; 161 of these references (46%) are self-citations. This pattern doesn’t seem limited to his papers published in Perspectives: 51 of the 66 references (77%) in a recent paper on intelligence1 are self-citations as well.

Let’s take a step back. PoPS is a very prestigious journal of the Association for Psychological Science, with a current impact factor of about 10. The current editor-in-chief is Robert Sternberg, an eminent and widely known psychologist and psychometrician.

A few months ago, in December 2017, Sanjay Srivastava pointed out that Sternberg had cited himself 37 times in a paper he had published in Perspectives.

This here is the paper in question, entitled “Some Lessons From a Symposium on Cultural Psychological Science”. I did a recount, and find 40 self-citations, out of 97 total citations (41%).

Today, a new issue of Perspectives was published, and I noticed that Sternberg has a new piece with the title “The Scientific Work We Love: A Duplex Theory of Scientific Impact and Its Application to the Top-Cited Articles in the First 30 Years of APS Journals”; 23 of the 59 citations are self-citations (39%). In the piece, Sternberg actually gives advice on how to produce “work that will have the highest impact and receive the most citations”. I took the liberty to highlight some key components to this success.

Because this is the second paper of Sternberg I encountered within a short period of time with a stunning self-citation ratio, I looked at his other papers published in Perspectives from 2016-2018, and found 7 papers.

Here is some basic R-code, I’m too tired to make figures now, but the results should speak for themselves.

# 7 papers
self  <- c(23, 23, 10, 40, 17, 11, 37)  #number of self citations per paper
total <- c(38, 59, 16, 97, 39, 17, 85)  #number of total citations per paper
net <- self/total
mean(net)              #0.51
median(net)            #0.44
sum(self)              #161
sum(total)             #351 
sum(self)/sum(total)   #0.46

Of the 351 citations across the 7 Perspectives pieces, 161 (46%) are self-citations.

This is just a very small selection of papers, and obviously I’m not drawing any inference beyond this small selection of papers. But the selection is meaningful in that it contains all his work published in the last 2 years in the journal he runs2. Across these articles, a consistent pattern emerges regarding the relationship between self-citations to total references: 23/38 (61%), 23/59 (39%), 10/16 (63%), 17/39 (41%), 11/17 (65%), and 37/85 (44%)3. The self-citation rate was never below 39%.

Reference lists like this one are not uncommon:

You can find the above syntax and a list of the 7 papers including URLs here4. I’m not linking to the papers in this blog post — after all, they will have plenty of citations soon enough.

PS: Roger Giner-Sorolla tweeted the Committee on Publication Ethics (COPE) guidelines on the topic that seem relevant here:

Update May 7th 2018: A lot has happened in the last month. A few days after my blog post, a large group of psychologists, spearheaded by Chris Crandall, wrote an open letter to APS pointing out issues with Sternberg’s editorial practices. A few days later, Nick Brown blogged about severe cases of self-plagiarism in Sternberg’s papers. On April 30th, Sternberg retired from his position as Editor in Chief at PoPS, after putting out a statement.

Update June 6th 2018: The first Sternberg paper has been retracted due to self-plagiarism, or as the journal put it: “for reasons of redundant publication”. The journal further writes: “Although the content in the aforementioned article is scientifically valid, the article has substantial unreferenced overlap with the following works by the same author”.

  1. http://www.mdpi.com/2079-3200/6/1/4
  2. To obtain the papers, I searched Sternberg’s google scholar profile by time & publication outlet; if I missed any articles, please let me know.
  3. I hope there are no mistakes; I counted every value twice, but I’ve had 2 glasses of orange juice AND a cup of black tea, so someone might want to double-check.
  4. zip file, WordPress doesn’t like .R files.

18 thoughts on “7 Sternberg papers: 351 references, 161 self-citations

  1. Pingback: O scarrafone e altre letture da week-end – L'archivio di Oca Sapiens

  2. Pingback: Looking back at 2018 - Eiko Fried

  3. Pingback: Brief psychology news 06/2018 - Eiko Fried

  4. Pingback: Reflections on APS 18: open science, transparency, and inclusion – Eiko Fried

  5. Pingback: Sternberg-er And Fries – Neuroskeptic – KESIMPULAN

  6. Pingback: Sternberg-er And Fries – Neuroskeptic – The Star News

  7. Pingback: Hei_PI: Psychologisches Institut Heidelberg : Robert Sternberg in der Kritik

  8. Pingback: O scaraffone e altre letture da week-end - Ocasapiens - Blog - Repubblica.it

  9. Robert

    And the worst thing is, most of his papers are just bla bla. No substance, no science, just meaningless opinion-sharing.

  10. Pingback: Weekend reads: “Weaponized transparency;” fighting academic spam with humor; NIH cracks down – Retraction Watch

  11. Anonymous

    Hmm, i wonder what the real issue/cause of possible outrage is here. Is it 1) self-citations, or 2) is it that citations can be/are used as some sort of measure for evaluating researchers?

    I reason that 1) is not necessarily a problem. In fact, i reason that the possible “outrage” about this may in large part reflect current (possibly severely flawed) research- and publication practices where it seems to me that researchers jump fom topic to topic, from paper to paper, not interpret non-published (non-significant?) papers, hereby alltogether not really systemically investigating anything really.

    What if i were to have a “real research program” and systematically investigated a theory or phenonemon? (possibly see here for my best attempt at describing what i view as a “real research program”: http://andrewgelman.com/2017/12/17/stranger-than-fiction/#comment-628652). Would it not make sense to cite this work in every paper following the prior investigations?

    Now if self-citations are a possible problem because of 2), i reason that the possible “outrage” should be about using citations numbers as evaluation metrics, not necesssarily self-citations.

    Just my 2 cents.

    1. Eiko Post author

      I largely limited the blog to presenting facts over drawing conclusions because things are indeed complicated. Older folks have more papers to self-cite than younger folks, while the pool of total citable papers is the same. I don’t think this should have such a huge impact, given the absurd amount of total papers, but still. And then there’s the fact that Sternberg has worked on many of the topics his papers I cover here tackle; this complicates the situation. But I think that a self-citation rate of nearly 80% in a paper with nearly 70 references is an issue that is worth calling out, no matter the fact that things are complicated.

      To your two points. First, I don’t see any “outrage” here.
      Second, I agree that the system is flawed, and tackled this in my last blog post on the topic of scientific publishing. I also agree that this blog is not in any shape or form a systematic investigation, and I also say this clearly in the blog; inferences only follow for the sample studied here, not for the population of all Sternberg papers, because these 7 papers are definitely not a representative or random draw. But we can draw conclusions about the population “all 2016-2018 Sternberg papers published in PPS” because the sample equals the full populations.
      Third, there is much to be desired in terms of research programs, and I very much like the work of Brian Haig on the topic. In fact, I’ve been very lucky to be in a lab where people have regularly chosen the strategy that our head of department Han van der Maas followed in his 2006 Mutualism paper where you a) describe something in terms of multiple phenomena — replicable observations that hold about a psychological construct — and then b) develop a theory that can account for these. Jonas Dalege has followed this approach in his work on attitude, and Donald Robinaugh and Maarten Marsman have papers forthcoming that use the same strategy to explain other psychological constructs.
      Finally, there is nothing categorically bad about self-citations, of course. If a researchers has spearheaded a new topic, written a number of papers on it, and is then invited by a journal to write a review paper, of course there will be many self-citations. So I’m not sure just excluding self-citations is generally viable, but it might make sense to complement citations with citations minus self-citations. In general, the question is how meaningful citations are today, but that goes a bit too far for now. And I haven’t even had breakfast…

    1. Eiko Post author

      Good idea to do this more systematically. Do you have more recent examples? In the blog post you link to, the URLs do not work, and if I try to reproduce the syntax -“name”, that does not work.

      1. Anonymous

        It’s also important to realize that Google hit counts are not precise; they are just estimates of search results.

  12. Anonymous

    Sternberg is well known for this. He is a bit of a laughing stock in some areas of psychology as a result and well respected in the areas that do not know any better. But one does not become the most cited psychologist of all time by chance alone, after all. It takes a concerted effort.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.