Saturday, April 9, 2016

Hyper Paper Productivity (or the most prolific authors of IJCAI submissions..)

The other day we were compiling statistics on IJCAI submissions, and I was struck by some authors who seemed to be a co-author on a rather unbelievably  large number of submissions.

At first I thought that the inflated numbers might be because the analysis is mistakenly merging multiple different people who happen to have the same name. 

So we redid the statistics, this time taking the author contact email addresses to resolve the identity. Surprisingly, the statistics on the hyper-prolific authors remained unchanged.

Being of the "basically lazy" kind, this level of productivity was beyond my meager ken. So I asked some of my less lazy colleagues to take a guess on how many submissions the most prolific co-authors had.

Unfortunately, my colleagues seemed to be just as behind times as I am. None of them could hazard a guess that is even within 50% of the true number. This even after I gave them a couple of chances to improve their guesses.

Clearly I need to associate with more productive colleagues! Be that as it may,  let's come back to these hyper-productive authors. 

The number of papers co-authored by the most prolific co-author in the IJCAI  turns out to be.. DRUMROLL  please... 

32 


Yes, you heard me right. *thirty two*.   And no, this is not a single outlier. Next in line were authors with 

23

21 

co-authored submissions respectively.   Without giving anything away, let me say that all these prolific authors are writing papers in  applied ML--an area that also has the maximum number of submissions to IJCAI). 

When I shared these numbers with my (admittedly lazy) colleagues, they countered that the number of papers accepted from these would surely be close to zero--as after all who other than a Map-Reduce program can churn out so many papers?

Well, little do these colleagues know! These papers are not the bottom of the barrel.  About the only damning thing that can be said is that an overwhelming majority were basically borderline. 

Which made me wonder--what if these prolific authors decided to focus their energies on fewer papers with potentially higher quality. (Reminded me of a colleague's backhanded compliment of one of his PhD students: "Having him is like having 10 mediocre students!")


Some colleagues pointed out that many IEEE conferences put an upper-bound on the number of submissions from any one author, and wondered if IJCAI should consider it too. Something to think about from a mechanism design point of view.

But clearly, there seem to be other pressures pushing people towards hyper-paper-productivity even at the expense of spreading themselves too thin. Here is hoping that some resistance will be developed on this front.

Even in these days of  inflated resume expectations, it is perhaps more important to be known for a  contribution rather than a count; for a significant idea,  than for breaking a guinness book record on the number of papers, or aiming for  a Kim Kardashian-esque "famous for being famous" distinction. 

As always, your comments are welcome!

Rao

ps: There are some obvious implications of hyper paper productivity to the "spinning the wheel of fortune" phenomenon I talked about earlier; but I will leave them to your imagination

Disclaimer: Author anonymity was never compromised during this analysis. There is no  implied correlation--positive or negative--between the number of submissions and the number of acceptances. 


Addendum; A colleague pointed out that ICSE, one of the main conferences of Software Engineering, instituted a maximum 3 papers per author policy starting 2017. There seems to be very interesting discussion about the policy in that community

2 comments:

  1. This is the 'statistical game' being played. And you pointed that -- an overwhelming majority were basically borderline. If I write a paper on application of ML to polish my shoes -- the chances of getting it better reviews are very high compared to my writing a paper on developing a really novel reasoning algorithm for finding what food makes me get acidity within 3 hours. A reviewer does not need to understand what is novel and hence is starts with a base which is higher in his/her understanding.
    Our system is creating scientists who believe in writing 32 papers with 0 or negative value rather than 1 paper that has a fair positive novel contribution.
    Just my $0.02.

    ReplyDelete
  2. The most common case where I've seen large numbers of submissions from one person is when it's the head of a large lab who has decided to focus his/her lab on a small number of publication venues. The lab has 20 people or so, and they each write one paper for IJCAI and put the lab head's name on the paper as a courtesy, and bam, 20 submissions from one author. (The lab head may or may not have even read these papers, much less written them, but that's another story.)

    ReplyDelete