Thursday, February 18, 2016

So what papers are IJCAI-16 PC members most interested in reading (based on the bid popularity)?

One of the unsaid things about a large conference like IJCAI is that the quality of reviews a paper gets is critically correlated with the number of bids that the paper gets.

When a paper doesn't get enough bids, it needs to be matched manually to PC members--something that is very much error-prone when we are talking about 2000 papers and 1700+ reviewers (and the ever condensed reviewing time).

Having just completed making some 10,000+ reviewer assignments using 53,000 reviewer bids, I am struck not just by the well known long tail phenomena in the number of bids papers get, but also how many papers get almost "shutout" (get zero bids).

So we thought it would be fund to do an analysis on what papers are getting a lot of bids (based on a word cloud analysis of the paper titles).

52892 bids were made by 1723 program committee members over 2000 papers in the main track(or an average of 25 bids/paper and 30 bids/PC member). Here is how they were distributed




So, we naturally wanted to find out which papers are getting more vs. fewer bids. Numbering the bins 0 (for <=3 combined bids), 1 (for 4-10 combined bids) etc, we made word clouds for papers in each of the bins (based on the titles). Thus the cloud (helpfully shaped) 8 has the words occurring in the papers getting  >70 bids ;-)

Here then are the bins from the "least bids (0)" to "most bids (8)"




So there you go... I have my own interpretation of this data, but I would rather hear your interpretations ;-)

Rao
(with all the help from Lydia Manikonda-- IJCAI-16 Data Scientist)


Tuesday, February 9, 2016

What's hot in AI? (from IJCAI-16 Main Track Submission Ttitles)


Here is the word cloud of IJCAI-16 paper titles.




The above is with tf/idf so it throws out non-discriminative words (like "is", "of" and "deep" :-)


Here is another tf/idf normalized view...


                                                                                            (Clouds Credit: Lydia Manikonda)

So apparently the IJCAI does remain the conference for the Whole AI Enterprise ;-)


Finally, here is the normal frequency word cloud...




Rao
Feb 9th

Sunday, February 7, 2016

My earliest IJCAI submissions... ;-)

As we are caught-up  in the  IJCAI-16 paper assignment phase, I recalled, with some fondness, my earliest IJCAI papers.

My very first submission was to IJCAI 1985. That paper was unfortunately rejected; I managed to find those reviews ;-)  It did go on to form the basis for my MS thesis and is currently my most cited paper...

My next submission was to IJCAI 1989 and it fared much better--getting accepted to the conference. I remember going to the huge conference at Detroit. I also remember that Tom Dean, who was tasked with giving a talk on the last day of the conference about what is new in planning and reasoning at IJCAI-89 asked me for a couple of my plastic transparencies, and covered them in his talk! Heady feeling.. ;-)  That paper went onto form the basis for my PhD thesis. I managed to find its reviews too .

I don't obviously have any submissions this time--but wish all the rest of you with submissions great reviews!

Rao
2/7/16