Title: Statistical Analysis for Information Access Evaluation
Speaker: Professor Tetsuya Sakai (Waseda University)
Time: September 17th Monday 19:00—20:00
Venue: FIT 1-515
Abstract:
This talk will be based on parts of my SIGIR 2018 tutorial: Conducting Laboratory Experiments Properly with Statistical Tools: An Easy Hands-on Tutorial. I will discuss: why it is often inappropriate to repeat t-tests many times and what are the right ways, why significance tests have received lots of criticisms, effect sizes, statistical power, topic set size design for building new test collections, and power analysis of existing research papers. I will try to calibrate the contents depending on the audience.
Bio:
Tetsuya Sakai is a professor and the head of department at the Department of Computer Science and Engineering, Waseda University, Japan. He is also a visiting professor at the National Institute of Informatics. He joined Toshiba in 1993 and obtained a Ph.D from Waseda in 2000. From 2000 to 2001, he was supervised by the late Karen Sparck Jones at the Computer Laboratory, University of Cambridge, as a visiting researcher. In 2007, he joined NewsWatch, Inc. as the director of the Natural Language Processing Lab. In 2009, he joined Microsoft Research Asia. He joined the Waseda faculty in 2013. He was Associate Dean (IT Strategies Division) from 2015 to 2017. He is an editor-in-chief of the Information Retrieval Journal (Springer) and an associate editor of ACM TOIS. He is also a vice-chair of ACM SIGIR Tokyo.