Laurent Itti: "Visual attention and object recognition algorithms for applied perception"
Laurent Itti, University of Southern California
Title
Visual attention and object recognition algorithms for applied perception
Abstract
Visual attention and eye movements in primates have been widely shown to be guided by a combination of stimulus-dependent or 'bottom-up' cues, as well as task-dependent or 'top-down' cues. Understanding the mechanisms of attention can give rise to numerous applications in machine vision, interactive video games, human-computer interfaces, rendering, and assessment of human cognitive state and intentions. Both the bottom-up and top-down aspects of attention and eye movements have been modeled computationally. Yet, is is not until recent work which I will describe that bottom-up models have been strictly put to the test, predicting significantly above chance the eye movement patterns, functional neuroimaging activation patterns, or most recently neural activity in the brains of monkeys inspecting complex dynamic scenes. In recent developments, models that increasingly attempt to capture top-down aspects have been proposed. In one system which I will describe, neuromorphic algorithms of bottom-up visual attention are employed to predict, in a task-independent manner, which elements in a video scene might more strongly attract attention and gaze. These bottom-up predictions have more recently been combined with top-down predictions, which allowed the system to learn from examples (recorded eye movements and actions of humans engaged in 3D video games, including flight combat, driving, first-person, or running a hot-dog stand that serves hungry customers) how to prioritize particular locations of interest given the task. Pushing deeper into real-time, joint online analysis of video and eye movements using neuromorphic models, we have recently been able to predict future gaze locations and intentions of future actions when a player is engaged in a task. Finally, employing deep neural networks, we show how neuroscience-inspired algorithms can also achieve state-of-the art results in the domain of object recognition, especially over a new dataset collected in our lab and comprising ~22M images of small objects filmed on a turntable, with available pose information that can be used to enhance training of the object recognition model.
Biographical Sketch
Laurent Itti received his M.S. degree in Image Processing from the Ecole Nationale Superieure des Telecommunications (Paris, France) in 1994, and his Ph.D. in Computation and Neural Systems from Caltech (Pasadena, California) in 2000. He has since then been an Assistant, Associate, and now Full Professor of Computer Science, Psychology, and Neuroscience at the University of Southern California. Dr. Itti's research interests are in biologically-inspired computational vision, in particular in the domains of visual attention, scene understanding, control of eye movements, and surprise. This basic research has technological applications to, among others, video compression, target detection, and robotics. Dr. Itti has co-authored over 150 publications in peer-reviewed journals, books and conferences, three patents, and several open-source neuromorphic vision software toolkits.
Coffee Break
Friday, July 22
10:00am-10:30am
Papers - Session 1
Friday, July 22
10:30am-12:30am
Jan Ondrej, Cathy Ennis, Niamh Merriman, Carol O'Sullivan,
"FrankenFolk: Distinctiveness and Attractiveness of Voice and Motion"
Nicholas T. Swafford, José A. Iglesias-Guitian, Charalampos Koniaris, Bochang Moon, Darren Cosker, Kenny Mitchell,
"User, Metric, and Computational Evaluation of Foveated Rendering Methods"
Eakta Jain, Lisa Anthony, Aishat Aloba, Amanda Castonguay, Isabella Cuba, Alex Shaw, Julia Woodward,
"Is the motion of a child perceivably different from the motion of an adult?"
Kamran Binaee, Gabriel Diaz, Jeff Pelz, Flip Phillips,
"Binocular Eye tracking Calibration During a Virtual Ball Catching task using Head Mounted Display"
Wenyan Bi, Bei Xiao,
"Perceptual constancy of mechanical properties of cloth under variation of external forces"
Pisut Wisessing, John Dingliana, Rachel McDonnell,
"Perception of lighting and shading for animated virtual characters"
Jonathan Gandrud, Victoria Interrante,
"Predicting Destination using Head Orientation and Gaze Direction During Locomotion in VR"
Lunch
Friday, July 22
12:30pm-1:30pm
Papers - Session 2
Friday, July 22
1:30pm-2:40pm
Ylva Ferstl, Elena Kokkinara, Rachel McDonnell,
"Do I trust you, abstract creature?: A study on personality perception of abstract virtual faces"
Sai Krishna Allani, Brendan John, Javier Ruiz, Saurabh Dixit, Jackson Carter, Cindy Grimm, Ravi Balasubramanian,
"Evaluating Human Gaze Patterns During Grasping Tasks: Robot versus Human Hand"
Bochao Li, Anthony Nordman, James Walker, Scott Kuhl,
"The Effects of Artificially Reduced Field of View and Peripheral Frame Stimulation on Distance Judgments in HMDs"
Yuanyuan Jiang, Elizabeth O'Neal, Pooya Rahimian, Junghum Paul Yon, Jodie M. Plumert, Joseph K. Kearney,
"Action Coordination with Agents: Crossing Roads with a Computer-Generated Character in a Virtual Environment"
Poster Fast Forward
Friday, July 22
2:40pm-3:00pm
Coffee Break and Poster Session
Friday, July 22
3:00pm-4:45pm
Katherine Breeden, Pat Hanrahan,
"Analyzing Gaze Synchrony in Cinema: A Pilot Study"
Daniel Simon, Srinivas Sridharan, Shagan Sah, Raymond Ptucha, Chris Kanan, Reynold Bailey,
"Automatic Scanpath Generation with Deep Recurrent Neural Networks"
Michihiro Mikamo, Kotaro Mori, Bisser Raytchev, Toru Tamaki, Kazufumi Kaneda,
"Binocular Tone Reproduction Display for an HDR Panorama Image"
Nargess Hassani, Michael J. Murdoch,
"Color Appearance Modeling in Augmented Reality"
Mehul Bhatt, Jakob Suchan, Vasiliki Kondyli, Carl Schultz,
"EMBODIED VISUO-LOCOMOTIVE EXPERIENCE ANALYSISmbodied Visuo-Locomotive Experience Analysis: Immersive Reality Based Summarisation of Experiments in Environment-Behaviour Studies"
Purnendu Kaul, Vijay Rajanna, Tracy Hammond,
"Exploring Users' Perceived Activities in a Sketch-based Intelligent Tutoring System Through Eye Movement Data"
Anahita Sanandaji, Cindy Grimm, Ruth West,
"How Experts' Mental Model Affects 3D Image Segmentation"
Jasper LaFortune, Kristen L. Macuga,
"Learning Movements from a Virtual Instructor"
Ishwarya Thirunarayanan, Sanjeev Koppal, John Shea, Eakta Jain,
"Leveraging Gaze Data for Segmentation and Effects on Comics"
Yugo Sato, Takuya Kato, Naoki Nozawa, Shigeo Morishima,
"Perception of Drowsiness based on Correlation with Facial Image Features"
Srinivas Sridharan, Reynold Bailey,
"Saliency and Optical Flow for Gaze Guidance in Videos"
Pallavi Raiturkar, Andrew Lee, Eakta Jain,
"Scan Path and Movie Trailers for Implicit Annotation of Videos"
Jakob Suchan, Mehul Bhatt, Stella Yu,
"THE PERCEPTION OF SYMMETRY IN THE MOVING IMAGE: Multi-Level Computational Analysis of Cinematographic Scene Structure and its Visual Reception"
Veronica U. Weser, Joel Hesch, Johnny Lee, Dennis R. Proffitt,
"User Sensitivity to Speed- and Height-Mismatch in VR"
Social Event/Drinks Reception
Friday, July 22
from 7:00pm on
Session name
Date
Time
Registration
Saturday, July 23
8:00am-8:15am
Papers - Session 3
Saturday, July 23
8:15am-10:00am
Manfred Lau, Kapil Dev, Julie Dorsey, Holly Rushmeier,
"Learning a Human-Perceived Softness Measure of Virtual 3D Objects"
Lorraine Lin, Sophie Jörg,
"Need a hand? How Appearance Affects the Virtual Hand Illusion"
Atul Rungta, Sarah Rust, Nicolas Morales, Roberta Klatzky, Ming Lin, Dinesh Manocha,
"Psychoacoustic Characterization of Propagation Effects in Virtual Environments"
Colin Ware, Daniel Bolan, Rikki Miller, David H. Rogers, James P. Ahrens,
"Animated versus Static Views of Steady Flow Patterns"
Elham Ebrahimi, Sabarish Babu, Christopher Pagano, Sophie Jörg,
"An Empirical Evaluation of Visuo-haptic Feedback on Physical Reaching Behaviors during 3D Interaction in Real and Immersive Virtual Environments"
Florian Soyka, Markus Leyrer, Joe Smallwood, Chris Ferguson, Bernhard E. Riecke, Betty J. Mohler,
"Enhancing Stress Management Techniques Using Virtual Reality"
Coffee Break
Saturday, July 23
10:00am-10:30am
Papers - Session 4
Saturday, July 23
10:30am-12:00pm
Pallavi Raiturkar, Andrea Kleinsmith, Andreas Keil, Arunava Banerjee, Eakta Jain,
"Decoupling Light Reflex from Pupillary Dilation to Measure Emotional Arousal in Videos"
Marc Spicker, Diana Arellano, Ulrich Schaller, Reinhold Rauh, Volker Helzle, Oliver Deussen,
"Emotion Recognition in Autism Spectrum Disorder: Does Stylization Help?"
Justin K. Bennett, Srinivas Sridharan, Brendan John, Reynold Bailey,
"Looking at Faces: Autonomous Perspective Invariant Facial Gaze Analytics"
Timofey Grechkin, Jerald Thomas, Mahdi Azmandian, Mark Bolas, Evan Suma,
"Revisiting Detection Thresholds for Redirected Walking: Combining Translation and Curvature Gains"
Takahiro Kawabe, Shin'ya Nishita,
"Seeing jelly: judging elasticity of a transparent object"
Lunch
Saturday, July 23
12:00pm-1:50pm
Keynote speaker
Saturday, July 23
1:50pm-3:00pm
Allison Okamura: "Haptics for Augmented Reality and Teleoperated Robots"
Allison Okamura, Stanford University
Title
Haptics for Augmented Reality and Teleoperated Robots
Biographical Sketch
Allison Okamura received her BS degree from the University of California at Berkeley in 1994, and her MS and PhD degrees from Stanford University in 1996 and 2000, respectively, all in mechanical engineering. She is currently a Professor in the mechanical engineering department at Stanford University. She has been an associate editor of the IEEE Transactions on Haptics, an editor of the IEEE International Conference on Robotics and Automation Conference Editorial Board, and co-chair of the IEEE Haptics Symposium. Her awards include the 2009 IEEE Technical Committee on Haptics Early Career Award, the 2005 IEEE Robotics and Automation Society Early Academic Career Award, and the 2004 NSF CAREER Award. She is an IEEE Fellow. Her research interests are in the areas of haptics, teleoperation, virtual environments and simulators, medical robotics, neuromechanics and rehabilitation, prosthetics, and engineering education. Outside academia, she enjoys spending time with her husband and two children, running, and playing ice hockey.