Differences

This shows you the differences between two versions of the page.

research:visualsearch [2010/06/09 16:10]
kimo created
research:visualsearch [2019/02/08 15:40] (current)
rosenholtz
Line 1: Line 1:
-====== Visual Search ======+====== Ruth Rosenholtz ====== 
 +=== Visual Search ===
---- ----
- 
//In scientific thought we adopt the simplest theory which will explain all the facts under consideration and enable us to predict new facts of the same kind. The catch in this criterion lies in the word ‘simplest’. It is really an aesthetic canon such as we find implicit in our criticisms of poetry or painting.// //In scientific thought we adopt the simplest theory which will explain all the facts under consideration and enable us to predict new facts of the same kind. The catch in this criterion lies in the word ‘simplest’. It is really an aesthetic canon such as we find implicit in our criticisms of poetry or painting.//
->>;> John Burdon Sanderson Haldane +<WRAP center 60%> 
->>> Possible Worlds (1927): Science and +John Burdon Sanderson Haldane Possible Worlds (1927): Science and Theology as Art-Forms\\ 
->>> Theology as Art-Forms+</WRAP>\\ 
 + 
 +===== Theories of Visual Search: The Old and the New ===== 
 + 
 +My lab's work on visual search predominantly addresses visual aspects of search: can we predict, based on the visual characteristics of the display, how easy search will be? We aim to see how well one can do with a simple model of critical visual processing stages, coupled with ideal or semi-ideal (ideal + limits) decision stages.  
 + 
 +Below are descriptions of three "generations" of research into visual search in my lab. The first section describes our current view of search, based on recent research in the lab on peripheral vision, particularly crowding. The second and third sections describe older work in the lab: the Statistical Saliency Model, which postulated that the visual system had an interest in detecting and quickly examining outliers; and my cautionary tale that one should not make too much of search asymmetries, because it is tricky to ensure that one's experimental design did not asymmetrically give an advantage to some search conditions over others.  
 + 
 +THIS PAGE UNDER CONSTRUCTION 
 + 
 +===== Visual Search: Strengths and Limitations of Peripheral Vision, or Presence/Absence of Basic Features? ===== 
 + 
 +Intriguingly, search is sometimes difficult even when an observer can clearly distinguish the search target from the other items in the display, known as "distractors".  For example, search for a randomly oriented T among randomly oriented Ls is difficult (Wolfe, Cave, & Franzel, 1989), even though we can easily tell an individual T from an L.  Similarly, search for a target defined by a conjunction of features – such as a white vertical bar among black verticals and white horizontals – is difficult relative to a “feature” search for a horizontal bar among verticals, or for a white bar among black (Treisman & Gelade, 1980; Treisman & Schmidt, 1982).  These phenomena imply that vision is not the same everywhere.  If it were, the easy discriminability of a focal target and distractor pair should lead to easy search.  
 + 
 +In what way is vision not the same everywhere?  Popular models of search have focused on potential differences between attended and unattended vision.  This includes not only theories such as the seminal Feature Integration Theory (Treisman & Gelade, 1980), and later Guided Search (Wolfe, 1994), but is also at least implicit in many other theories of search (e.g. Itti et al, 1998; Rosenholtz, 1999; Li, 2002; Torralba et al, 2006; Zhang et al, 2008).  According to such theories, unattended vision has access only to "basic features" like orientation, color, and motion. Therefore, a key factor determining task difficulty consists of whether or not the search target contains a "basic feature" not found in the distractors.  
 + 
 +However, an important way in which vision is not the same everywhere concerns the difference between the fovea and the periphery.  Clearly peripheral vision is important in visual search.  The periphery, being much larger than fovea, is inherently more likely to contain the target, and typically the target is peripheral until it is found. We have argued that search performance is constrained less by the capabilities of preattentive vision than by the abilities and limitations of peripheral vision. We have shown that one can predict the difficulty of a search task by measuring the discriminability of target-present from target-absent patches in the periphery.  Furthermore, we have developed a model of peripheral vision, known as the Texture Tiling Model, and shown that this model can predict not only peripheral d' but also relative search difficulty for a range of classic search conditions.  
 + 
 +In addition, some search results have seemed in conflict with the FIT story about basic features. Observers can quickly search among shaded cubes for one lit from a unique direction.  However, replace the cubes with similar 2-D patterns that do not appear to have a 3-D shape, and search difficulty increases.  These results have challenged models of visual search and attention, since such models have typically suggested that the "basic" features that lead to efficient search are lower level (orientation, motion, color) than lighting direction or 3-D shape.  We have shown that the same texture tiling model that predicts search difficulty for classic search conditions described above can also predict difficulty of these cube search tasks and their 2-D "equivalents". This suggests that cube search displays differ from those with “equivalent” 2-D search items, in terms of the informativeness of fairly low-level image statistics. This informativeness predicts peripheral discriminability of target-present from target-absent patches, which in turn predicts visual search performance, across a wide range of conditions. Comparing model performance on a number of classic search tasks, cube search does not appear unexpectedly easy. Easy cube search, per se, does not provide evidence for preattentive computation of 3-D scene properties.   
 + 
 +We can discriminate between traditional models of search and our recent texture tiling model 
 +(TTM) (Rosenholtz, Huang, Raj, Balas, & Ilie, 
 +2012b) by designing new experiments that directly pit 
 +these models against each other. We began with five classic experiments 
 +that FIT already claims to explain: T among Ls, 2 among 5s, 
 +Q among Os, O among Qs, and an orientation/luminance-contrast 
 +conjunction search. We then made fairly subtle changes to these search tasks. We made search items out of thinner or thicker bars, or lengthened or moved one of the component lines. We found that these changes lead to significant changes in 
 +performance, in a direction predicted by TTM, providing 
 +definitive evidence in favor of the texture tiling model as 
 +opposed to traditional views of search.
===== The Statistical Saliency Model: The Visual System Looks for Outliers ===== ===== The Statistical Saliency Model: The Visual System Looks for Outliers =====
Line 30: Line 59:
More recent work in our lab, in conjunction with Zhenlan Jin and Alvin Raj, has implemented this saliency model so that it can operate on arbitrary images as input.  This includes extracting motion saliency from video, and work demonstrating that saliency is predictive of where people look in video out the windshield of a car, and of time to detect a pedestrian about to cross the road. More recent work in our lab, in conjunction with Zhenlan Jin and Alvin Raj, has implemented this saliency model so that it can operate on arbitrary images as input.  This includes extracting motion saliency from video, and work demonstrating that saliency is predictive of where people look in video out the windshield of a car, and of time to detect a pedestrian about to cross the road.
 +
 +Viewed in light of our newer work on search and peripheral crowding, we view the old Statistical Saliency Model as descriptive; an attempt to predict with a simple model what stimuli would survive crowding and thus be easy to search for.
===== Search Asymmetries: Is There a Simpler Explanation? ===== ===== Search Asymmetries: Is There a Simpler Explanation? =====
Line 37: Line 68:
One difficulty in creating a simple model of visual search is that many early search results demonstrated search asymmetries. If the visual system requires a number of asymmetric mechanisms, this makes things difficult for modelers, since one must uncover all the mechanisms to have a reasonable model of visual search, and models will tend to need a new component for each asymmetric mechanism.  If this is the way the visual system is, then we need to accommodate it, of course. But I have suggested that a number of visual search experiments which gave asymmetric results were actually asymmetrically designed, and thus no asymmetric mechanism are necessary to explain the results. One difficulty in creating a simple model of visual search is that many early search results demonstrated search asymmetries. If the visual system requires a number of asymmetric mechanisms, this makes things difficult for modelers, since one must uncover all the mechanisms to have a reasonable model of visual search, and models will tend to need a new component for each asymmetric mechanism.  If this is the way the visual system is, then we need to accommodate it, of course. But I have suggested that a number of visual search experiments which gave asymmetric results were actually asymmetrically designed, and thus no asymmetric mechanism are necessary to explain the results.
-Example(s) to follow. 
- 
-Examples for which the background color matters. 
===== Bibliography ===== ===== Bibliography =====
 +---- publist ----
 +cols : title, authors, shortcite, year, summary_page, www_url, pdf_url
 +sort  : ^year
 +years : false
 +filter : type=publication
 +filter : keyword=rosenholtz
 +filter : keyword=search
 +----
 
research/visualsearch.1276114226.txt.gz · Last modified: 2010/06/09 16:10 by kimo