Ruth Rosenholtz

The Visual System as Statistician

In studying the workings of the visual system, we are often faced with quite complicated and seemingly messy empirical results from behavioral experiments. For instance, it is easier (against a neutral gray background) to search for a red target among pink distracting element, than vice versa. Is this because the brain has detectors that respond strongly to red and not pink, but no detectors that respond more strongly to pink than red? One of our key tasks as researchers is to ask whether the data necessitate an explanation like this, in which special status is given to certain features/operations/etc. Or is there a simpler explanation? Is the observed behavior in some sense ideal? Questions of this sort often lead us to examine statistical models of human behavior.

Much of the work in our lab has been aimed at testing the hypothesis that the visual system, in many instances, has an interest in operating as a statistician. In other words, the visual system samples (noisy) feature estimates from the visual input, in many cases computes summary statistics such as the mean and variance of those features, and makes intelligent decisions based upon those statistics.

Models of this form have proven quite powerful in predicting behavior at visual tasks, particularly when processing needs to be fast or “pre-attentive,” or when the stimulus is viewed in the periphery. For example, many of the existing results in visual search can be qualitatively predicted by a model that extracts the mean and covariance of basic features like motion, color, and orientation.

Pre-attentive texture segmentation, on the other hand, is well modeled by a process that takes a sample of features from each side of a hypothesized texture boundary, and does the equivalent of a t- and F-test to see if the textures differ significantly in their statistics. If so, the observer is likely to perceive a boundary.

More recent work in our lab suggests that one can predict the difficulty of doing a task in peripheral vision (i.e. under conditions of crowding) with a model that represents peripheral stimuli not through the exact configuration of their parts, but rather through the joint statistics of responses of V1-like receptive fields. This model shows promise at being able to predict a wide variety of visual phenomena, from optical illusions through predicting search reaction time for arbitrary search displays.

There are various benefits to this approach to studying vision. First and foremost, the resulting models often work quite well. They often have fewer parameters than neurally inspired models (which often, when analyzed, turn out to be implicitly performing similar calculations). Statistical models are often surprisingly easy to implement in biologically inspired hardware, and similarly easy to implement in computer vision algorithms which can then make predictions for arbitrarily complex natural images.

See the new FAQ here.====


  • Capacity limits and how the visual system copes with them Rosenholtz, R., HVEI, 2017. Abstract PDF
  • Search performance is better predicted by tileability than by the presence of a unique basic feature Chang, H., Rosenholtz, R., Journal of Vision, 2016. Abstract PDF
  • Capabilities and limitations of peripheral vision Rosenholtz, R., Annual Review of Vision Science, 2016. Abstract PDF
  • A general account of peripheral encoding also predicts scene perception performance Ehinger, K. A., Rosenholtz, R., Journal of Vision, 2016. Abstract PDF
  • Cube search, revisited X. Zhang, J. Huang, S. Yigit-Elliott, and R. Rosenholtz, Journal of Vision, 2015. Abstract PDF
  • Texture Perception R. Rosenholtz, Handbook of Perceptual Organization, 2014. Abstract PDF
  • A summary statistic representation in peripheral vision explains visual search R. Rosenholtz, J. Huang, A. Raj, B. J. Balas, and L. Ilie, Journal of Vision, 2012. Abstract PDF
  • Rethinking the role of top-down attention in vision: Effects attributable to a lossy representation in peripheral vision Ruth Rosenholtz, Jie Huang, and Krista A. Ehinger, Front. Psych., 2012. Abstract PDF
  • What your visual system sees where you are not looking Ruth Rosenholtz, Proc. SPIE: HVEI, 2011. Abstract PDF
  • What Your Design Looks Like to Peripheral Vision Alvin Raj, Ruth Rosenholtz, APGV, 2010. Abstract PDF
  • A summary-statistic representation in peripheral vision explains visual crowding B. Balas, L. Nakano, and R. Rosenholtz, Journal of Vision, 2009. Abstract PDF
  • Uniform versus random orientation in fading and filling-in C. H. Attar, K. Hamburger, R. Rosenholtz, H. Gotzl, & L. Spillman, Vision Research, 2007. Abstract PDF
  • The effect of background color on asymmetries in color search R. Rosenholtz, A.L. Nagy, N. R. Bell, Journal of Vision, 2004. Abstract PDF
  • Search asymmetries? What search asymmetries? R. Rosenholtz, Perception & Psychophysics, 2001. PDF
  • A simple saliency model predicts a number of motion popout phenomena R. Rosenholtz, Vision Research, 1999. Abstract PDF
  • General-purpose localization of textured image regions R. Rosenholtz, NIPS, 1999. Abstract PDF
research/visualstatistician.txt · Last modified: 2011/10/11 10:19 by rosenholtz