The Use of Image Blur as a Depth Cue in Human Vision

(Research project funded by EPSRC, ref. GR/L20948)

George Mather, University of Sussex

 

This document summarises research conducted under a grant awarded by EPSRC. The work was done in collaboration with Dr. David R. R. Smith as Post-Doctoral Fellow.

Introduction

Human depth perception is supported by a range of visual cues such as stereopsis, interposition, relative size, and texture gradient (review in Howard and Rogers, 1995). Research conducted under this project concerned the utility of image blur as a cue to depth. Cameras and eyes have limited depth of focus, so images of objects nearer or farther than the point of focus or fixation are blurred. The presence of blur at a particular point in the image therefore constitutes a cue to the distance of that point from the observer. Prior to the project there had been very little research on blur mediated depth (Pentland, 1987; Marshall et al., 1996). The main aims of the project were to:

  1. Assess the precision with which human observers can discriminate blur extent in complex images.
  2. Determine the extent to which such image blur is used as a depth cue.
  3. Investigate interactions between image blur and other depth cues.

Blur Discrimination

Previous research on human blur discrimination has been confined to the study of blur discrimination in isolated luminance edges (Watt and Morgan, 1983; Georgeson, 1994; Paakkonen and Morgan, 1994). A series of experiments was conducted under the project to investigate blur discrimination in more complex images, including random fractal texture, binary texture, extended luminance edges (L-edges), and extended texture edges (T-edges; blurred border between different textures). The figure below shows examples of the stimuli used in these experiments.

 

 

A separate experiment was conducted for each image type. In each experiment, observers made forced-choice decisions between pairs of images that had been blurred by different amounts, identifying the image that appeared more blurred. Probit analysis (Foster & Bischof, 1997) was used to estimate the threshold for detecting a difference in blur between the two images. Results of these experiments were reported in Mather (1997), Mather (2000), and Mather and Smith (in preparation). The figure below shows blur discrimination thresholds obtained in each experiment. Data are expressed in terms of the minimum change in blur required for reliable discrimination, as a function of initial or ‘reference’ blur.

 

Data conform to a typical ‘dipper’ function, showing an initial improvement in performance at small reference blurs. The minimum of the discrimination function occurs at a reference blur equal to the intrinsic blur of the visual system (due to optical and neural factors). The fitted curves assume a constant Weber fraction for blur discrimination (Db), allowing for a fixed degree of intrinsic blur (si):

 

Db = Ö (w2 . (se2 + si2) – si2) – se

 

se is reference blur. The table below gives the estimated Weber fraction and intrinsic blur space constant from each experiment.

 

Weber Fraction (1-w)

Intrinsic Blur
 si (min arc)

r2

Luminance
edge

0.21

1.31

0.96

Texture
edge

0.59

1.30

0.97

Binary texture

0.28

0.73

0.98

Fractal texture

0.12

0.74

0.89

 

 

Blur discrimination for texture edges was much worse than for the other edges, with a Weber fraction of nearly 60%. Discrimination was best for fractal textures, at 12%, indicating that these textures optimally stimulated the visual filters mediating performance. However, Weber fractions for blur discrimination were generally rather large compared to the optimal value of 2-4% observed for moderate spatial frequency discrimination (Hirsch & Hylton, 1982). Blur discrimination may rely on responses at high spatial frequencies (Mather, 1997), and Smallman et al. (1996) reported Weber fractions of approximately 10% at these frequencies.

Estimated intrinsic blur may represent the spatial scale of the filters mediating discrimination. For textures, the estimated filter space constant is approximately 0.74 min arc, while for large-scale edges (both luminance and texture defined) the estimated filter space constant is approximately 1.3 min arc. It is important to note that texture edges belong to a large class of second-order images, and are thus invisible to linear filters. Blur discrimination in texture edges must involve a non-linear transformation, such as that proposed by Bergen and Landy (1991) in their model of texture analysis.

In summary, blur discrimination performance indicates that blur variation is likely to offer a relatively coarse quantitative cue to depth variation. Differences in blur between texture regions are likely to offer a more sensitive cue than differences between large-scale edges.

Use of image blur as a depth cue.

Computational work done under the project clarified the physical nature of the depth cue offered by blur variation. The lefthand equation below relates image blur width (s) to viewing parameters, and was derived in Mather and Smith (2000).

 

 

The equation is very similar in form to the equation for stereoscopic disparity (shown on the right). The figure below plots computed blur and disparity as a function of object distance, assuming fixation at 100cm.

Mather and Smith (2000) showed that there is a direct relation between blur and disparity, given two scaling constants (inter-pupillary distance and pupil diameter). Disparity is signed, but blur is not, so a difference in blur between two regions cannot specify which region is nearer. However, if an occluding edge is present between the regions, then its degree of blur can be used to establish depth ordering. If the edge is sharp, and one region contains sharply defined texture, then that region must be nearer than the blurred region. If the edge is blurred, then it must belong to the blurred region, which must therefore be nearer than the sharp region.

We conducted a series of experiments to establish the extent to which blur variation in regions and borders influences perceived depth ordering. Observers were shown images containing two textured regions separated by a vertical sinusoidal border. One region was always sharp, and the other was blurred (see example below). The two regions had equal space-average luminance, and equal Michelson contrast. Observers reported whether the lefthand or righthand region appeared furthest away, by means of a button press recorded by computer.

 

In 50% of presentations the border between the regions was sharp, and in the remaining presentations the border was blurred. Border blur and region blur were varied independently. Region blur was either 4, 8, or 16 min arc. Border blur was either 0, 4, 8, or 16 min arc. Two viewing distances were employed (57 and 114cm). Texture pattern, and laterality of the blurred region, varied randomly from trial to trial. The figure below shows the mean percentage of trials in which the blurred region was judged as “far” as a function of border blur space constant (six observers). Each data set represents a different combination of region blur and viewing distance (see legend).

As expected on the basis of blur-mediated depth, the blurred region was seen as far when the border between the blurred region and the sharp region was sharp, but was seen as near when the border was blurred. However, it is also clear that:

1)       There are no consistent effects of region blur space constant (over a four-fold range) or of viewing distance, since the data sets largely overlap. There was also little effect of viewing distance and pupil diameter in pilot experiments.

2)       Border blur must exceed approximately 6 min arc in order for the blurred image region to be seen as near rather than far, regardless of the extent of region blur.

3)      There was some degree of ambiguity in all displays, since judgements never approached 100% in one direction or the other, clustering instead near 75% (a similar degree of ambiguity was obtained in pilot experiments using different stimuli).

The same pattern of results was obtained using a sinusoidal border of lower amplitude. The lack of quantitative effects for region blur, viewing distance, and pupil diameter indicates that blur is used as a qualitative cue to depth. The need for a relatively large border blur to reverse apparent depth ordering can be related to the relatively poor discriminability of border blur obtained in the experiments reported earlier. The ambiguity of the displays may imply that blur cues are most effective when present in combination with other depth cues. Experiments on depth cue interaction are reported in the next section.

Interactions between image blur and other depth cues

Interactions between blur and stereopsis

Two series of experiments were conducted in this area of the project. The first series investigated interactions between blur and stereopsis. Observers viewed two random dot stereograms (RDS) in a 2AFC task, and were required to identify the RDS depicting the greatest depth. In control observations all dots in both RDSs were sharply defined, or all dots were blurred. In experimental observations one RDS (comparison) contained sharply defined dots, and the other RDS (reference) contained differential image blur as well as binocular disparity (eg. a sharply defined central square with near disparity against blurred background dots). We measured the comparison disparity required to produce an apparent depth match with a specific reference disparity. Blur space constant in reference RDSs was fixed at 4.5 min arc. Disparity ranged from –1.76 to +1.76 min arc. According to the formulae presented earlier, the two cues defined very different depth intervals. The blur cue corresponded to a depth interval of 113 cm from the screen (at the 114 cm viewing distance), and the stereo cue corresponded to a very much smaller depth interval of +/- 1.1 cm. If the two cues are summed or averaged, then apparent depth should be greater in RDSs containing both stereo and blur cues than in RDSs containing only the disparity cue. If stereo dominates or vetos the blur cue, there should be little or no difference in apparent depth between experimental and control conditions (cue combination is discussed in Howard & Rogers, 1995).

Results for six observers showed only a marginal effect of blur on depth judgements when the blur cue was consistent with the disparity cue. The lefthand figure above shows that apparent depth matches in control stimul (no blur variation) were obtained at matching physical disparity values. For experimental stimuli (righthand graph), matches departed from unit slope (equal physical disparity) only for ecologically valid stimuli (a sharp central square with near disparity against blurred dots). However the increase in apparent depth of less than 1 min arc is negligible compared with that required for equality with the depth signalled by blur (1.5 deg arc of disparity). To test whether stereo dominates only when there is a gross discrepancy between the two cues, we conducted a subsequent experiment in which the discrepancy between the cues was reduced. The same result was obtained, namely stereo dominance.

We concluded that image blur makes a negligible contribution to the impression of depth seen in RDSs, at least when measured against depth seen in RDSs without blur cues. This may be because the two cues operate over different distance ranges in natural viewing conditions, with blur becoming detectable only at depth intervals beyond the range of disparity coding mechanisms (see below). It may be more appropriate to investigate the integration of blur cues with pictorial depth cues, using a technique that does not involve stereo judgements. This was attempted in the final series of experiments.

Interactions between blur, interposition and contrast

A series of experiments investigated interactions between blur and two well-known pictorial depth cues, interposition and contrast. We developed stimuli of the kind depicted on the right. The image contained four textured tiles. The apparent depth order of the tiles was defined by a combination of (i) blur (increasing at greater depths); (ii) contrast (decreasing at greater depths); and (iii) interposition (nearer tiles occluding farther tiles). The example on the right includes all three depth cues, but we also generated images containing single cues and all possible pairwise combinations of cues.

To assess the effectiveness of the images, we employed a task akin to navigating through layered windows in a graphical computer interface. Subjects were required to indicate the apparent depth ordering of the tiles by moving the mouse pointer inside each tile in turn (starting with the nearest) and clicking once on each. The computer recorded errors in reported depth ordering, and the time taken to register each click.

Since there were 24 permutations of depth ordering, the probability of reporting the correct order by chance was 0.0417. The lefthand graph below shows that all cue combinations permitted reliable depth ordering.

 

 

 

Accuracy was always highest for combinations including blur, and lowest for combinations including interposition, indicating that blur is the more salient cue to depth ordering. The righthand graph shows the mean time between successive clicks (open bars), and the time taken to make the first click (hatched bars), for correct responses only. The main difference between the stimuli was in the length of time needed before making the first click. Responses were slowest for single cues, and fastest for all three cues, and for combinations of contrast and interposition.

General Conclusions

In physical terms, blur variation offers a lawful quantitative cue to depth, as shown in earlier figures. However, sensory limitations on the ability of the visual system to discriminate small differences in blur restrict its utility as a quantitative cue. The Weber fraction for stereopsis is approximately 0.06 (Howard & Rogers, 1995), much smaller than the values of 0.12 - 0.59 found in our experiments for blur discrimination.

The graph on the left illustrates the physical relation between binocular disparity and retina image blur at two extreme pupil diameters, 1mm (broken line), and 8mm (solid line), based on formulae presented in Mather and Smith (2000).

The effective range of disparities stretches up to approximately 50 min arc, whereas retinal blur becomes detectable reliably only at space constants above 2 min arc (see hatched areas). As a result, disparity and blur cues cover different depth ranges, with disparity operating at relatively small depth intervals, and blur operating over relatively large intervals.

 

We conclude that blur is probably best regarded as a qualitative, pictorial cue to depth operating over relatively large depth intervals, comparable to other pictorial cues such as interposition, contrast, etc.

References

Bergen JR, Landy MS (1991) Computational modelling of visual texture segregation. In MS Landy & JA Movshon (Eds) Computational models of visual processing. MIT Press.

Foster D. H. & Bischof W. F. (1997). Bootstrap estimates of the statistical accuracy of thresholds obtained from psychometric functions. Spatial Vision, 11, 135-139.

Georgeson M (1994) From filters to features: location, orientation, contrast and blur. In Higher Order Processing in the Visual System, 147-165. Ciba Foundation Symposium 184. Chichester, Wiley.

Hirsch J, Hilton R (1982) Limits of spatial frequency discrimination as evidence of neural interpolation. J. Opt. Soc. Amer. 72, 1367-1374.

Howard I. P. & Rogers B. J. (1995). Binocular Vision and Stereopsis. Oxford: Oxford University Press.

Marshall J., Burbeck C., Ariely D., Rolland J. & Martin K. (1996). Occlusion edge blur: a cue to relative visual depth. Journal of the Optical Society of America A, 13, 681-688.

Mather G (1997) The use of image blur as a depth cue. Perception, 26, 1147-1158.

Mather G, Smith DRR. (2000) Depth cue integration: stereopsis and image blur. Accepted by Vision Research.

Mather G, Smith DRR (in prep) Blur discrimination as a limiting factor in blur-mediated depth perception.

Pentland A. P. (1987). A new sense for depth of field. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9, 523-531.

Paakkonen A and Morgan M (1994) Effects of motion on blur discrimination. J. Opt. Soc. Amer. A11, 993-1002.

Smallman HS, MacLeod DIA, He S, Kentridge RW (1996) Fine grain of the neural representation of human vision. J. Neurosci. 16, 1852-1859.

Watt RJ, Morgan MJ (1983) The recognition and representation of edge blur: evidence for spatial primitives in human vision. Vision Res. 23, 1465-1477.