The Internal Statistics of Natural Images and Videos

A picture describing The Internal Statistics of a Single Natural Image/Video

Small image patches tend to repeat "as is" many times inside a single natural image, both with as well as across different scales of the image. This property also holds for small space-time patches inside natural videos. In a series of papers, we have:

  1. quantified this strong patch recurrence property within natural images/videos.
  2. compared and showed the superiority of such internal image-specific patch prior over external generic patch prior (extracted from a large collection of natural images).
  3. have shown how the internal patch recurrence property forms a strong prior for solving a variety of ill-posed vision problems, including:
    object and action detection, image and video completion, visual summarization, super-resolution (in space and in time), and more.
  4. have shown that this strong patch-recurrence property significantly diminishes under non-ideal imaging conditions. The deviations from the perfect patch recurrence behavior can be exploited to recover the unknown degradation process of the sensor (e.g., its blur), or the unknown degraded physical conditions of the scene (e.g., atmospheric haze). In particular, we have shown how this approach leads to Blind Super-Resolution, Blind-Deblurring and Blind-Dehazing.