Types of deconvolution include linear, nonlinear, and a combination of the two.

Large objects, which are low-frequency and high-contrast, are easily detected by an optical system, while small objects increase in the frequency domain and typically have reduced contrast.

Reducing contrast with increasing spatial frequency, at some point the “cut-off frequency” is met in the frequency domain and the optical system collects no information above this spatial frequency point.

Linear deconvolution

Linear deconvolution involves amplification of contrast only within the cut-off frequency; it will not create higher frequency components above that spatial threshold, and thus is condidered reliable. Linear deconvolution may have very positive effects on image quality if artifacts are minimized. However, linear deconvolution can impart a ringing artifact, meaning that positive and negative intensity oscillations appear outside the imaged object. Ringing can reduce overall image resolution, while individual measurable objects may exhibit reduced FWHM measurements.

Nonlinear deconvolution

More effective to improve an image’s appearance than with linear deconvolution. an approach that estimates the object by revisiting the result over multiple iterations. (MLE, Fast MLE, Gold, etc.)

Estimation accuracy depends on object structure.

• A point can be easy to estimate
• Estimation of denser structures is more difficult. Structures will shrink, but resolution may not improve.

Components above the cut-off frequency can be created by estimation, and data not captured by the optical system can produce artifacts.With too many estimation iterations, a 100 nm bead can be reduced to 80 nm, resulting in a loss of data.

Besides object density, estimation results can also be impacted by factors such as imaging condition and processing parameters such as PSF shape, iteration number, and nonlinear deconvolution mode. The validity of estimation can be judged only by the appearance of the image.

• linear methods can produce ringing effects that reduce FWHM without improving resolution. And only the components within the cut-off frequency are enhanced.
• Nonlinear methods may also offer lower reliability because of potential amplification above the cut-off frequency.

In practice, nonlinear methods that dominate the market are great tools for reducing blur in three dimensions as well as reducing noise.

Tips to keep in mind

• Weigh the pros and cons of the various deconvolution modes
• Deconvolution can be overdone—seek a balance between artifacts and restored resolution;
• FWHM does not equal resolution—it is possible to reduce FWHM and not improve resolution;
• While artifacts occur with all deconvolution methods, they can be minimized;
• Deconvolution is an excellent tool for improving image quality, as long as the factors above are adequately considered.

Deconvolution algorithms

• Wiener invert filter
• the modified Gold iterative constrianed algrotihm (MGold)
• the quick MLE algorithm
• the quick ICTM algorithm
• blind MLE deconvolution.

Deblurring algorithms (fundamentally 2-D)

No-neighbor/unsharp masking

Subtract contributions from approximated out-of-focus infromation.

nearest-neighbor (NN)

NN was the first pragmatic attempt to deblur 3D microscopic with a 2D method and insensitive to axial sampling.

Operate on a plane z by blurring its neighboring planes z +- 1 (using a digital blurring filter), then subtracting the blurred planes from z.

multi-neighbor

Extend the concept of nearest-neighbor to a user-selectable number of planes

Advs and disadvs

• computationally economical, allow to be part of data acquisition.
• noise from several planes is added together.
• remove blurred signal and thus reduce signal levels.
• features whose PSFs overlap in z may be sharpened in planes where they do not really belong (i.e., the apparent position of features may be altered).

Deblurring algorithms improve contrast, but they do so at the expense of decreasing signal-to-noise ratio and may also introduce structural artifacts. these algorithms cause artifactual changes in the relative intensities of pixels;

Morphometric measurements, quantitative fluorescence intensity measurements, and intensity ratio calculations should never be performed after applying a 2-D deblurring algorithm.

Image restoration algorithms

Attempt to reassign blurred light to an in-focus location. Restoration algorithms estimate the object, following the logic that a good estimate of the object is one that, when convolved with the PSF, gives back the raw image.

Inversed filters

Take the Fourier transform of an image and divides it by the Fourier transform of the PSF.

Assuming Gaussian noise model, the MLE can be expressed as

\[\arg\min_x{||y-hx||^2_2},\]

set its derivative equal to zero, we can get

\[F(\tilde{x})=\frac{F(y)}{\max{F(h),\epsilon}},\]

where $F$ is Fourier Transformation, $\epsilon$ is a small value.

A direct, linear inversion of the forward model. It can also be seen as a standard convolutional filter using an inverted PSF.

• Poor resistance to noise.
• The calculation is rapid, limited by noise amplification.
• An artifact known as “ringing” can be introduced.

Regularized Inverse Filter (RIF)/Wiener Filter

To counter instability due to ill-posedness (e.g., noise and PSF singularities) regularization (l2 norm) is added. (MAP)

\[\arg\min_x{||y-hx||_2^2+\lambda||x||_2^2}\]

To find the extremum, we get:

\[F(\tilde{x})=\frac{F^*(h)\cdot F(y)}{F(h)^2+\lambda}\]

where x is the restored signal, $\lambda$ is the regularization parameter, * denotes conjugate.

Noise amplification and ringing can be reduced by making some assumptions about the structure of the object that gave rise to the image.

Regularization can be applied in one step within an inverse filter, or it can be applied iteratively.

Much of the “roughness” being removed here occurs at Fourier frequencies well beyond the resolution limit and, therefore, does not eliminate structures recorded by the microscope.

Image restoration methods preserve total signal intensity but improve contrast by adjustment of signal position. Therefore, quantitative analysis of restored images is possible and, because of the improved contrast, often desirable.

Constrianed iterative algrotihms

High sensitivity to PSF quality

1. An estimate of the object is made (this is usually the raw image itself).
2. The estimate is convolved with the PSF, and the resulting “blurred estimate” is compared with the raw image.
3. This comparison is used to compute an error criterion that represents how similar the blurred estimate is to the raw image.
4. This error criterion (or “figure of merit”) is then used to alter the estimate in such a way that the error is reduced. [Tihkonov, Landweber, Tikhonov–Miller, Fast iterative soft-thresholing, Richardson–Lucy, Richardson–Lucy with total-variation regularization]

As iterations proceed, the algorithm will tend to amplify noise, so most implementations suppress this with a smoothing or regularization filter.

Cosntraints

• ‘nonnegtivity’. (any pixel value in the estimate that becomes negative during the course of an iteration is automatically set to zero)
• “boundary constraints” on pixel saturation
• constraints on noise statistics
• TV suppresses noise and preserves edges but can produce a staircase effect.
• Hessian can suppress the staircase effect but will also cause edge blurring.
• HDTV (high degree total variation)
• Non-local regularization¹.

Landweber (LW) deconvolution

It is considered effective in mitigating the noise amplification problem caused by direct inverse filtering. Solving the maximum likelihood through iterative gradient descent is highly computationally intensive. It is overly sensitive to noise.

Meinel/Gold’s method

Posotivity is implicit. Requires only a few (less than 10) iterations to vomplete. This method only works correctly on perfectly symmetric PSFs which, unfortunately, are rarely present when immersion oil is the primary source for aberrations not known to the system .

Richardson-Lucy iterative

A rigorous statistical approach.

\[\tilde{x}^{n+1}=x^n[(\frac{y}{h*\tilde{x}^n})\cdot h]\]

• Many iterations (sometimes thousands) are required.
• With a rising number of itertions, the result becomes instable (noise and artifacts), and the likelihood will decrease again at some point.

Reference

1. Roels, J. et al. Bayesian deconvolution of scanning electron microscopy images using point-spread function estimation and non-local regularization. in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 443–447 (IEEE, Orlando, FL, USA, 2016). doi:10.1109/EMBC.2016.7590735.