The uEye LE AF board-level camera is one of a range of autofocus-capable industrial cameras from IDS that feature an innovative new form of contrast-based autofocus. This novel feature has considerably extended IDS’ application range.
The technology’s ability to automatically focus leverages the active liquid lens control feature of these camera models - a feature that can be used when triggered in single-shot mode or as a continuous focus control.
The autofocus control can be individually configured to meet the needs of the application in question, delivering perfectly focused images in almost every circumstance.
Focus dialogue window in the uEye Cockpit. Image Credit: IDS Imaging Development Systems GmbH
Automatic Focus Control
Version 4.93 of the IDS Software Suite saw the addition of an automatic, software-based focus control feature to the uEye LE AF board-level camera.
This additional control complements the existing manual liquid lens control, opening up a range of new use cases for the cameras, particularly in applications working with variable object distances.
Image Credit: IDS Imaging Development Systems GmbH
It is now possible to adjust both the size and position of the region of interest while a range of image sharpness measurement algorithms are available, with each algorithm offering a variety of configuration options.
These powerful features help deliver rapid, reliable determination of maximum image sharpness in the required focus range, irrespective of the application or circumstances of use.
The IDS Software Suite 4.93 manual provides information on the use of the automatic focus feature of the uEye LE USB 3.1 Gen 1 AF. The range of configuration options available in the uEye cockpit focus dialog can also be utilized in users’ individual applications via the uEye API function is_Focus().
The manual provides comprehensive information on this function, detailing the complete range of interface parameters and a series of source code samples.
Automatic Focus Functionality
The autofocus function ensures a sharp image by automatically adjusting the camera lens to the appropriate object distance. The autofocus system makes use of the liquid lens control of the uEye LE AF board-level camera to focus at various distances.
The single-shot mode sees the autofocus triggered by the software, deactivating itself once the maximum sharpness has been determined for the measuring window.
Continuous operation mode sees image sharpness measured on an ongoing basis, continuously readjusting the lens as and when necessary throughout the entire process.
Working in a closed loop, the autofocus function manages the camera’s liquid lens based on image data analysis conducted on the host PC.
Determining Sharpness Maximum via a Closed Control Loop
The liquid lens and pixel path setting (for example, contrast and binning) will directly affect the image content when working with closed loop autofocus. This will, in turn, impact the image’s sharpness measurement result.
This evaluation will inform any liquid lens adjustment prior to the acquisition of the following image.
It is not possible to immediately calculate the optimal focus value using this image-based method, so it is necessary to pass through the closed control loop a number of times - including the image acquisition, measurement and focal length adjustment steps.
Analysis of value pairs ascertained via the focus value and calculated image sharpness is used to inform the best possible (maximum) image sharpness in the appropriate focus field.
Determining Image Sharpness
A number of approaches to measuring image sharpness exist, but each method is based on two fundamental principles.
The first of these principles is known as ‘edge sharpness’. A search is performed to look for large gray value jumps between adjacent pixels in the image content.
This process allows the edges or contours to be highlighted, with the contrast of these providing useful information on image sharpness. Clearer image edges imply a higher degree of sharpness in the original image.
Large grey value spread. Image Credit: IDS Imaging Development Systems GmbH
Low grey value spread. Image Credit: IDS Imaging Development Systems GmbH
The second of these principles is based upon the analysis of an image’s histogram values. Standard deviation calculations can provide a measuring tool for the variation of pixel values around the average gray value of the image.
The greater the standard deviation, the higher the existing gray value edges and contrast of the image. Unfocussed images would likely include large gray value jumps (edges), which take on a gradient in the form of a ramp.
This variation leads to a decrease in image contrast - a sharper image will essentially exhibit higher grayscale spread and, as a result, higher contrast.
In practice, however, object surfaces are seldom ideal for optical focus measurements, typically leading to unstable and unreliable measurement results. The uEye software's automatic focus function offers a number of measurement methods for specific image situations to help address this issue.
Algorithms, such as Tenengrad and Mean Score, use the principle of edge sharpness to facilitate image data analysis. These algorithms will analyze a captured image pixel by pixel, evaluating relationships between direct neighboring pixels.
The Mean Score algorithm uses a more basic pixel calculation and a smaller neighborhood than its Tenengrad counterpart, resulting in faster analysis but also increased sensitivity to noise.
The Histogram Deviation algorithm makes use of histogram values from the region of interest (ROI) to ascertain image sharpness – a holistic approach that provides this algorithm with filtering capabilities. Noise only has a negligible impact on results, and sharpness limits in the measured values window are displayed as clear curves.
The presence of numerous near-focus maxima in the searched focus area can cause issues, however, because the histogram deviation cannot clearly differentiate between these features. Under these circumstances, the algorithm will smooth the maxima to a covering curve, much like a low pass.
Analysis of smaller pixel neighborhoods using Mean Score and Tenengrad has a much clearer impact on gray value changes, improving edge visibility. Both of these algorithms exhibit clearly differentiable individual maxima, but these are still considerably more affected by noise and other interferences.
Image Credit: IDS Imaging Development Systems GmbH
Automatically Focussing uEye Cameras
As they operate on different computing principles, each of the three algorithms outlined here will take a different amount of time to calculate image sharpness.
The use of complex pixel transformations, such as Sobel and Laplace filters, involves a higher computing load than a basic standard deviation calculation taken from a mean gray value.
It should also be noted that the analysis of larger regions of interest will quickly reduce the acquired camera frame rate. It is possible to influence the measurement’s speed and accuracy by opting to use an appropriate algorithm, though this is dependent on the requirements of the application in question and the scene.
Mean Score and Histogram Deviation algorithms are better suited to time-critical calculations where only average PC hardware is available, but any efficiency improvements afforded by the Tenengrad calculation will generally be at the expense of achievable frame rate when working with larger measurements windows.
Selecting a Region of Interest and Focus Range
Calculation time can be enhanced by performing a sharpness measurement in a region of interest - often referred to as a ‘measurement window’.
More computing time and power from the host CPU are required to complete a sharpness calculation on larger measurement windows.
Set frame rate may also be adversely affected if the calculation time (per image) is too long. It is prudent to opt for a measurement window that is as small as possible to ensure sufficiently fast frame rates.
It is advisable to specify the size and position of the measurement window, ensuring that only the image content to be focused is marked. This should feature one focus plane and, therefore, one maximum focus.
A number of local sharpness maxima in the specified search range alongside a peak search algorithm utilized with a termination condition will result in the algorithm prematurely terminating the search upon detection of the first clear maximum.
Under these circumstances, several options are available to specify the search more precisely to accommodate the desired focus level.
Measurement window and focus range can contain several focus maxima. Image Credit: IDS Imaging Development Systems GmbH
Restricting the Focus Range of the Lens
Restricting the lens’ focus range involves limiting the focus measurements to just the focus range of the lens needed for the application.
A small area may be adequate, though this depends on the lens’ depth of field and the distance between lens and image scene. This solution reduces maximum search time due to its use of fewer sharpness calculations.
Should users be unsure of where to set the focus range, it is recommended that they show the measured values window and perform a full scan of the whole focus range using a small hysteresis.
Restricting the focus range of the lens. Image Credit: IDS Imaging Development Systems GmbH
Change the Position and Size of the Measurement Window
Only a few clear features are required in order for the algorithms to determine sharpness; for example, a clear edge, small structures or clear gray value scattering. A smaller measurement window will allow the focus to be determined faster, with only one image area more clearly in focus.
It is, therefore, important to make the measurement window as small as possible while keeping this as large as necessary. The more reproducible the image position of the focus objects is in an application, the easier it is to pre-set the size and position of the measurement window.
Observe size and position of measurement window. Image Credit: IDS Imaging Development Systems GmbH
Determine Maximum Sharpness
The autofocus function has been developed to rapidly determine the focus setting offering the optimum sharpness values in the measurement window of the captured images.
To achieve this, the uEye software offers a range of basic peak search algorithms. These algorithms are suitable for different types of scenes (image content) or can be selected according to the user’s speed and quality requirements.
The slowest but most reliable method sequentially sets the focus by acquiring one image at a time to determine its sharpness.
The search for optimal sharpness value can be expedited by selecting a number of random samples according to different search strategies, for example, skipping focus settings first (intervals) and locating peaks during further runs. These are scanned much more closely.
These individual algorithms employ a number of abort conditions to optimize search time, but this approach does not always yield ideal results.
The uEye software empowers users to choose the most appropriate search strategy for their needs.
As the focus calculation is based on the image content and a number of measurements are required to locate an optimum value, it is important that the image content remains constant throughout the search process.
Search Algorithm Characteristics
Golden Ratio Search
The Golden Ratio Search algorithm refines the maximum search by subdividing the search area by the golden ratio. This approach offers exceedingly fast search speeds, even for substantial search areas.
The maximum sharpness may occasionally be skipped during the rough search when using a large search range, an issue which can be addressed via advance reduction of the search range or by only employing this algorithm with a unimodal sharpness gradient (with a single sharpness maximum).
Hill Climbing Search
The Hill Climbing Search algorithm starts the rough search by employing a wide range and a large search interval. Once a maximum value is detected (when the sharpness values no longer increase), the algorithm will abort searching in the next focus area to exhibit a decrease in sharpness value.
The algorithm will then refine measurement results according to the defined maximum range, continuing to do so using smaller search intervals in subsequent search runs.
This search only aborts when the hysteresis is undercut. It is less time-consuming than Global Search, but there is a risk that the first maximum determined is not the actual global maximum.
Since the maximum sharpness may be skipped during the rough search when employing a wide search range, it is also advisable to reduce the search range in advance or only utilize this algorithm with a unimodal sharpness gradient.
Global Search
This algorithm also starts the rough search with a large search interval and in a wide range. The rough search is undertaken in the entire search, with measurement results refined according to the found maximum range in subsequent search runs. Subsequent runs will also use smaller search intervals.
The Global Search algorithm will only abort when the hysteresis is undercut. While this approach is quicker than Full Scan, maximum sharpness may be skipped during the rough search when working with a large search range.
It is often prudent to reduce the search range in advance or to only utilize this approach with a unimodal sharpness gradient. Both of these approaches can help address the limitations of this algorithm.
Full Scan
The Full Scan algorithm calculates sharpness values in the entire search range in a single pass. This approach does not use a search strategy; rather, hysteresis is used to define the measuring interval.
This algorithm is ideal for blind searching. It is able to reliably locate the global sharpness maximum with a small hysteresis, but this search can be time-consuming when working with small hysteresis and wide search ranges.
In the worst-case scenario, a total of 1024 focus values will be determined; the maximum count of focus settings of the uEye LE AF.
Some algorithms may end a search prematurely or finish this too close to focus settings. If the depth of field in the area of interest is not large enough, the global maximum may not be located. Only a Global Search or Full Scan is likely to be successful under these circumstances.
Maximum Search Accuracy
Hysteresis determines a minimum step size, and once this is reached, the search for the maximum sharpness is halted. This also dictates the maximum search’s achievable accuracy.
In Full Scan, however, hysteresis defines the constant step size employed in scanning the entire search area.
Commonly Encountered Focus Camera Applications
Liquid lens and automatic focus capabilities allow the uEye LE AF series of board-level cameras to be employed in any application which involves variable object distances.
The camera’s software enables either manual or automatic adjustment of the focus plane – a highly beneficial feature in scenarios where it is not possible to access the lens itself.
When employed in mobile applications such as movable robot arms, a small board-level camera is able to focus on objects or codes, allowing these to be read clearly following each movement.
When used with image processing functions such as HALCON, the uEye automatic focus system offers excellent potential. HALCON can be used to calculate a continuously sharp image that clearly illustrates an object scene at all focus levels, irrespective of focus setting, effectively providing an HDR image related to focus rather than exposure (High Dynamic Focus).
Depth information, calculated from focal values of all focused planes. Image Credit: IDS Imaging Development Systems GmbH
HALCON is also able to scan an object scene on every focal plane, meaning it can calculate depth information (Depth from Focus) based on sharpness measurements derived from these images.
This powerful functionality enables the determination of object volume without a 3D camera, as well as ascertaining whether or not objects reside on the same floor.
A typical HALCON installation incorporates a range of ready-made focus image processing examples suitable for use directly with the uEye LE AF.
To help users leverage the power of this technology, the IDS Software Suite 4.93 manual features further code examples, a comprehensive description of the focus function and details on its complete range of interface parameters.
This information has been sourced, reviewed and adapted from materials provided by IDS Imaging Development Systems GmbH.
For more information on this source, please visit IDS Imaging Development Systems GmbH.