AZoMaterials speaks to Bettina Trauernicht and Lord Hess at Bruker about how the ACQUIFER HIVE system is transforming microscopy-based research by tackling the growing challenge of big image data management.
Watch the full webinar
What challenge does the HIVE system aim to solve in microscopy-based research?
The Aquifer HIVE has been developed by an interdisciplinary team of engineers, biologists, microscopists, physicists, and IT experts with many years’ experience in big data management. At its core, HIVE is a solution for handling large image data, currently deployed in over 100 imaging facilities
sCMOS detectors are continually improving, and it is possible to acquire a lot of information from them, but this has led to a massive deluge of big image data that must be processed.
For example, a single FISH recording could feature five tiles with 340 slices each, each slice recorded in two colors. All of these tiles could be recorded six times in one hour over a period of 16 hours. At the end of this process, we could be dealing with around 2.5 terabytes of image data.
Other experimental applications work with even more image data. For example, an experiment looking at the development of a Drosophila fruit fly with Bruker's Luxendo light sheet technology could record 21,600 time points from both sides. This highly optimized system records 100 frames per second, and each frame has a file size of 8 megabytes. This experiment would result in a data set that is 17 terabytes in size.
With data volumes like these becoming increasingly common, the HIVE system is designed to efficiently tackle the major bottleneck in microscopy workflows: handling and transferring massive image datasets.
Image Credit: MAXIMUM ART/Shutterstock.com
How does the HIVE system minimize data transfer delays and microscope downtime?
A typical processing scenario in either a multi-microscope environment or a single-microscope environment begins with the microscope itself.
The microscope starts to acquire data, which is stored on the computing unit that controls the microscope. Before analyzing this data, we could typically copy it to an internal server and then onto a processing workstation able to analyze it. HDDs or SSDs may also be used to move the data around, copying data onto transportable devices and then onto workstations from there.
There are typical limitations in these processing scenarios. First of all, depending on the network infrastructure, the transfer of the data itself could take a long time. Microscopes must often remain idle while copying this data, because copying data and acquiring data at the same time is not always possible. This decreases the uptime of the microscope.
Using transportable SSDs or HDDs introduces additional risks, including potential security vulnerabilities or system instability.
A typical 1 gigabit network transfers data at around 100 megabytes per second, which is relatively slow for large datasets. For example, copying the 2.5-terabyte dataset mentioned earlier would take about 7.3 hours per step, first from the microscope control unit to an internal server, and then to a processing workstation.
Copying the 17-terabyte Drosophila dataset in two steps would take about 49.5 hours per transfer.
An SSD would take around 3 hours to move the 2.5 terabytes of data from the control unit of the device to the system for analyzing, but the 17-terabyte data set would pose a challenge because it is hard to find SSDs with data transfer rates of 500 megabytes per second.
These issues can also stem from multiple devices. For example, wide field confocal or spinning disc confocal systems are producing data in parallel, resulting in so much data that it can crash a 10-gigabit network.
HIVE solves these issues in big image data handling and microscopy by combining networking, storage, and computing. Data streams are taken from the microscopes and live-streamed onto the HIVE COMPUTE platform and into the HIVE DATA module. Our philosophy centers around not moving the data once it has been acquired. Instead, we move the user to the data.
Four HIVE modules are available. The HIVE NET module is used for access and communication, and the HIVE DATA module is the storage space for microscopy data coming from high-content screening, omics data, sequencing data, and more. The HIVE CORE unit is able to provide a multi-user login for data processing with a wide range of software solutions available, and the HIVE GPU module is used for visualization and analysis.
Can you explain the function of the HIVE NET module in the system architecture?
The HIVE NET module is the internode for the microscope. This module features HIVE Switch, a multi-plug for microscopes that can plug into more than 21 devices.
A DHCP router system and firewall help users generate a secure subnetwork for microscopy, while an uninterruptible power supply (UPS) inside the HIVE NET helps keep the HIVE running in the case of electrical fluctuations or loss. This UPS will also safely shut down the system in the event of a prolonged power outage.
It was important that we tested and approved the system to ensure its compatibility with a wide range of different microscopes and hardware vendors. This allows us to guarantee that the HIVE COMPUTE platform is available to stream the data from all a user’s devices.
What makes the HIVE DATA module suitable for large-scale scientific environments?
The HIVE DATA module is available in multiple sizes, ranging from 52 terabytes to 238 terabytes of usable space. Because it is a modular system, we also have the option to connect plug-and-play extensions of the first data module with other data modules, upgrading this data volume up to a one-petabyte solution.
This module uses RAID 6 technology to ensure data safety. This means the HIVE DATA module combines 15 hard disk drives into one volume, and even if two hard drives fail, the data is still safe. The module also offers a fast data transfer rate of at least 3 gigabytes per second.
How does the HIVE CORE module support parallel, multi-user processing?
The HIVE CORE module is a multi-user processing platform running on Windows Server 2022. It includes everything you need for processing, including a CPU, RAM, NVMe, SSDs, and a 10-terabyte SSD RAID 5 system for data storage.
The HIVE CORE unit is tested with software from a range of vendors, so we can be sure that a user’s software will run on the system.
It can also be adapted according to user requirements, accommodating the number of people working in parallel in a facility, lab, or other scientific environment.
What types of advanced processing tasks is the HIVE GPU module optimized for?
Different software needs different hardware to run smoothly, so the HIVE CORE can be extended with the HIVE GPU module for GPU-accelerated processing. This module can accommodate up to four dual-slot GPUs in one box or up to eight single-slot GPUs.
At the moment, we are using RTX Ada Generation graphics cards because the HIVE GPU module is designed to provide the GPU-accelerated processing power needed for applications like 3D denoising, deconvolution, or AI applications.
How does the HIVE Dashboard enhance user and system management?
Our whole team is focused on ensuring that the HIVE system is as easy to use as possible. The HIVE Dashboard is designed to remotely monitor the health of the HIVE system.
For example, if a hard disk drive breaks down, the HIVE Dashboard app sounds an alarm and provides a visual cue, along with information on the faulty hardware needing to be replaced. The Dashboard can also monitor LAN health, instrument temperature, network stability, and UPS load.
It is important to mention that users do not need to be logged into the HIVE Dashboard to receive these alerts, because the HIVE also communicates with users by email. For example, an administrator will get an email notification if something happens that requires an alert to be issued.
Another important feature of the HIVE Dashboard is that it indicates if there are new system updates. From within the system, you can also contact Bruker for service and support.
User management in the HIVE Dashboard allows users to be assigned to the HIVE, with user credentials and passwords set from within the system. The new user will then get an email notification with information on how to log on to the system and how to change the password. Users can also be assigned different permissions, for example, with access to the Remote Desktop Protocol in order to access the HIVE via the internet.
Users can be set up as project leaders, and it is also possible to time limit user access, for example, where a user is collaborating on a project for a limited time only or within specific working hours. Project data is stored in shared folders, and we can add specific users to specific folders on a permanent or time-limited basis.
All of these capabilities are combined in one software tool, so there is no need to use separate systems for health monitoring, user management, and project management.
What makes the HIVE system future-proof and scalable for evolving lab needs?
The Acquifer HIVE is an efficient system with excellent data safety capabilities. Maintaining the central storage of image data in scientific imaging environments avoids the unnecessary duplication of data while offering users direct access to the data via the HIVE CORE computing nodes.
The HIVE is an extremely future-proof investment—the modular system grows in line with the needs of the lab. Users start with a small HIVE and build up to a powerful workhorse system with a lot of storage. Each module comes in its own housing, which can be stacked on top of one another and connected via easy plug-and-play extensions.
The HIVE uses the Windows Server 2022 operating system, which means it is compatible with virtually all microscopy and microimaging processing software.
The system supports a server grade file system able to accommodate multiple users and applications. For example, the HIVE can support virtualization with Hyper-V, or it can run an Omero server on the system. HIVE CORE units can also be provided with Linux operating systems.
The HIVE is an independent and flexible solution that provides a fast, powerful standalone network that is independent of network traffic in the wider institution or organization. Its firewall, battery pack, and router allow users to protect the HIVE from access from the outside, or it can be fully integrated into existing IT infrastructures.
Setup is provided by Bruker’s trained technicians, who will install and integrate the system, connect any microscopes or other devices, and check that everything works smoothly before users begin working with the system.
The HIVE system is useful for any type of user, from the core facility lead or the single microscope user to, for example, staff working with detailed scientific data computing.
Watch the full webinar
About the Speakers
Dr. Bettina Trauernicht is a Product Sales Specialist at Bruker with a strong focus on high-throughput imaging and handling big image data from microscopy. Dr. Trauernicht finished her Ph.D. at the Karlsruhe Institute of Technology / Germany where she first got in contact with image data-intensive projects in life science. Prior to joining Bruker, she was involved in the sales of the ACQUIFER Imaging Machine and HIVE, now integrated into the Bruker Fluorescence Microscopy portfolio.
Lord Hess is the head of Research and Development for Bruker´s ACQUIFER HIVE. His years of successful work at the “CERN – DataGrid Project” / Switzerland have laid the foundation for the ACQUIFER HIVE to become one of the most popular solutions for large amounts of data in the life sciences. Lord Hess brings his expertise into the Data Management segment to Bruker
This information has been sourced, reviewed and adapted from materials provided by Bruker Nano Surfaces and Metrology.
For more information on this source, please visit Bruker Nano Surfaces and Metrology.
Disclaimer: The views expressed here are those of the interviewee and do not necessarily represent the views of AZoM.com Limited (T/A) AZoNetwork, the owner and operator of this website. This disclaimer forms part of the Terms and Conditions of use of this website.