Sponsored Content by e-con SystemsJan 26 2023Reviewed by Louis Castel
IVD tests are a crucial component of the medical diagnosis and treatment processes. They entail examining and testing bodily fluid samples like blood, mucus, or saliva in an effort to both diagnose and treat current diseases and also gauge the likelihood of developing new ones in the future.
Image Credit: e-con Systems
In-vitro, which means “in-glass,” refers to the fact that tests are conducted using glass instruments, such as test tubes and glass slides. Excellent examples of IVD analyses are the procedures used to diagnose COVID-19.
The automation and speeding up of in-vitro diagnostics have been significantly facilitated by cameras. There are numerous camera-enabled IVD devices used in clinical and medical testing, such as digital microscopes and spectrophotometers.
In-vitro diagnostic procedures could involve intermediate steps in addition to the actual analysis, where a camera can be used to decrease human labor and speed up diagnosis.
An illustration of this is test tube classification, in which the type of sample present inside a test tube is determined by a camera’s determination of the color of the cap. Other examples include barcode decoding on trays or racks and liquid-level detection.
This article takes a detailed look at the edge AI camera solution from e-con Systems for the test tube classification use case and explains why it is ideal for AI-based in-vitro diagnostic applications.
e-CAM512 USB-5MP edge AI camera for in-vitro diagnostics
The e-CAM512 USB, a 5MP smart USB camera based on the AR0521 sensor from Onsemi, is the device that E-con has specifically developed for AI-based pre-analytic tasks in in-vitro diagnostics.
This product will soon be put to use in a real-world test tube classification demonstration. The salient characteristics of this cutting-edge AI camera from e-con Systems are described below.
The e-CAM512_USB is a USB 2.0 UVC-compliant camera that can stream VGA at 30 frames per second (fps) and 960p at 10 fps in YUV format. It can stream in 960 p at 20 fps and VGA at 60 fps in the grayscale Y8 format. It has an M12 lens holder, which means that users can switch lenses to suit their needs for the final application requirements.
A highly optimized camera solution, this AI camera has an onboard image signal processor that is used for delayering, color correction, tonal balancing, noise reduction, and autoexposure and auto white balance algorithms.
Brightness, contrast, sharpness, and gamma are some of its image quality controls that can be used to post-process the video output. Users can manually adjust exposure, white balance, and sensor gain as necessary.
Furthermore, this smart camera is equipped with an NXP i.MX RT1170 processor, which gives it real edge AI capabilities. The EdgeVerse™ edge computing platforms’ i.MX RT1170 crossover MCUs, which operate at 1 GHz, are breaking speed records. The Arm Cortex-M4 core operates at 400 MHz, and the Arm Cortex-M7 core operates at 1 GHz in the dual-core processor.
The i.MX RT1170 also provides support over a broad temperature range. Additionally, for AI, the e-CAM512_USB supports the deployment of neural networks using the Tensor Flow lite micro, DeepViewRT, and Glow inference engines.
Product developers can directly capture images for data sets with the help of e-con’s machine learning infrastructure and test and benchmark the trained models on the camera. As a result, all pre-analytic tasks in in-vitro diagnostic devices can be easily handled by e-CAM512_USB.
Seeing machine learning models in action on e-CAM512_USB
This next section focuses on benchmarking data and how some common machine learning models perform on the e-CAM512_USB.
The inference time for the Mobilenet V1 model with a 3232-input image size and an alpha of 0.5 is approximately 18.5 milliseconds in int8 and 97.4 milliseconds in float32. This model can be used to perform many simple detection tasks, such as verifying the existence of an object in an image.
In contrast, Mobilenet V2, a more improvised model than V1, has an inference time of roughly 22.9 milliseconds in int8 and 89.1 milliseconds in float32 with an input image size of 3232 and alpha of 0.5.
With an input image size of 3232 and an alpha of 0.5, the most recent Mobilenet model V3-small has an inference time of approximately 22 milliseconds in int8 and 60 milliseconds in float32. The inference time for models with larger input sizes, such as 320×320 and an alpha of 1, is approximately 0.9 seconds.
The Mobilenet V3-large model is the next to be discussed. The inference time is approximately 36.9 milliseconds in int8 and 112 milliseconds in float32 with a 3232-input image and an alpha of 0.5. The inference time is roughly 2.7 seconds for models with a larger input size, like 320×320 and an alpha of 1.
Demonstration of test tube classification using e-CAM512_USB
Now that the features of e-CAM512_USB and the machine learning models used on it have been outlined, it is important to see how the product uses these features for a test tube classification process:
Blood Collection Tubes Classification Demo using Edge AI Smart camera | e-con Systems
Video Credit: e-con Systems
As seen in the video, the use case entails the detection of the color of the vial caps, which aids in the identification of the sample contained within the vials.
The e-CAM512_USB camera was positioned approximately 50 cm away from the vials for this demonstration. The initial data set for training was taken with the same camera. The camera was then used to automatically identify the color of the vial caps using the trained model.
Images were taken in various lighting conditions and vial positions to train the model. A total of 1500 images were taken, enhanced, and used as training data. The model can predict and display colors in real-time, as can be seen in the video.
The ML model at work here is a constrained object detection model that is based on the centroid of the object. Using the camera, high-resolution images with a size of 720 × 720 pixels were taken and resized in the PXP block of the RT1170 processor.
Additionally, the PXP converts the YUV to RGB format before feeding it to the inference engine. A 96 × 96 RGB image is used as the model’s input.
For this, a custom object detection model was initially trained based on Mobilenet V2 with a single shot detector, and the inferencing took about 2.4 seconds. The average inference time was reduced to 32 milliseconds using the constrained object detection model. This is merely one application for which e-CAM512_USB can be used.
Tools and accelerators offered by e-con Systems to run the inference engines
A mode for configuring the camera at the resolution needed to train the actual model is included with the e-CAM512_USB. This implies that the camera itself can be used for data collection. Users can utilize this camera for their AI-based use case at any stage of development, whether it be data collection or if they already have a pre-trained model.
After data collection, the model can be trained and optimized using NXP’s eiQ toolkit (or any other toolkit), resulting in a Tensor flow lite micro model. Then using e-CAM512_USB, product developers can easily benchmark and test their models with e-con's easy-to-use tools that come with the camera. The tool for benchmarking enables users to get a true sense of their model’s performance without having to test it in a real-world scenario.
After benchmarking and optimizing the Tensor flow lite micro model, users can then use e-con’s model loading software to run the models and get real-time inferencing information. e-con also provides detailed information on how to use these tools and program the camera. It will also be extending full support to all customers during the development and integration phase.
About e-con Systems
e-con Systems™ is a leading OEM camera manufacturer with 18+ years of experience and expertise in embedded vision. It focuses on delivering vision and camera solutions to industries such as medical, retail, and industrial. The company’s wide portfolio of products includes MIPI camera modules, USB 3.1 Gen 1 cameras, GMSL cameras, stereo cameras, etc. It has built over 250+ product-based solutions and shipped millions of cameras around the globe. What sets the company apart is its deep expertise in building customized product designs while ensuring rapid prototyping and custom modifications in hardware and software. e-con Systems™ has close partnerships with Sony, ON Semiconductor®, Omnivision, NVIDIA, Xilinx, Socionext™, Cypress, Connect Tech, Toradex, Variscite, Toshiba, Diamond Systems, Avermedia and Texas Instruments.
Giving sight to medical applications:
e-con Systems™ offers end-to-end camera solutions to meet the needs of the medical and healthcare industry. It has a strong foothold in the medical device industry – having empowered clients to integrate unique camera solutions for medical and life science applications in ophthalmology, dentistry, dermatology, laboratory equipment, microscopy, assistive technology, point-of-care technology etc. e-con Systems also has a solid track record of working with the medical device industry leaders such as Thermofisher, Idexx, Perkin Elmer, Amwell, Welch Allyn, Seegene and others.”
Sponsored Content Policy: News-Medical.net publishes articles and related content that may be derived from sources where we have existing commercial relationships, provided such content adds value to the core editorial ethos of News-Medical.Net which is to educate and inform site visitors interested in medical research, science, medical devices and treatments.