Computer vision in healthcare: how to bring superpowers to doctors

In the healthcare industry, people’s well-being and even their lives depend on how well doctors perform their tasks. Machines can now handle many such tasks better than humans: they never tire, nor do they feel the effects of pressure during even the most daunting procedures.

Advances in computer vision in healthcare, especially in object recognition, give doctors the superpowers of constant attention and tireless observation. For example, a machine’s error rate is just 3.5% compared to a 5% human rate. Simply put, computer vision object detection and recognition can do these tasks better.

The latest improvements in object recognition technology involve deep learning: machine learning algorithms using multi-layer neural networks. Through deep learning, object recognition accuracy on both open and private data sets has improved significantly.

Computer vision applications in the healthcare industry are based on adopting artificial intelligence and deep learning capabilities for medical imaging. Here are just a few:

• Detection of anomalies in MRI, CAT, and X-ray scans
• Skin anomalies detection, including cancer
• DICOM image analysis
• Diagnostic assistance
• Surgical assistance, e.g., preventing inadvertent retention of surgical instruments
• Blood analysis
• Retina scans that detect structural changes
• Telemedicine, remote monitoring, and patient care

Now that we can see the significant range of computer vision use cases in healthcare let’s take a deeper dive into how this tech is applied in specific areas.

What is computer vision in healthcare?

Computer vision is a branch of the umbrella term “artificial intelligence.” It deals with the analysis of still images or video streams, aiming to make conclusions followed by specific actions.

Highly complex mathematical algorithms (neural networks) of computer vision systems use collections of images or videos related to particular subject matter called datasets. The more of these images or videos a neural network analyses, the higher the accuracy it provides.

During the past decade, the sophistication and optimization of deep learning algorithms have resulted in improved average accuracy, from around 50% to 99%. Additionally, the rapidly increasing number of datasets, particularly digital images on the web, have played a significant role in training deep learning neural networks.

Computer vision in healthcare offers additional value in providing care services for consumers in medical settings as it can detect the very beginning of a disease. As new findings have stated, artificial intelligence not only enhances physicians’ speed but improves value-based care. A doctor does not need to spend much time on manual, time-consuming work. Most importantly, with the neural network acting as the doctor’s advisor, the error rate decreases as machines do not tire.

Computer vision use cases in healthcare

In medicine, the routine tasks associated with image analysis are numerous, and doctors spend significant time on them. Thus, the list of benefits of computer vision implementation is significant as well: the enhanced early disease recognition and increased accuracy via medical image analysis provided by computer vision solutions result in faster and more accurate diagnoses and more effective treatment.

As many diagnostic tools rely on the ability to discern minute details in visual data, the four tasks below illustrate the main ways computer vision implementation addresses medical needs:

• A lung disease abnormality (e.g., lung cancer) detected on X-ray images (via object detection).
• Cancerous skin moles’ detection (via object recognition).
• Early detection of polyps from colonoscopy videos (via semantic segmentation).
• Abdominal organ recognition from CT scans (via instance segmentation).

Computer vision technology in Radiology

Radiology, a pioneering branch of medicine, has adopted CV-powered medical tools to great effect. Since DICOM medical imaging data from various sources are essential for radiologists, precise object recognition and computer vision models analyzing medical images have become an invaluable practice. Hence, computer vision is valuable for many tasks, such as:

• Microscopic bone fracture detection
• Computer-aided inspection of CT, X-ray, and MRI scans
• Treatment results monitoring and change detection
• New tissue formations, including tumors, early stage detection

For example, Stanford ML Group’s CheXNet can detect pneumonia from chest X-rays with an accuracy exceeding that of practicing radiologists.

The most expensive and lengthy project, IBM Watson for Oncology, remains in the wings, preparing to apply six years of development and $5B in training data to the medical field.

So far, while human medical professionals will always have the final decision, as full diagnostic automation is not yet possible, the combined effort of a robust deep learning algorithm and an experienced clinician with a trained human vision can yield great results.

AI-enhanced medical imaging in Orthopedics

Another medical area that frequently utilizes computer vision technology is orthopedics. A vital innovation is transforming two-dimensional pictures into a three-dimensional model. Thus, with the help of a 3D module comparing a pre-operative CT scan to intraoperative fluoroscopy, doctors can assess the accurate location and positioning of an implant and bones.
Machines cover a wide range of activities, including pre-operative, intraoperative, and postoperative processes, such as:

• Surgery planning
• Intraoperative navigation
• Integration with robotic surgery planning software
• MRI-based arthroplasty planning
• CT, X-ray, MRI segmentation
• 2D/3D landmark detection

For example, a computer vision system can use images from a live camera to aid in real-time analysis during an operation, enabling the surgeon to identify where to drill a canal or place the implants. The machine can also determine an operation’s stages, providing doctors with suggestions on the tools to use next.

Medical image analysis in Cardiology

Computer vision techniques are involved in CT and MRT image analysis in cardiology, utilizing image segmentation. This tool divides a picture into parts defined as areas that look alike in specific selected properties, such as gray level, color, or texture, thereby facilitating the further processing of medical image interpretation. Hence, segmentation usually becomes the first step of many image recognition and analysis algorithms.
Computer vision systems aid cardiologists and other medical staff in various tasks, such as:

• Monitoring the progression of congenital heart disease
• Visualizing arteries and blood flow during surgeries
• Quantitative analysis of cardiac variables
• Blood loss estimation and prediction during surgeries

Computer vision in Ophthalmology

Healthcare computer vision projects can allay clinical workloads by providing even physicians with minimal experience with the ability to analyze medical images and to detect diseases efficiently and objectively. In ophthalmology, computer vision has gained increasing interest because it can be used in detecting clinically significant features for diagnostic and prognostic purposes, such as:

• Retina scans and subsequent analysis
• Early-stage detection of eye development anomalies
• Pre-operative planning
• Eye-tracking for eyesight correction

Computer vision technology in Dermatology

Healthcare computer vision turned out to be a natural choice for skin diagnostics and treatment, especially skin cancer. Healthcare professionals can quickly and accurately diagnose diseases using the output of a deep learning model. By analyzing a series of photo/video observations of a particular skin formation, an algorithm trained on thousands of cases of confirmed benign formations and cancers can highlight potentially harmful spots on the skin image.

For example, dermatologists and researchers from MIT created a system capable of identifying suspicious pigmented lesions (SPLs) with over 90% sensitivity. Additionally, researchers have used wide-field images made with smartphones as a basis for the classification of cancer.

Additional use cases of computer vision in dermatology include:

• High-resolution scanning of skin formations to monitor any growth
• Detecting early stages signs of skin disease
• Provides key information for developing personalized skincare treatment plans based on skin type, sensitivity, and reactions to different drugs.

Computer vision pattern recognition in Neonatology

Computer vision contributes to academic medical research and the improvement of medical diagnostic methods via machine vision. There are all kinds of key signs that may go unnoticed but, if detected, could support healthcare professionals in their day-to-day work.

The experts can use computer vision systems: high-resolution video with long-form recordings for AI development. For instance, using the scientifically proven connection between sleeping-generated movement patterns and those exhibited during awake time, artificial intelligence can detect a child’s movement patterns and alert a doctor, who can make a formal assessment. To this end, Neolook Solutions has created Screen2Screen Academic Extend, an AI to assess pattern recognition algorithms. The computer vision solution can monitor the child’s movement patterns and detect whether the child is at risk.

Autonomous system for telemedicine

The coronavirus pandemic became not only a crisis but also a strong impulse for telemedicine development. Computer vision recognition systems helped carry out primary diagnoses by video or images and provided remote patient care advice without needing an in-person doctor visit.

For example, diabetic retinopathy (DR) and foot ulcers (DFU) can be already diagnosed through medical imaging analysis performed by machines. An Intelligent Diabetic Assistant makes diagnoses and prioritizes treatment after analyzing observations from the screen with an 88.5 accuracy score, which is considered a proper result.

Particularly for patients in isolated rural areas, computer vision machine learning solutions have become the way to replace a first visit to well-trained clinicians who, as usual, would conduct follow-up examinations of patients for more complex cases.

Computer vision for medical labs

Computer vision in healthcare isn’t limited to diagnostics via image recognition, even though it undoubtedly looks the most popular application.
Recent high demand for tests has pushed laboratories to their limit. During times of wide-spread illness, such as flu season or even pandemics, people could wait for a long time to receive results.
Analysis automated by clinical laboratory machines allows for rapid test processing while reducing the human error rate.

First and foremost, computer vision technologies evaluate blood samples by a taken picture to identify aberrations. Other computer vision use cases in medical laboratories include analyzing the levels of separated phases of blood and the quality of centrifuged blood samples ranked and classified by machines.

Conclusion

The healthcare industry is both a pioneer and a beneficiary of the rapid advancements in computer vision technologies. Computer vision in healthcare has improved multiple medical disciplines, saving thousands of lives due to diagnostic improvement, earlier health issues detection, and better treatment plans.

Both medical professionals and their patients have benefitted from computer vision systems in healthcare. On the clinicians’ side, computer vision helps reduce the number of diagnostic errors. Additionally, it can decrease false negatives by detecting the slightest anomalies and deviations that doctors can overlook during manual observations.

Furthermore, computer vision can be a priceless asset for surgical medical staff, helping them prepare for operations, keep track of surgical instruments (by detecting objects in the video), and even train younger surgeons.

For patients, the numerous types of computer vision applications in medicine provide access to self-service kiosks, faster admissions, remote health monitoring scenarios, and much more.

Most importantly, computer vision in healthcare helps save lives and makes treatments less aggressive, traumatic, and expensive.