Authors: Joshua Michael Lojzim & Marcus Fries
Institution: Eastern Nazarene College 23 East Elm Ave.Quincy, MA 02170
Date: August 2017
doi:10.22186/jyi.33.3.55-62
Abstract
Medical imaging is key for the successful diagnosis and treatment of brain tumors, but the initial detection of tumors is, by nature, difficult. Image segmentation, a technique often used to aid detection, is highly dependent on the resolution of the segmented image. Many common morphological segmentation methods often suffer from a lack of resolution which hinders tumor detection. Thus, in this paper, two tumor segmentation techniques are developed and compared using MATLAB – one based on morphological processing, and a second which combines the discrete wavelet transform with morphological processing. Both proposed approaches begin with skull stripping via binary erosion, followed by image contrast enhancement and histogram thresholding. In the wavelet-based technique, the key step is to perform a fourth level discrete wavelet decomposition followed by manipulations of the wavelet. The resulting image is then morphologically opened, contrast enhanced, and gray thresholded. Both approaches were successfully tested on several magnetic resonance images, and it was shown that the wavelet transform method generally produces higher resolution segmented images. Additionally, it was found that the choice of wavelet basis function used plays a key role in the resolution of segmentation, with the Symlet 20 wavelet basis able to segment out almost 18% more pixels on average from an MR image than the Haar wavelet basis. These results can serve as a useful future reference as they provide convincing evidence of the necessity for careful choice of wavelet basis and suggest a basis that seems well suited for this application.
Introduction
Within Magnetic Resonance (MR) image processing, one major problem is the segmentation of brain tumors. Segmentation is the process of partitioning an image into several distinct sections to simplify the image or to focus on a region for further study (Kaur, G., & Rani, 2016). In the last several decades, there has been a push for automating the process of segmentation of pathological regions in the brain in MR scans. Current research on brain tumor segmentation uses a wide variety of methods which can be grouped as “intelligent” or “non-intelligent.” “Intelligent” techniques include machine learning methods such as neural-networks and support vector machine. The focus of this investigation, however, was on non-intelligent techniques, which include local and global thresholding, histogram operations, and morphology (Kaur & Banga, 2013).
Each of these methods has certain deficiencies. For example, global thresholding is not localized enough to produce segmentations with the desired resolution, but local thresholding is often not robust enough because of variations in how tumors present on MR images. Statistical models and morphological processing can also struggle to differentiate tissues in complex cases (Kaus et al., 1999). Wavelet-based techniques have become more popular in recent years because of the robustness they provide, and many hybrid wavelet methods are present in the literature (e.g., Sawakare & Chaudhari, 2014). When the correct method is found for a specific application, image segmentation can be a highly effective and important process because it allows clinicians to better understand the nature of a tumor and to plan more effective treatments (Xuan & Liao, 2007). Non-intelligent segmentation techniques are particularly important because they are often used for refining and improving intelligent segmentation methods (Kaur & Banga, 2013). In this research, a non-intelligent segmentation method, different from but inspired by those that are currently in the literature (Saini & Singh, 2015), was created based on morphology. Additionally, enhancements to this technique were developed using the discrete wavelet transform. These improvements were tested using several different wavelet basis functions. The major result of this investigation shows the key role that the choice of wavelet basis function plays in the resolution of the segmented image, a notion generally not discussed in the literature.
Morphology
The first of the two main mathematical concepts underlying this research is morphological image processing. Morphology consists of several operations that can be performed on a given image, represented by an image matrix, using other matrices called structuring elements (SE’s) in order to alter the image in desirable ways. Morphological processing is driven by operations performed by the SE’s on the image matrix, which use ones and zeroes to perform geometrical transformations based on the distributions of said ones and zeroes. The morphological operation used for segmentation in this investigation is called morphological opening. Morphological opening is made up of the successive operations of morphological erosion and dilation, which are both performed using the same SE.
Morphological erosion is defined in terms of set notation as follows. Given image A and structuring element B, both sets in Euclidean N-space, then the erosion of A by B, denoted A A B, “is the set of all elements x for which (x+b) ϵ A for every b ϵ B” (Sternberg, Haralick, & Zhuang, 1987):
Essentially, erosion shrinks the geometric features within an image based on the distribution of ones and zeroes within the SE. Morphological dilation, denoted A A B (where A and B are the same as above), is defined as (Sternberg et al., 1987):
Essentially, dilation inflates the geometric features within an image based on the nature of the SE.
The combination of these operations in succession, denotedA o B, is called morphological opening, and it is defined as (Sternberg et al., 1987):
Discrete Wavelet Transform
The second concept central to this investigation is the discrete wavelet transform. Traditionally, Fourier processing is used in most signal and image processing applications. Fourier bases are frequency localized however, meaning that small changes in space produce great changes in frequency and vice versa. As a result, Fourier based processing methods work very well for analyzing periodic signals, but abrupt changes are not easily detected through Fourier transform based analysis (Ingale, 2014). Due to these limitations, the decision was made to use the wavelet transform in support of morphological segmentation instead of the Fourier transform. The most important property of wavelets, the localization of their basis functions, sharply contrasts with the nature of Fourier basis functions; wavelet basis functions provide a degree of localization in both the space and frequency domains, meaning that small changes in space create small changes in frequency and vice versa.
Wavelet basis functions consist of a father wavelet (scaling function) notated φ(x), and a mother wavelet function, the first level of which is notated ψ(x). These mother wavelet functions can be scaled and shifted so that they cover the entire x-axis. These shifted, scaled functions are used to decompose a signal into its component parts. This decomposition, which allows for a more in-depth analysis of a particular region of the signal, is known as the discrete wavelet transform (DWT).
The DWT in one dimension is used to decompose and analyze signals such as an electrocardiogram or audio signal (a signal which can be represented by a single row or column vector). However, an image is not a one-dimensional signal like an EKG, but can instead be thought of as a two-dimensional signal (represented by a matrix instead of a column or row vector). Thus, for the computation of the DWT of an image, the first step is to extend the wavelet basis function to two dimensions, which is done using both the father wavelet φ(x) and the mother wavelet ψ(x). The wavelet function is extended to two dimensions in the following manner (Tolba, Mostafa, Gharib, & Megeed, 2001):
where the superscripts denote horizontal (h), vertical (v), and diagonal (d) basis functions. These new wavelet functions are used to define high and low pass decomposition and reconstruction filters (Lo-D and Hi-D in the above figure represent said decomposition filters), which are used to compute the DWT of the image by the convolution algorithm pictured in Figure 1 (“DWT2,” n.d.). This decomposition creates sub-images, the “CA,” “CD-horizontal,” “CD-vertical,” and “CD-diagonal,” in Figure 1. These sub-images each contain unique data about the original image; the approximation image, “CA,” contains a smaller, lower resolution version of the original image, and the detail (“CD”) images contain information about the vertical, horizontal, and diagonal aspects of the original image. Manipulation of these sub-images allows for effective analysis of important data, such as a tumor region, and has proved to be the powerful analysis tool that enhances morphological segmentation in this investigation.
Figure 1. Algorithm for DWT decomposition computation.
Methods
Step 1: Skull Stripping
For both segmentation methods, the first step is to remove the skull from the original MR image. This is an important step because in many MR images, the skull appears as one of the brightest regions of the image, and is sharply contrasted with other regions of the brain, such as grey matter. In the types of MR images used in this investigation, tumors also appear as bright regions, and thus to limit false positive segmentations from occurring, the skull must be removed from the image. To do this, the original Red, Green, and Blue (RGB) or Grayscale image was converted to a binary image. Next, the pixels that corresponded to the skull were eroded away (i.e. removed from the outside in, until the skull was eliminated). Then, the smaller, eroded binary image was used as a mask and placed over top of the original image, so that only the pixels in the original image that correspond to ones in the eroded binary image are kept as they are in the original image, and all other pixels (those of the skull) are set to 0 (eliminated or set to black). The result is a skull stripped version of the original MR image. This is shown in Figure 2 (Clark et al., 2013).
Figure 2. The results of performing skull stripping. A. Original brain MR image obtained from The Cancer Imaging Archive. B. The results of skull stripping the image a via the processes described in Methods: Step 1.
Step 2: Contrast Enhancement and Thresholding
The next step in the proposed segmentation method is to perform contrast enhancement and thresholding based on Otsu’s thresholding method (Otsu, 1979). These operations are performed as preliminary, global segmentation steps which help to illuminate the tumor region. The contrast enhancement method used is based on the shape of a specified curve. All the MR images used in this investigation are weighted such that pathological regions appear as brighter than other regions of the brain. Thus, to illuminate the tumor region and fade other regions, the general shape of the curve used made light regions lighter and dark regions darker. The results of contrast enhancement are shown in Figure 3.
Figure 3. The results of performing contrast enhancement on the skull-stripped image shown in Figure 2B.
Figure 4. The results of performing Otsu thresholding on the image shown in Figure 3.
After contrast enhancement is performed, the next step is to perform thresholding by Otsu’s method. The goal of thresholding is to decide which pixels in the image correspond to the foreground and which correspond to the background. Once the background and foreground pixels have been determined based on the distribution of pixel values within the image, the image is converted to binary, sending the background pixels to a value of 0 and the foreground pixels to a value of 1. The skull stripped image is then masked with this thresholded binary image to produce the final image for this step in the segmentation (Figure 4).
Step 3: The DWT
For the technique involving the discrete wavelet transform, the next step is the wavelet step (for the technique not involving the DWT, the next step is morphological opening, which will be discussed in Step 4). The first aspect of this step is the choice of wavelet basis function. Since different basis functions provide different degrees of localization in the space and frequency domains, they also allow for different degrees of resolution in segmentation. The four main bases tested in this investigation are shown in Figure 5.
Figure 5. Wavelet Basis Functions Tested. A-D from left to right, top to bottom. A. Haar Basis. B. Daubechies 2 Basis (‘db2’). C. Symlet 4 Basis (‘sym4’). D. Symlet 20 Basis (‘sym20’).
After much testing, it was found that the best wavelet basis for this application was the Symlet 20 wavelet, pictured in Figure 6 with the corresponding high and low pass filters. Comparisons between bases will be examined in the results section.
Figure 6. Father and mother wavelet functions, as well as the high and low pass decomposition and reconstruction filters, for the Symlet 20 wavelet.
Once the wavelet basis is chosen, the DWT can be performed on the image. The computation of the DWT in MATLAB decomposes the image into four outputs: the approximation image and the three detail images (“CA,” “CD-horizontal,” “CD-vertical,” and “CD-diagonal). Once the first DWT is computed, a second level DWT is computed using the first level approximation image as the input, and the small magnitude coefficients of the approximation matrix are set to 0. This pattern is repeated two more times for a total of four operations of the DWT.
Then, the detail sub-images are set to 0. The first reconstruction produces an image which is then used to reconstruct the second image and so on until the final image is the result of four reconstructive processes using the inverse DWT (IDWT) (four operations of the IDWT are needed for reconstruction to “undo” the four operations of the DWT used for deconstruction). The resultant image is then contrast enhanced and thresholded, for a second time, via the methods used in Step 2 above. The results of these operations are used as a mask which is overlaid on the original image (Figure 7).
Figure 7. The results of performing wavelet decomposition and reconstruction on the image pictured in Figure 4.
Step 4: Morphological Opening
In this step, the two approaches converge again. For the hybrid, wavelet-morphology based approach, the next step is to morphologically open the results of the wavelet processing step (Figure 7) to remove all regions that do not correspond to the tumor. For the morphological (non-wavelet) approach, the step is to morphologically open the skull-stripped, thresholded image (Figure 4). After much testing, the most suitable structuring element for this application was found to be a “disk” (i.e. a circle of 1’s with 0’s outside of the circle in the SE matrix). The radius of the SE used in this research was 14 pixels.
The results of morphologically opening the wavelet reconstructed image with the above structuring element are shown below in Figure 8a, and the results of morphologically opening the skull-stripped, thresholded image without the wavelet step are pictured in Figure 8b.
Figure 8. A. The results of morphologically opening the image shown in Figure 7. B. The results of morphologically opening the image shown in Figure 4.
Step 5: Final Contrast Enhancement and Thresholding
The final step is to perform another contrast enhancement and thresholding on the morphologically opened images created in the previous step. This makes it so that the only object left in the image after this step is the segmented tumor region (Figure 9). This region is shown in red (Figure 10) overlaid on the original MR image for qualitative comparison.
Figure 9. A. The results of contrast enhancing and thresholding the image shown in Figure 8A. B. The results of contrast enhancing and thresholding the image shown in Figure 8B.
Results
This section will present two sets of results: a comparison of the two segmentation methods developed in this paper (i.e. segmentation with and without the DWT) and a comparison of the resolutions produced when using the different wavelet bases which shown in Figure 5.
The comparisons made in this section show the difference in the ability of each technique to segment out data. They indicate the percent difference in how much data was segmented out of the image by each technique (they are shown in the form of percent difference instead of actual pixel numbers because the size of each tumor varies greatly from image to image, making raw pixel number comparisons meaningless).
Comparison Between Segmentation Methods
Morphological segmentation was successfully used as a baseline for tumor segmentation in this paper. With the addition of the wavelet transform step, the resolution of segmentation (i.e. the number of correct pixels that were partitioned from the image) fluctuated based on the choice of wavelet basis. The following tables shows comparisons between morphological segmentation and hybrid wavelet-morphological segmentation for each image tested and each of the four wavelet bases shown in Figure 5 above.
Comparison Between Wavelet Bases
From the comparisons made in Table 1, it is clear that certain wavelet bases perform better than others. Since the Symlet 20 basis consistently outperformed the other three in comparison to morphological processing, the following section serves to compare the resolution of the Symlet 20 basis with the Symlet 4, Daubechies 2, and Haar bases. The following table illustrates how the Symlet 20 wavelet basis compared with each of the wavelet bases used above for each of the images tested.
Table 1. Comparisons Between Techniques. The percent difference in resolution represents the number of pixels that were segmented using the given wavelet basis for the DWT compared with the number of pixels.
Discussion
As the data shows, the hybrid wavelet-morphological technique performs consistently better segmentation than just morphological segmentation when using the Symlet 4 and Symlet 20 wavelet bases, but is inconsistent when using the Daubechies 2 and Haar bases. Table 1 indicates that the Symlet 20 basis is the best suited basis tested, as the worst resolution that it produced was still about thirteen percent better than the morphological approach. Additionally, the Symlet 20 basis consistently outperformed each other wavelet basis, with an average of seventeen percent more data segmented when compared with the Haar basis and two and a half percent better resolution when compared with the Symlet 4 basis. This underscores the importance of the choice of wavelet basis function for tumor segmentation, a result not previously reported in the literature. Thus, with the right choice of wavelet basis, using a combination of wavelet processing and morphological techniques instead of just morphological processing can help to eliminate data loss in tumor segmentation. On average, the Symlet 20 wavelet basis could segment out about 20 percent more data than morphological processing alone.
Even though it is clear that the Symlet 20 basis is best suited for this technique, it is still uncertain as to exactly why this is the case. One thought, at least when it comes to comparing the Haar and Symlet 20 bases, is that the Haar basis is more ‘rigid’ and is less able to detect smaller changes in pixel characteristics than the Symlet 20 basis. This is an unproven idea, meaning that one main area of future work is to develop a more concrete theory as to why some wavelet bases perform so much better than others. An additional area of future work is to improve on the skull stripping method used in this technique. This skull stripping method is somewhat “primitive” and not highly adaptable, meaning that for tumors near the skull region, significant data loss is a distinct possibility. As a result, another highly important area of future work would be to develop a more robust, wavelet-based skull stripping method.
Conclusion
Brain tumor segmentation is an inherently difficult problem because of the widely varying nature of pixel distributions of pathological tissues in MR images. The segmentation methods and results presented in this paper are just two of many methods that have been developed over the past several decades with the goal of automating and improving brain tumor segmentation. The results underscore the effectiveness of the wavelet transform, as well as present novel findings about the impact of basis function choice on the resolution of the segmented image. Although many techniques exist that use the wavelet transform in some way, none that combine morphology and the DWT in the way presented in this paper have been reported in the literature. Additionally, there have never been clear results reported on the impact of the wavelet basis function choice on the results of segmentation. The results of the investigation in this paper can therefore serve as a useful guide for the development of future segmentation techniques involving the wavelet transform because they provide convincing evidence that the choice of wavelet basis can be vital to the resolution of a segmented image, as well as point to a basis with useful properties for this application.
Acknowledgments
The author would like to thank his mentors, Dr. Marcus Fries, Dr. Pierre-Richard Cornely, and Dr. Jill Macko for their invaluable ideas and support.
References
Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moorse, S., Phillips, S., Maffit, D., Pringle, M., Tarbox, L., & Prior, F. (2013). The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging, 26(6), 1045-1057. doi:10.1007/s10278-013-9622-7. DWT2. (n.d.). Retrieved from https://www.mathworks.com/help/wavelet/ref/dwt2.html. x
Ingale, R. (2014). Harmonic Analysis Using FFT and STFT. International Journal
of Signal Processing, Image Processing and Pattern Recognition, 7(4), 345-362. doi:10.14257/ijsip.2014.7.4.33
Kaur, G., & Rani, J. (2016). MRI Brain Tumor Segmentation Methods- A Review. International Journal of Current Engineering and Technology, 6(3). Retrieved from http://inpressco.com/category/ijcet/.
Kaur, M., & Banga, V. K. (2013). Thresholding and Level Set Based Brain Tumor Detection Using Bounding Box as Seed. International Journal of Engineering Research & Technology, 2(4). Retrieved from https://www.ijert.org.
Kaus, M. R., Warfield, S. K., Nabavi, A., Chatzidakis, E., Black, P. M., Jolesz, F. A., & Kikinis, R. (1999). Segmentation of Meningiomas and Low Grade Gliomas in MRI. In Medical Image Computing and Computer-Assisted Intervention – MICCAI’99: Second International Conference, Cambridge, UK, September 19-22, 1999 (Vol. 1679, Lecture Notes in Computer Science, pp. 1-10). Springer Berlin Heidelberg.
Otsu, N. (1979). A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62-66. doi:10.1109/tsmc.1979.4310076.
Saini, P., & Singh, M. (2015). Brain Tumor Detection in Medical Imaging Using MATLAB. International Research Journal of Engineering and Technology, 2, 2nd ser., 191-196. Retrieved from https://www.irjet.net.
Sawakare, S., & Chaudhari, D. (2014). Classification of Brain Tumor Using Discrete Wavelet Transform, Principal Component Analysis and Probabilistic Neural Network. International Journal for Research in Emerging Science and Technology, 1, 6th ser., 13-19. Retrieved from http://ijrest.net/.
Sternberg, S. R., Haralick, R. M., & Zhuang, X. (1987). Image Analysis Using Mathematical Morphology. IEEE Transactions on Pattern Analysis & Machine Intelligence, 9, 532-550. doi:10.1109/TPAMI.1987.4767941.
Tolba, M. F., Mostafa, M. G., Gharib, T. F., & Megeed, M. A. (2001). Medical Image Segmentation Using a Wavelet-Based Multiresolution EM Algorithm. IEEE International Conference on Industrial Electronics, Technology & Automation, IETA’2001, Cairo, 19th-21st Dec., 2001. Retrieved from http://www.academia.edu/25106125/Medical_Image_Segmentation_Using_a_Wavelet-Based_Multiresolution_EM_Algorithm.
Xuan, X., & Liao, Q. (2007). Statistical Structure Analysis in MRI Brain Tumor Segmentation. Fourth International Conference on Image and Graphics, 421-426. doi:10.1109/icig.2007.181.