Deep learning techniques are commonly utilized to tackle various computer vision problems, including recognition, segmentation, and classification from RGB images. With the availability of a diverse range of sensors, industry-specific datasets are acquired to address specific challenges. These collected datasets have varied modalities, indicating that the images possess distinct channel numbers and pixel values that have different interpretations. Implementing deep learning methods to attain optimal outcomes on such multimodal data is a complicated procedure. To enhance the performance of classification tasks in this scenario, one feasible approach is to employ a data fusion technique. Data fusion aims to use all the available information from all sensors and integrate them to obtain an optimal outcome. This paper investigates early fusion, intermediate fusion, and late fusion in deep learning models for bulky waste image classification. For training and evaluation of the models, a multimodal dataset is used. The dataset consists of RGB, hyperspectral Near Infrared (NIR), Thermography, and Terahertz images of bulky waste. The results of this work show that multimodal sensor fusion can enhance classification accuracy compared to a single-sensor approach for the used dataset. Hereby, late fusion performed the best with an accuracy of 0.921 compared to intermediate and early fusion, on our test data.
Waste from electronic equipment (WEEE) is a fast-growing complex waste stream, and plastics represent around 25% of its total. The proper recycling of plastics from WEEE depends on the identification of polymers prior to entering the recycling chain. Technologies aiming for this identification must be compatible with conveyor belt operations and fast data acquisition. Therefore, we selected three promising sensor types to investigate the potential of optical spectroscopy-based methods for identification of plastic constituents in WEEE. Reflectance information is obtained using Hyperspectral cameras (HSI) in the short-wave infrared (SWIR) and mid-wave infrared (MWIR). Raman point acquisitions are well-suited for specific plastic identification (532 nm excitation). Integration times varied according to the capabilities of each sensor, never exceeding 2 seconds. We have selected 23 polymers commonly found in WEEE (PE, PP, PVC ABS, PC, PS, PTFE, PMMA), recognising spectral fingerprints for each material according to literature reports. Spectral fingerprint identification was possible for 60% of the samples using SWIR-HSI; however, it failed to produce positive results for black plastics. Additional information from MWIR-HSI was used to identify two black samples (70% identified using SWIR + MWIR). Fingerprint assignment in shorttime Raman acquisition (1 -2 seconds) was successful for all samples. Combined with the efficient mapping capabilities of HSI at time scales of milliseconds, further developments promise great potential for fast-paced recycling environments. Furthermore, integrated solutions enable increased accuracy (cross-validations) and hence, we recommend a combination of at least 2 sensors (SWIR + Raman or MWIR + Raman) for recycling activities.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.