Ethical data splitting utilizing L1-norm principal component analysis

Garrett I. Cayce; Arthur C Depoian II; Colleen P. Bailey; Parthasarathy Guturu

doi:10.1117/12.2664127

13 June 2023 Ethical data splitting utilizing L1-norm principal component analysis

Garrett I. Cayce, Arthur C Depoian II, Colleen P. Bailey, Parthasarathy Guturu

Proceedings Volume 12522, Big Data V: Learning, Analytics, and Applications ; 1252208 (2023) https://doi.org/10.1117/12.2664127
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Abstract

Ethical data splitting is of paramount importance to ensure the validity of any solution that is based on data. If data is biased, it will not accurately represent how the solution will solve the problem. To ethically split data, the overall variance of the data needs to be fairly represented in the training and the testing sets of the dataset. To do this, the outliers of the data need to be determined so that they can be accounted for when splitting the data. Finding the principal components of the data using the L2-norm has been shown as an effective way to identify outliers of data to make a robust dataset that is resistant to outliers. It has been shown that the L1-norm is more resistant to outliers than the L2-norm, so it will allow the dataset to become more resistant to outliers. Therefore, utilizing L1-norm principal components when determining ethical data splits will result in more robust datasets.

Conference Presentation

Citation Download Citation

Garrett I. Cayce, Arthur C Depoian II, Colleen P. Bailey, and Parthasarathy Guturu "Ethical data splitting utilizing L1-norm principal component analysis", Proc. SPIE 12522, Big Data V: Learning, Analytics, and Applications , 1252208 (13 June 2023); https://doi.org/10.1117/12.2664127

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available