Improving Reliability in N-version Machine Learning Systems: A Theoretical Model

Monday - February 5 2024, 07:59 UTC - 9 months ago

Researchers at University of Tsukuba created a theoretical model to evaluate the impact of diversity in machine learning models and input data on the reliability of system outputs. This can aid in exploring optimum system configurations. The study also highlights practical challenges and future research directions for improving reliability while reducing cost and overhead.

Reliable and safe output is crucial for machine learning systems used in applications such as autonomous driving and medical imaging. To achieve this, researchers at University of Tsukuba have developed a theoretical model for evaluating the effect of diversity in machine learning models and input data on the reliability of the system's output. This model can help in exploring appropriate system configurations, and the study has been published in IEEE Transactions on Emerging Topics in Computing.

One of the system designs for improving reliability is the N-version machine learning system. In this system, multiple machine learning models and input data are combined so that inference errors in one model do not directly affect the final output. However, while empirical evidence shows that diversity in models and data can improve reliability, a theoretical model to explain this has not yet been developed.

In this study, the researchers introduced diversity metrics for machine learning models and input data in relation to inference errors. They then constructed a theoretical model to evaluate the reliability of the system's output. The results showed that system configurations that utilize the diversity of models and data are the most stable in improving reliability in generally assumed situations.

But practical system design also has its challenges, such as the overhead and cost of performing multiple inference processes. To address this, the researchers are continuing to investigate and develop methods to achieve high reliability in N-version machine learning systems while reducing cost, power consumption, and overhead from both theoretical and experimental perspectives.