Communications in Information and Systems

Volume 18 (2018)

Number 1

An investigation for loss functions widely used in machine learning

Pages: 37 – 52

DOI: https://dx.doi.org/10.4310/CIS.2018.v18.n1.a2

Authors

Feiping Nie (School of Computer Science and Center for Optical Imagery Analysis and Learning, Northwestern Polytechnical University, Xi’an, China)

Zhanxuan Hu (School of Computer Science and Center for Optical Imagery Analysis and Learning, Northwestern Polytechnical University, Xi’an, China)

Xuelong Li (Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, China)

Abstract

Over past few decades, numerous machine learning algorithms have been developed for solving various problems arising in practical applications. And, loss function is one of the most significant factors influencing the performance of algorithm. Nevertheless, most readers may be confused about the reason why these loss functions are effective in corresponding models. The confusion further interfere them to select reasonable loss functions for their algorithms. In this paper, we take a comprehensive investigation for some representative loss functions and analyse the latent properties of them. One of the goals of the investigation is to find the reason why bilateral loss functions are more suitable for regression task, while unilateral loss functions are more suitable for classification task. In addition, a significant question we discuss is that how to judge the robustness of a loss function. The investigation is useful for readers to develop or improve their future works.

Published 7 June 2018