<P> In mathematical optimization, statistics, econometrics, decision theory, machine learning and computational neuroscience, a loss function or cost function is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost" associated with the event . An optimization problem seeks to minimize a loss function . An objective function is either a loss function or its negative (in specific domains, variously called a reward function, a profit function, a utility function, a fitness function, etc .), in which case it is to be maximized . </P> <P> In statistics, typically a loss function is used for parameter estimation, and the event in question is some function of the difference between estimated and true values for an instance of data . The concept, as old as Laplace, was reintroduced in statistics by Abraham Wald in the middle of the 20th century . In the context of economics, for example, this is usually economic cost or regret . In classification, it is the penalty for an incorrect classification of an example . In actuarial science, it is used in an insurance context to model benefits paid over premiums, particularly since the works of Harald Cramér in the 1920s . In optimal control the loss is the penalty for failing to achieve a desired value . In financial risk management the function is mapped to a monetary loss . </P> <P> Parameter estimation for supervised learning tasks such as regression or classification can be formulated as the minimization of a loss function over a training set . The goal of estimation is to find a function that models its input well: if it were applied to the training set, it should predict the values (or class labels) associated with the samples in that set . The loss function quantifies the amount by which the prediction deviates from the actual values . </P> <P> Formally, we begin by considering some family of distributions for a random variable X, that is indexed by some θ . </P>

The tendency to over or under estimate the value of a reward is called