Lagrange Multipliers and Constrained Optimization

 1 In a Nut Shell:  Sometimes you wish to maximize or minimize a function subject to one or more limitations (called constraints).  For example, one may wish to find the maximum volume of an object subject to a restriction on its surface area or you may wish to maximize profit subject to a restriction on available investment opportunities.   The use of Lagrange multipliers offers a convenient approach.  Three cases will be discussed below. 2 Case 1  Suppose you wish to maximize/minimize a function with two independent variables,  f(x, y), subject to one constraint,  g(x, y) .   The functions involved are:                              f(x, y)  =  0     and    g(x, y)  =  0   Let  f(x,y) be the function to be optimized subject to the constraint relation, g(x,y). Both   f(x, y) and  g(x, y) must be continuously differentiable functions.   Then introduce an arbitrary constant,  λ , called the Lagrange multiplier such that:               Grad f   =  λ  Grad g      ------------------------------------ ( 1 )    where Grad g means gradient of g   So     ∂f/∂x  i   +  ∂f/∂y  j     =   λ  [∂g/∂x  i   +  ∂g/∂y  j ]   In scalar form:  ∂f/∂x   =   λ  [∂g/∂x ],  ∂f/∂y   =   λ  [ ∂g/∂y ],  and  g(x, y)  =    0   Which gives 3 equations in the 3 unknowns,   x,  y,  and  λ   Once solved, the values of  x  and  y  can be input to  f(x, y)  to obtain the optimum. 3 Case 2    A similar approach applies to functions with three independent variables subject to one constraint.  i.e. The result is 4 equations in the 4 unknowns,   x, y, z, and  λ   The functions involved are:        f(x, y, z)  =  0    subject to the constraint    g(x, y, z)  =  0                                    ∂f/∂x   =   λ  [∂g/∂x ],    ∂f/∂y   =   λ  [ ∂g/∂y ],                                    ∂f/∂z   =   λ  [ ∂g/∂z ],   and   g(x, y, z)  =    0