tf.contrib.constrained_optimization.ConstrainedMinimizationProblem

View source on GitHub

Abstract class representing a ConstrainedMinimizationProblem.

A ConstrainedMinimizationProblem consists of an objective function to minimize, and a set of constraint functions that are constrained to be nonpositive.

In addition to the constraint functions, there may (optionally) be proxy constraint functions: a ConstrainedOptimizer will attempt to penalize these proxy constraint functions so as to satisfy the (non-proxy) constraints. Proxy constraints could be used if the constraints functions are difficult or impossible to optimize (e.g. if they're piecewise constant), in which case the proxy constraints should be some approximation of the original constraints that is well-enough behaved to permit successful optimization.

constraints Returns the vector of constraint functions.

Letting g_i be the ith element of the constraints vector, the ith constraint will be g_i <= 0.

num_constraints Returns the number of constraints.
objective Returns the objective function.
pre_train_ops Returns a list of Operations to run before the train_op.

When a ConstrainedOptimizer creates a train_op (in minimize minimize_unconstrained, or minimize_constrained), it will include these ops before the main training step.

proxy_constraints Returns the optional vector of proxy constraint functions.

The difference between constraints and proxy_constraints is that, when proxy constraints are present, the constraints are merely EVALUATED during optimization, whereas the proxy_constraints are DIFFERENTIATED. If there are no proxy constraints, then the constraints are both evaluated and differentiated.

For example, if we want to impose constraints on step functions, then we could use these functions for constraints. However, because a step function has zero gradient almost everywhere, we can't differentiate these functions, so we would take proxy_constraints to be some differentiable approximation of constraints.