Welcome to CVXPY¶

Join the CVXPY mailing list for the best CVXPY support!

CVXPY is a Python-embedded modeling language for convex optimization problems. It allows you to express your problem in a natural way that follows the math, rather than in the restrictive standard form required by solvers.

For example, the following code solves a least-squares problem where the variable is constrained by lower and upper bounds:

from cvxpy import *
import numpy

# Problem data.
m = 30
n = 20
numpy.random.seed(1)
A = numpy.random.randn(m, n)
b = numpy.random.randn(m)

# Construct the problem.
x = Variable(n)
objective = Minimize(sum_squares(A*x - b))
constraints = [0 <= x, x <= 1]
prob = Problem(objective, constraints)

# The optimal objective is returned by prob.solve().
result = prob.solve()
# The optimal value for x is stored in x.value.
print x.value
# The optimal Lagrange multiplier for a constraint
# is stored in constraint.dual_value.
print constraints[0].dual_value

This short script is a basic example of what CVXPY can do. CVXPY also supports simple ways to solve problems in parallel, higher-level abstractions such as object oriented convex optimization, and extensions for non-convex optimization.

CVXPY was designed and implemented by Steven Diamond, with input from Stephen Boyd and Eric Chu.

CVXPY was inspired by the MATLAB package CVX. See the book Convex Optimization by Boyd and Vandenberghe for general background on convex optimization.

CVXPY relies on the open source solvers ECOS, CVXOPT, and SCS.

Install Guide¶

Mac OS X¶

Install the Command Line Tools for Xcode.

Download from the Apple developer site.
Install Anaconda.

Follow the instructions on the website.
Create a new conda environment for cvxpy. We’ll call it cvxpy_env in these instructions.
```
conda create -n cvxpy_env python=2 scipy numpy pip nose
```
Activate the new conda environment.
```
source activate cvxpy_env
```

In the future, make sure you have the cvxpy_env environment activated whenever you use cvxpy. You know the environment is activated if you see (cvxpy_env) on the left hand side of your terminal prompt.

Install cvxpy with pip from the command-line.
```
pip install cvxpy
```
Test the installation with nose.

nosetests cvxpy

Ubuntu 14.04¶

Make sure apt-get is up-to-date.

sudo apt-get update

Install ATLAS and gfortran (needed for scs).

sudo apt-get install libatlas-base-dev gfortran

Install python-dev.
```
sudo apt-get install python-dev
```
Install pip.
```
sudo apt-get install python-pip
```

Install numpy and scipy.

sudo apt-get install python-numpy python-scipy

Install cvxpy.
```
sudo pip install cvxpy
```

or to install locally
pip install --user cvxpy

Install nose.

sudo apt-get install python-nose

Test the installation with nose.

nosetests cvxpy

Windows¶

Here is a step-by-step guide to installing CVXPY on a Windows machine.

If you have Python installed already, it’s probably a good idea to remove it first. (Sorry!)
Download the latest version of Python(x,y).
Install Python(x,y). When prompted to select optional components, make sure to check cvxopt and MinGW, as shown below.

We need to set the default compiler as mingw32. Open Notepad and type the following, save the file at C:\Python27\Lib\distutils\distutils.cfg. (This is the default location. If you installed Python somewhere else, use the appropriate location.)

Open Python(x,y) and launch the interactive console (highlighted button in the picture). This will bring up a console.

From the console, run “pip install ecos” to install ecos.
We need to install BLAS and LAPACK libraries, and make the scs package use them. Go here to download the win32 version of the dll and lib files of both BLAS and LAPACK. Put them under some directory, say C:blaslapack, as shown below.

The system needs to know where to find the libraries. Right click on This PC (or My Computer), click Properties, Advanced system settings, then Environment Variables. Under the System variables list, find a variable named Path, and press Edit. Then, at the end of the list, put the address to the directory where you put the library files. All paths must be separated by semicolons.

Download the SCS package as a zip file and unzip it.
Browse to scs-master directory, and edit line 48 of the file scs.mk to “USE_LAPACK = 1”. Without this, scs won’t be able to solve SDPs.

Browse to the src directory, and open the file cones.c. Edit lines 11 and 13 to look like the following.

We have to change the numpy settings so that it knows where to find the libraries. Open C:\Python27\Lib\site-packages\numpy\distutils\site.cfg and add the following lines to the end of the file:

[blas]
library_dirs = C:\blaslapack
blas_libs = blas
[lapack]
library_dirs = C:\blaslapack
lapack_libs = lapack

You can remove what’s already in there, and replace the file with just the six lines above.

Go back to the Python(x,y) terminal, and browse to the python directory of scs-master. From there, type “python setup.py build” to build scs. (If this step results in some error, remove the build directory and try again.) After the build is successful, run “python setup.py install” to install.
After scs is installed, run “pip install cvxpy” to install CVXPY.
Reboot your computer so that the path environment variable we set in step 8 takes effect.
CVXPY should work now. You can use the Spyder IDE from the Python(x,y) home window. Click on the Spyder button to launch it. This IDE allows you to code, run, and view the console all in the same window. In order to check if the installation was successful, open a terminal, browse to C:\Python27\Lib\site-packages\cvxpy, and run “nosetests tests”. This runs all unit tests and reports any error found.

Other Platforms¶

The CVXPY installation process on other platforms is less automated and less well tested. Check this page for instructions for your platform.

Install from source¶

CVXPY has the following dependencies:

Python 2.7 or Python 3.4
setuptools >= 1.4
toolz
CVXOPT >= 1.1.6
ECOS >= 1.0.3
SCS >= 1.0.1
NumPy >= 1.8
SciPy >= 0.13

To test the CVXPY installation, you additionally need Nose.

CVXPY automatically installs ECOS, CVXOPT, SCS, and toolz. NumPy and SciPy will need to be installed manually. Once you’ve installed NumPy and SciPy, installing CVXPY from source is simple:

Clone the CVXPY git repository.
Navigate to the top-level of the cloned directory and run
```
python setup.py install
```

Install with GLPK support¶

CVXPY supports the GLPK solver, but only if CVXOPT is installed with GLPK bindings. To install CVXPY and its dependencies with GLPK support, follow these instructions:

Install GLPK. We recommend either installing the latest GLPK from source or using a package manager such as apt-get on Ubuntu and homebrew on OS X.

Install CVXOPT with GLPK bindings.

CVXOPT_BUILD_GLPK=1
CVXOPT_GLPK_LIB_DIR=/path/to/glpk-X.X/lib
CVXOPT_GLPK_INC_DIR=/path/to/glpk-X.X/include
pip install cvxopt

Follow the standard installation procedure to install CVXPY and its remaining dependencies.

Tutorial¶

What is CVXPY?¶

CVXPY is a Python-embedded modeling language for convex optimization problems. It automatically transforms the problem into standard form, calls a solver, and unpacks the results.

The code below solves a simple optimization problem in CVXPY:

from cvxpy import *

# Create two scalar optimization variables.
x = Variable()
y = Variable()

# Create two constraints.
constraints = [x + y == 1,
               x - y >= 1]

# Form objective.
obj = Minimize(square(x - y))

# Form and solve problem.
prob = Problem(obj, constraints)
prob.solve()  # Returns the optimal value.
print "status:", prob.status
print "optimal value", prob.value
print "optimal var", x.value, y.value

status: optimal
optimal value 0.999999989323
optimal var 0.999999998248 1.75244914951e-09

The status, which was assigned a value “optimal” by the solve method, tells us the problem was solved successfully. The optimal value (basically 1 here) is the minimum value of the objective over all choices of variables that satisfy the constraints. The last thing printed gives values of x and y (basically 1 and 0 respectively) that achieve the optimal objective.

prob.solve() returns the optimal value and updates prob.status, prob.value, and the value field of all the variables in the problem.

Namespace¶

The Python examples in this tutorial import CVXPY using the syntax from cvxpy import *. This is done to make the examples simpler and more concise. But for production code you should always import CVXPY as a namespace. For example, import cvxpy as cvx. Here’s the code from the previous section with CVXPY imported as a namespace.

import cvxpy as cvx

# Create two scalar optimization variables.
x = cvx.Variable()
y = cvx.Variable()

# Create two constraints.
constraints = [x + y == 1,
               x - y >= 1]

# Form objective.
obj = cvx.Minimize(cvx.square(x - y))

# Form and solve problem.
prob = cvx.Problem(obj, constraints)
prob.solve()  # Returns the optimal value.
print "status:", prob.status
print "optimal value", prob.value
print "optimal var", x.value, y.value

Nonetheless we have designed CVXPY so that using from cvxpy import * is generally safe for short scripts. The biggest catch is that the built-in max and min cannot be used on CVXPY expressions. Instead use the CVXPY functions max_elemwise, max_entries, min_elemwise, or min_entries.

The built-in sum can be used on lists of CVXPY expressions to add all the list elements together. Use the CVXPY function sum_entries to sum the entries of a single CVXPY matrix or vector expression.

Changing the problem¶

After you create a problem object, you can still modify the objective and constraints.

# Replace the objective.
prob.objective = Maximize(x + y)
print "optimal value", prob.solve()

# Replace the constraint (x + y == 1).
prob.constraints[0] = (x + y <= 3)
print "optimal value", prob.solve()

optimal value 1.0
optimal value 3.00000000006

Infeasible and unbounded problems¶

If a problem is infeasible or unbounded, the status field will be set to “infeasible” or “unbounded”, respectively. The value fields of the problem variables are not updated.

from cvxpy import *

x = Variable()

# An infeasible problem.
prob = Problem(Minimize(x), [x >= 1, x <= 0])
prob.solve()
print "status:", prob.status
print "optimal value", prob.value

# An unbounded problem.
prob = Problem(Minimize(x))
prob.solve()
print "status:", prob.status
print "optimal value", prob.value

status: infeasible
optimal value inf
status: unbounded
optimal value -inf

Notice that for a minimization problem the optimal value is inf if infeasible and -inf if unbounded. For maximization problems the opposite is true.

Vectors and matrices¶

Variables can be scalars, vectors, or matrices.

# A scalar variable.
a = Variable()

# Column vector variable of length 5.
x = Variable(5)

# Matrix variable with 4 rows and 7 columns.
A = Variable(4, 7)

You can use your numeric library of choice to construct matrix and vector constants. For instance, if x is a CVXPY Variable in the expression A*x + b, A and b could be Numpy ndarrays, SciPy sparse matrices, etc. A and b could even be different types.

Currently the following types may be used as constants:

Numpy ndarrays
Numpy matrices
SciPy sparse matrices

Here’s an example of a CVXPY problem with vectors and matrices:

# Solves a bounded least-squares problem.

from cvxpy import *
import numpy

# Problem data.
m = 10
n = 5
numpy.random.seed(1)
A = numpy.random.randn(m, n)
b = numpy.random.randn(m, 1)

# Construct the problem.
x = Variable(n)
objective = Minimize(sum_entries(square(A*x - b)))
constraints = [0 <= x, x <= 1]
prob = Problem(objective, constraints)

print "Optimal value", prob.solve()
print "Optimal var"
print x.value # A numpy matrix.

Optimal value 4.14133859146
Optimal var
[[ -2.76479783e-10]
 [  3.59742090e-10]
 [  1.34633378e-01]
 [  1.24978611e-01]
 [ -3.67846924e-11]]

Constraints¶

As shown in the example code, you can use ==, <=, and >= to construct constraints in CVXPY. Equality and inequality constraints are elementwise, whether they involve scalars, vectors, or matrices. For example, together the constraints 0 <= x and x <= 1 mean that every entry of x is between 0 and 1.

If you want matrix inequalities that represent semi-definite cone constraints, see Semidefinite matrices. The section explains how to express a semi-definite cone inequality.

You cannot construct inequalities with < and >. Strict inequalities don’t make sense in a real world setting. Also, you cannot chain constraints together, e.g., 0 <= x <= 1 or x == y == 2. The Python interpreter treats chained constraints in such a way that CVXPY cannot capture them. CVXPY will raise an exception if you write a chained constraint.

Parameters¶

Parameters are symbolic representations of constants. The purpose of parameters is to change the value of a constant in a problem without reconstructing the entire problem.

Parameters can be vectors or matrices, just like variables. When you create a parameter you have the option of specifying the sign of the parameter’s entries (positive, negative, or unknown). The sign is unknown by default. The sign is used in Disciplined Convex Programming. Parameters can be assigned a constant value any time after they are created. The constant value must have the same dimensions and sign as those specified when the parameter was created.

# Positive scalar parameter.
m = Parameter(sign="positive")

# Column vector parameter with unknown sign (by default).
c = Parameter(5)

# Matrix parameter with negative entries.
G = Parameter(4, 7, sign="negative")

# Assigns a constant value to G.
G.value = -numpy.ones((4, 7))

You can initialize a parameter with a value. The following code segments are equivalent:

# Create parameter, then assign value.
rho = Parameter(sign="positive")
rho.value = 2

# Initialize parameter with a value.
rho = Parameter(sign="positive", value=2)

Computing trade-off curves is a common use of parameters. The example below computes a trade-off curve for a LASSO problem.

from cvxpy import *
import numpy
import matplotlib.pyplot as plt

# Problem data.
n = 15
m = 10
numpy.random.seed(1)
A = numpy.random.randn(n, m)
b = numpy.random.randn(n, 1)
# gamma must be positive due to DCP rules.
gamma = Parameter(sign="positive")

# Construct the problem.
x = Variable(m)
error = sum_squares(A*x - b)
obj = Minimize(error + gamma*norm(x, 1))
prob = Problem(obj)

# Construct a trade-off curve of ||Ax-b||^2 vs. ||x||_1
sq_penalty = []
l1_penalty = []
x_values = []
gamma_vals = numpy.logspace(-4, 6)
for val in gamma_vals:
    gamma.value = val
    prob.solve()
    # Use expr.value to get the numerical value of
    # an expression in the problem.
    sq_penalty.append(error.value)
    l1_penalty.append(norm(x, 1).value)
    x_values.append(x.value)

plt.rc('text', usetex=True)
plt.rc('font', family='serif')
plt.figure(figsize=(6,10))

# Plot trade-off curve.
plt.subplot(211)
plt.plot(l1_penalty, sq_penalty)
plt.xlabel(r'\|x\|_1', fontsize=16)
plt.ylabel(r'\|Ax-b\|^2', fontsize=16)
plt.title('Trade-Off Curve for LASSO', fontsize=16)

# Plot entries of x vs. gamma.
plt.subplot(212)
for i in range(m):
    plt.plot(gamma_vals, [xi[i,0] for xi in x_values])
plt.xlabel(r'\gamma', fontsize=16)
plt.ylabel(r'x_{i}', fontsize=16)
plt.xscale('log')
plt.title(r'\text{Entries of x vs. }\gamma', fontsize=16)

plt.tight_layout()
plt.show()

Trade-off curves can easily be computed in parallel. The code below computes in parallel the optimal x for each $\gamma$ in the LASSO problem above.

from multiprocessing import Pool

# Assign a value to gamma and find the optimal x.
def get_x(gamma_value):
    gamma.value = gamma_value
    result = prob.solve()
    return x.value

# Parallel computation (set to 1 process here).
pool = Pool(processes = 1)
x_values = pool.map(get_x, gamma_vals)

Disciplined Convex Programming¶

Disciplined convex programming (DCP) is a system for constructing mathematical expressions with known curvature from a given library of base functions. CVXPY uses DCP to ensure that the specified optimization problems are convex.

This section of the tutorial explains the rules of DCP and how they are applied by CVXPY.

Visit dcp.stanford.edu for a more interactive introduction to DCP.

Expressions¶

Expressions in CVXPY are formed from variables, parameters, numerical constants such as Python floats and Numpy matrices, the standard arithmetic operators +, -, *, /, and a library of functions. Here are some examples of CVXPY expressions:

from cvxpy import *

# Create variables and parameters.
x, y = Variable(), Variable()
a, b = Parameter(), Parameter()

# Examples of CVXPY expressions.
3.69 + b/3
x - 4*a
sqrt(x) - min_elemwise(y, x - a)
max_elemwise(2.66 - sqrt(y), square(x + 2*y))

Expressions can be scalars, vectors, or matrices. The dimensions of an expression are stored as expr.size. CVXPY will raise an exception if an expression is used in a way that doesn’t make sense given its dimensions, for example adding matrices of different size.

import numpy

X = Variable(5, 4)
A = numpy.ones((3, 5))

# Use expr.size to get the dimensions.
print "dimensions of X:", X.size
print "dimensions of sum_entries(X):", sum_entries(X).size
print "dimensions of A*X:", (A*X).size

# ValueError raised for invalid dimensions.
try:
    A + X
except ValueError, e:
    print e

dimensions of X: (5, 4)
dimensions of sum_entries(X): (1, 1)
dimensions of A*X: (3, 4)
Incompatible dimensions (3, 5) (5, 4)

CVXPY uses DCP analysis to determine the sign and curvature of each expression.

Sign¶

Each (sub)expression is flagged as positive (non-negative), negative (non-positive), zero, or unknown.

The signs of larger expressions are determined from the signs of their subexpressions. For example, the sign of the expression expr1*expr2 is

Zero if either expression has sign zero.
Positive if expr1 and expr2 have the same (known) sign.
Negative if expr1 and expr2 have opposite (known) signs.
Unknown if either expression has unknown sign.

The sign given to an expression is always correct. But DCP sign analysis may flag an expression as unknown sign when the sign could be figured out through more complex analysis. For instance, x*x is positive but has unknown sign by the rules above.

CVXPY determines the sign of constants by looking at their value. For scalar constants, this is straightforward. Vector and matrix constants with all positive (negative) entries are marked as positive (negative). Vector and matrix constants with both positive and negative entries are marked as unknown sign.

The sign of an expression is stored as expr.sign:

x = Variable()
a = Parameter(sign="negative")
c = numpy.array([1, -1])

print "sign of x:", x.sign
print "sign of a:", a.sign
print "sign of square(x):", square(x).sign
print "sign of c*a:", (c*a).sign

sign of x: UNKNOWN
sign of a: NEGATIVE
sign of square(x): POSITIVE
sign of c*a: UNKNOWN

Curvature¶

Each (sub)expression is flagged as one of the following curvatures (with respect to its variables)

Curvature	Meaning
constant	$f(x)$ independent of $x$
affine	$f(\theta x + (1-\theta)y) = \theta f(x) + (1-\theta)f(y), \; \forall x, \; y,\; \theta \in [0,1]$
convex	$f(\theta x + (1-\theta)y) \leq \theta f(x) + (1-\theta)f(y), \; \forall x, \; y,\; \theta \in [0,1]$
concave	$f(\theta x + (1-\theta)y) \geq \theta f(x) + (1-\theta)f(y), \; \forall x, \; y,\; \theta \in [0,1]$
unknown	DCP analysis cannot determine the curvature

using the curvature rules given below. As with sign analysis, the conclusion is always correct, but the simple analysis can flag expressions as unknown even when they are convex or concave. Note that any constant expression is also affine, and any affine expression is convex and concave.

Curvature rules¶

DCP analysis is based on applying a general composition theorem from convex analysis to each (sub)expression.

$f(expr_1, expr_2, ..., expr_n)$ is convex if $\text{ } f$ is a convex function and for each $expr_{i}$ one of the following conditions holds:

$f$ is increasing in argument $i$ and $expr_{i}$ is convex.
$f$ is decreasing in argument $i$ and $expr_{i}$ is concave.
$expr_{i}$ is affine or constant.

$f(expr_1, expr_2, ..., expr_n)$ is concave if $\text{ } f$ is a concave function and for each $expr_{i}$ one of the following conditions holds:

$f$ is increasing in argument $i$ and $expr_{i}$ is concave.
$f$ is decreasing in argument $i$ and $expr_{i}$ is convex.
$expr_{i}$ is affine or constant.

$f(expr_1, expr_2, ..., expr_n)$ is affine if $\text{ } f$ is an affine function and each $expr_{i}$ is affine.

If none of the three rules apply, the expression $f(expr_1, expr_2, ..., expr_n)$ is marked as having unknown curvature.

Whether a function is increasing or decreasing in an argument may depend on the sign of the argument. For instance, square is increasing for positive arguments and decreasing for negative arguments.

The curvature of an expression is stored as expr.curvature:

x = Variable()
a = Parameter(sign="positive")

print "curvature of x:", x.curvature
print "curvature of a:", a.curvature
print "curvature of square(x):", square(x).curvature
print "curvature of sqrt(x):", sqrt(x).curvature

curvature of x: AFFINE
curvature of a: CONSTANT
curvature of square(x): CONVEX
curvature of sqrt(x): CONCAVE

Infix operators¶

The infix operators +, -, *, / are treated exactly like functions. The infix operators + and - are affine, so the rules above are used to flag the curvature. For example, expr1 + expr2 is flagged as convex if expr1 and expr2 are convex.

expr1*expr2 is allowed only when one of the expressions is constant. If both expressions are non-constant, CVXPY will raise an exception. expr1/expr2 is allowed only when expr2 is a scalar constant. The curvature rules above apply. For example, expr1/expr2 is convex when expr1 is concave and expr2 is negative and constant.

Example 1¶

DCP analysis breaks expressions down into subexpressions. The tree visualization below shows how this works for the expression 2*square(x) + 3. Each subexpression is shown in a blue box. We mark its curvature on the left and its sign on the right.

Example 2¶

We’ll walk through the application of the DCP rules to the expression sqrt(1 + square(x)).

The variable x has affine curvature and unknown sign. The square function is convex and non-monotone for arguments of unknown sign. It can take the affine expression x as an argument; the result square(x) is convex.

The arithmetic operator + is affine and increasing, so the composition 1 + square(x) is convex by the curvature rule for convex functions. The function sqrt is concave and increasing, which means it can only take a concave argument. Since 1 + square(x) is convex, sqrt(1 + square(x)) violates the DCP rules and cannot be verified as convex.

In fact, sqrt(1 + square(x)) is a convex function of x, but the DCP rules are not able to verify convexity. If the expression is written as norm(vstack(1, x), 2), the L2 norm of the vector $[1,x]$, which has the same value as sqrt(1 + square(x)), then it will be certified as convex using the DCP rules.

print "sqrt(1 + square(x)) curvature:",
print sqrt(1 + square(x)).curvature
print "norm(vstack(1, x), 2) curvature:",
print norm(vstack(1, x), 2).curvature

sqrt(1 + square(x)) curvature: UNKNOWN
norm(vstack(1, x), 2) curvature: CONVEX

DCP problems¶

A problem is constructed from an objective and a list of constraints. If a problem follows the DCP rules, it is guaranteed to be convex and solvable by CVXPY. The DCP rules require that the problem objective have one of two forms:

Minimize(convex)
Maximize(concave)

The only valid constraints under the DCP rules are

affine == affine
convex <= concave
concave >= convex

You can check that a problem, constraint, or objective satisfies the DCP rules by calling object.is_dcp(). Here are some examples of DCP and non-DCP problems:

x = Variable()
y = Variable()

# DCP problems.
prob1 = Problem(Minimize(square(x - y)), [x + y >= 0])
prob2 = Problem(Maximize(sqrt(x - y)),
                [2*x - 3 == y,
                 square(x) <= 2])

print "prob1 is DCP:", prob1.is_dcp()
print "prob2 is DCP:", prob2.is_dcp()

# Non-DCP problems.

# A non-DCP objective.
prob3 = Problem(Maximize(square(x)))

print "prob3 is DCP:", prob3.is_dcp()
print "Maximize(square(x)) is DCP:", Maximize(square(x)).is_dcp()

# A non-DCP constraint.
prob4 = Problem(Minimize(square(x)), [sqrt(x) <= 2])

print "prob4 is DCP:", prob4.is_dcp()
print "sqrt(x) <= 2 is DCP:", (sqrt(x) <= 2).is_dcp()

prob1 is DCP: True
prob2 is DCP: True
prob3 is DCP: False
Maximize(square(x)) is DCP: False
prob4 is DCP: False
sqrt(x) <= 2 is DCP: False

CVXPY will raise an exception if you call problem.solve() on a non-DCP problem.

# A non-DCP problem.
prob = Problem(Minimize(sqrt(x)))

try:
    prob.solve()
except Exception as e:
    print e

Problem does not follow DCP rules.

Functions¶

This section of the tutorial describes the functions that can be applied to CVXPY expressions. CVXPY uses the function information in this section and the DCP rules to mark expressions with a sign and curvature.

Operators¶

The infix operators +, -, *, / are treated as functions. + and - are certainly affine functions. * and / are affine in CVXPY because expr1*expr2 is allowed only when one of the expressions is constant and expr1/expr2 is allowed only when expr2 is a scalar constant.

Indexing and slicing¶

All non-scalar expressions can be indexed using the syntax expr[i, j]. Indexing is an affine function. The syntax expr[i] can be used as a shorthand for expr[i, 0] when expr is a column vector. Similarly, expr[i] is shorthand for expr[0, i] when expr is a row vector.

Non-scalar expressions can also be sliced into using the standard Python slicing syntax. For example, expr[i:j:k, r] selects every kth element in column r of expr, starting at row i and ending at row j-1.

Transpose¶

The transpose of any expression can be obtained using the syntax expr.T. Transpose is an affine function.

Scalar functions¶

A scalar function takes one or more scalars, vectors, or matrices as arguments and returns a scalar.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
entr(X)	$\sum_{ij}-X_{ij} \log (X_{ij})$	$X_{ij} > 0$	unknown	concave	None
geo_mean(x) geo_mean(x, p) $p \in \mathbf{R}^n_{+}$ $p \neq 0$	$x_1^{1/n} \cdots x_n^{1/n}$ $\left(x_1^{p_1} \cdots x_n^{p_n}\right)^{\frac{1}{\mathbf{1}^T p}}$	$x \in \mathbf{R}^n_{+}$	positive	concave	incr.
kl_div(X, Y)	$\sum_{ij} X_{ij} \log(X_{ij}/Y_{ij}) -X_{ij}+Y_{ij}$	$X_{ij} > 0$ $Y_{ij} > 0$	positive	convex	None
lamdba_max(X)	$\lambda_{\max}(X)$	$X \in \mathbf{S}^n$	unknown	convex	None
lambda_min(X)	$\lambda_{\min}(X)$	$X \in \mathbf{S}^n$	unknown	concave	None
lambda_sum_largest(X, k) $k = 1,\ldots, n$	$\text{sum of $k$ largest}\\ \text{eigenvalues of $X$}$	$X \in\mathbf{S}^{n}$	unknown	convex	None
lambda_sum_smallest(X, k) $k = 1,\ldots, n$	$\text{sum of $k$ smallest}\\ \text{eigenvalues of $X$}$	$X \in\mathbf{S}^{n}$	unknown	concave	None
log_det(X)	$\log \left(\det (X)\right)$	$X \in \mathbf{S}^n_+$	unknown	concave	None
log_sum_exp(X)	$\log \left(\sum_{ij}e^{X_{ij}}\right)$	$X \in\mathbf{R}^{n \times m}$	unknown	convex	incr.
matrix_frac(x, P)	$x^T P^{-1} x$	$x \in \mathbf{R}^n$ $P \in\mathbf{S}^n_{++}$	positive	convex	None
max_entries(X)	$\max_{ij}\left\{ X_{ij}\right\}$	$X \in\mathbf{R}^{n \times m}$	same as X	convex	incr.
min_entries(X)	$\min_{ij}\left\{ X_{ij}\right\}$	$X \in\mathbf{R}^{n \times m}$	same as X	concave	incr.
mixed_norm(X, p, q)	$\left(\sum_k\left(\sum_l\lvert x_{k,l}\rvert^p\right)^{q/p}\right)^{1/q}$	$X \in\mathbf{R}^{n \times n}$	positive	convex	None
norm(x) norm(x, 2)	$\sqrt{\sum_{i}x_{i}^2 }$	$X \in\mathbf{R}^{n}$	positive	convex	for $x_{i} \geq 0$ for $x_{i} \leq 0$
norm(X, “fro”)	$\sqrt{\sum_{ij}X_{ij}^2 }$	$X \in\mathbf{R}^{n \times m}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
norm(X, 1)	$\sum_{ij}\lvert X_{ij} \rvert$	$X \in\mathbf{R}^{n \times m}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
norm(X, “inf”)	$\max_{ij} \{\lvert X_{ij} \rvert\}$	$X \in\mathbf{R}^{n \times m}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
norm(X, “nuc”)	$\mathrm{tr}\left(\left(X^T X\right)^{1/2}\right)$	$X \in\mathbf{R}^{n \times m}$	positive	convex	None
norm(X) norm(X, 2)	$\sqrt{\lambda_{\max}\left(X^T X\right)}$	$X \in\mathbf{R}^{n \times m}$	positive	convex	None
quad_form(x, P) constant $P \in \mathbf{S}^n_+$	$x^T P x$	$x \in \mathbf{R}^n$	positive	convex	for $x_i \geq 0$ for $x_i \leq 0$
quad_form(x, P) constant $P \in \mathbf{S}^n_-$	$x^T P x$	$x \in \mathbf{R}^n$	negative	concave	for $x_i \geq 0$ for $x_i \leq 0$
quad_form(c, X) constant $c \in \mathbf{R}^n$	$c^T X c$	$X \in\mathbf{R}^{n \times n}$	depends on c, X	affine	depends on c
quad_over_lin(X, y)	$\left(\sum_{ij}X_{ij}^2\right)/y$	$x \in \mathbf{R}^n$ $y > 0$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$ decr. in $y$
sum_entries(X)	$\sum_{ij}X_{ij}$	$X \in\mathbf{R}^{n \times m}$	same as X	affine	incr.
sum_largest(X, k) $k = 1,2,\ldots$	$\text{sum of } k\text{ largest }X_{ij}$	$X \in\mathbf{R}^{n \times m}$	same as X	convex	incr.
sum_smallest(X, k) $k = 1,2,\ldots$	$\text{sum of } k\text{ smallest }X_{ij}$	$X \in\mathbf{R}^{n \times m}$	same as X	concave	incr.
sum_squares(X)	$\sum_{ij}X_{ij}^2$	$X \in\mathbf{R}^{n \times m}$	positive	convex	for $X_{ij} \geq 0$ for $X_{ij} \leq 0$
trace(X)	$\mathrm{tr}\left(X \right)$	$X \in\mathbf{R}^{n \times n}$	same as X	affine	incr.

Clarifications¶

The domain $\mathbf{S}^n$ refers to the set of symmetric matrices. The domains $\mathbf{S}^n_+$ and $\mathbf{S}^n_-$ refer to the set of positive semi-definite and negative semi-definite matrices, respectively. Similarly, $\mathbf{S}^n_{++}$ and $\mathbf{S}^n_{--}$ refer to the set of positive definite and negative definite matrices, respectively.

For a vector expression x, norm(x) and norm(x, 2) give the Euclidean norm. For a matrix expression X, however, norm(X) and norm(X, 2) give the spectral norm.

The function norm(X, "fro") is called the Frobenius norm and norm(X, "nuc") the nuclear norm. The nuclear norm can also be defined as the sum of X‘s singular values.

The functions max_entries and min_entries give the largest and smallest entry, respectively, in a single expression. These functions should not be confused with max_elemwise and min_elemwise (see Elementwise functions). Use max_elemwise and min_elemwise to find the max or min of a list of scalar expressions.

The function sum_entries sums all the entries in a single expression. The built-in Python sum should be used to add together a list of expressions. For example, the following code sums the columns of a matrix variable:

X = Variable(100, 100)
col_sum = sum([X[:, i] for i in range(X.size[1])])

Elementwise functions¶

These functions operate on each element of their arguments. For example, if X is a 5 by 4 matrix variable, then abs(X) is a 5 by 4 matrix expression. abs(X)[1, 2] is equivalent to abs(X[1, 2]).

Elementwise functions that take multiple arguments, such as max_elemwise and mul_elemwise, operate on the corresponding elements of each argument. For example, if X and Y are both 3 by 3 matrix variables, then max_elemwise(X, Y) is a 3 by 3 matrix expression. max_elemwise(X, Y)[2, 0] is equivalent to max_elemwise(X[2, 0], Y[2, 0]). This means all arguments must have the same dimensions or be scalars, which are promoted.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
abs(x)	$\lvert x \rvert$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
exp(x)	$e^x$	$x \in \mathbf{R}$	positive	convex	incr.
huber(x, M=1) $M \geq 0$	$\begin{cases}x^2 &\|x\| \leq M \\2M\|x\| - M^2&\|x\| >M\end{cases}$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
inv_pos(x)	$1/x$	$x > 0$	positive	convex	decr.
log(x)	$\log(x)$	$x > 0$	unknown	concave	incr.
log1p(x)	$\log(x+1)$	$x > -1$	sign(x)	concave	incr.
max_elemwise(x1, ..., xk)	$\max \left\{x_1, \ldots , x_k\right\}$	$x_i \in \mathbf{R}$	max(sign(xi))	convex	incr.
min_elemwise(x1, ..., xk)	$\min \left\{x_1, \ldots , x_k\right\}$	$x_i \in \mathbf{R}$	min(sign(xi))	concave	incr.
mul_elemwise(c, x) $c \in \mathbf{R}$	c*x	$x \in\mathbf{R}$	sign(c*x)	affine	depends on c
neg(x)	$\max \left\{-x, 0 \right\}$	$x \in \mathbf{R}$	positive	convex	decr.
pos(x)	$\max \left\{x, 0 \right\}$	$x \in \mathbf{R}$	positive	convex	incr.
power(x, 0)	$1$	$x \in \mathbf{R}$	positive	constant
power(x, 1)	$x$	$x \in \mathbf{R}$	same as `x`	affine	incr.
power(x, p) $p = 2, 4, 8, \ldots$	$x^p$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
power(x, p) $p < 0$	$x^p$	$x > 0$	positive	convex	decr.
power(x, p) $0 < p < 1$	$x^p$	$x \geq 0$	positive	concave	incr.
power(x, p) $p > 1,\ p \neq 2, 4, 8, \ldots$	$x^p$	$x \geq 0$	positive	convex	incr.
scalene(x, alpha, beta) $\text{alpha} \geq 0$ $\text{beta} \geq 0$	$\alpha\mathrm{pos}(x)+ \beta\mathrm{neg}(x)$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$
sqrt(x)	$\sqrt x$	$x \geq 0$	positive	concave	incr.
square(x)	$x^2$	$x \in \mathbf{R}$	positive	convex	for $x \geq 0$ for $x \leq 0$

Vector/Matrix functions¶

A vector/matrix function takes one or more scalars, vectors, or matrices as arguments and returns a vector or matrix.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
conv(c, x) $c\in\mathbf{R}^m$	$c*x$	$x\in \mathbf{R}^n$	depends on c, x	affine	depends on c
diag(x)	$\left[\begin{matrix}x_1 & & \\& \ddots & \\& & x_n\end{matrix}\right]$	$x \in\mathbf{R}^{n}$	same as x	affine	incr.
diag(X)	$\left[\begin{matrix}X_{11} \\\vdots \\X_{nn}\end{matrix}\right]$	$X \in\mathbf{R}^{n \times n}$	same as X	affine	incr.
hstack(X1, ..., Xk)	$\left[\begin{matrix}X_1 \cdots X_k\end{matrix}\right]$	$X_i \in\mathbf{R}^{n \times m_i}$	sign(sum([x1, ..., xk]))	affine	incr.
reshape(X, n’, m’)	$X' \in\mathbf{R}^{n' \times m'}$	$X \in\mathbf{R}^{n \times m}$ $n'm' = nm$	same as X	affine	incr.
vec(X)	$x' \in\mathbf{R}^{nm}$	$X \in\mathbf{R}^{n \times m}$	same as X	affine	incr.
vstack(X1, ..., Xk)	$\left[\begin{matrix}X_1 \\ \vdots \\X_k\end{matrix}\right]$	$X_i \in\mathbf{R}^{n_i \times m}$	sign(sum([x1, ..., xk]))	affine	incr.

Clarifications¶

The output $y$ of conv(c, x) has size $n+m-1$ and is defined as $y[k]=\sum_{j=0}^k c[j]x[k-j]$.

The output $x'$ of vec(X) is the matrix $X$ flattened in column-major order into a vector. Formally, $x'_i = X_{i \bmod{n}, \left \lfloor{i/n}\right \rfloor }$.

The output $X'$ of reshape(X, n', m') is the matrix $X$ cast into an $n' \times m'$ matrix. The entries are taken from $X$ in column-major order and stored in $X'$ in column-major order. Formally, $X'_{ij} = \mathbf{vec}(X)_{n'j + i}$.

Advanced Features¶

This section of the tutorial covers features of CVXPY intended for users with advanced knowledge of convex optimization. We recommend Convex Optimization by Boyd and Vandenberghe as a reference for any terms you are unfamiliar with.

Dual variables¶

You can use CVXPY to find the optimal dual variables for a problem. When you call prob.solve() each dual variable in the solution is stored in the dual_value field of the constraint it corresponds to.

from cvxpy import *

# Create two scalar optimization variables.
x = Variable()
y = Variable()

# Create two constraints.
constraints = [x + y == 1,
               x - y >= 1]

# Form objective.
obj = Minimize(square(x - y))

# Form and solve problem.
prob = Problem(obj, constraints)
prob.solve()

# The optimal dual variable (Lagrange multiplier) for
# a constraint is stored in constraint.dual_value.
print "optimal (x + y == 1) dual variable", constraints[0].dual_value
print "optimal (x - y >= 1) dual variable", constraints[1].dual_value
print "x - y value:", (x - y).value

optimal (x + y == 1) dual variable 6.47610300459e-18
optimal (x - y >= 1) dual variable 2.00025244976
x - y value: 0.999999986374

The dual variable for x - y >= 1 is 2. By complementarity this implies that x - y is 1, which we can see is true. The fact that the dual variable is non-zero also tells us that if we tighten x - y >= 1, (i.e., increase the right-hand side), the optimal value of the problem will increase.

Semidefinite matrices¶

Many convex optimization problems involve constraining matrices to be positive or negative semidefinite (e.g., SDPs). You can do this in CVXPY using the Semidef constructor. Semidef(n) constructs an n by n variable constrained to be positive semidefinite. For example,

# Creates a 100 by 100 positive semidefinite variable.
X = Semidef(100)

# You can use X anywhere you would use
# a normal CVXPY variable.
obj = Minimize(norm(X) + sum_entries(X))

The following code shows how to use Semidef to constrain matrix expressions to be positive or negative semidefinite:

# expr1 must be positive semidefinite.
constr1 = (expr1 == Semidef(n))

# expr2 must be negative semidefinite.
constr2 = (expr2 == -Semidef(n))

To constrain a matrix expression to be symmetric (but not necessarily positive or negative semidefinite), simply write

# expr must be symmetric.
constr = (expr == expr.T)

Mixed-integer programs¶

In mixed-integer programs, certain variables are constrained to be boolean or integer valued. You can construct mixed-integer programs using the Bool and Int constructors. These take the same arguments as the Variable constructor, and they return a variable constrained to have only boolean or integer valued entries.

The following code shows the Bool and Int constructors in action:

# Creates a 10-vector constrained to have boolean valued entries.
x = Bool(10)

# expr1 must be boolean valued.
constr1 = (expr1 == x)

# Creates a 5 by 7 matrix constrained to have integer valued entries.
Z = Int(5, 7)

# expr2 must be integer valued.
constr2 = (expr2 == Z)

Solve method options¶

The solve method takes optional arguments that let you change how CVXPY solves the problem. Here is the signature for the solve method:

solve(solver=None, verbose=False, **kwargs)¶

Solves a DCP compliant optimization problem.

Parameters:	solver (str, optional) – The solver to use. verbose (bool, optional) – Overrides the default of hiding solver output. kwargs – Additional keyword arguments specifying solver specific options.
Returns:	The optimal value for the problem, or a string indicating why the problem could not be solved.

We will discuss the optional arguments in detail below.

Choosing a solver¶

CVXPY is distributed with the open source solvers ECOS, ECOS_BB, CVXOPT, and SCS. CVXPY also supports GLPK and GLPK_MI via the CVXOPT GLPK interface. The table below shows the types of problems the solvers can handle.

	LP	SOCP	SDP	EXP	MIP
GLPK	X
GLPK_MI	X				X
ECOS	X	X
ECOS_BB	X	X			X
CVXOPT	X	X	X	X
SCS	X	X	X	X

Here EXP refers to problems with exponential cone constraints. The exponential cone is defined as

$\{(x,y,z) \mid y > 0, y\exp(x/y) \leq z \} \cup \{ (x,y,z) \mid x \leq 0, y = 0, z \geq 0\}$.

You cannot specify cone constraints explicitly in CVXPY, but cone constraints are added when CVXPY converts the problem into standard form.

By default CVXPY calls the solver most specialized to the problem type. For example, ECOS is called for SOCPs. SCS and CVXOPT can both handle all problems (except mixed-integer programs). CVXOPT is preferred by default. For many problems SCS will be faster, though less accurate. ECOS_BB is called for mixed-integer LPs and SOCPs.

You can change the solver called by CVXPY using the solver keyword argument. If the solver you choose cannot solve the problem, CVXPY will raise an exception. Here’s example code solving the same problem with different solvers.

# Solving a problem with different solvers.
x = Variable(2)
obj = Minimize(x[0] + norm(x, 1))
constraints = [x >= 2]
prob = Problem(obj, constraints)

# Solve with ECOS.
prob.solve(solver=ECOS)
print "optimal value with ECOS:", prob.value

# Solve with ECOS_BB.
prob.solve(solver=ECOS_BB)
print "optimal value with ECOS_BB:", prob.value

# Solve with CVXOPT.
prob.solve(solver=CVXOPT)
print "optimal value with CVXOPT:", prob.value

# Solve with SCS.
prob.solve(solver=SCS)
print "optimal value with SCS:", prob.value

# Solve with GLPK.
prob.solve(solver=GLPK)
print "optimal value with GLPK:", prob.value

# Solve with GLPK_MI.
prob.solve(solver=GLPK_MI)
print "optimal value with GLPK_MI:", prob.value

optimal value with ECOS: 5.99999999551
optimal value with ECOS_BB: 5.99999999551
optimal value with CVXOPT: 6.00000000512
optimal value with SCS: 6.00046055789
optimal value with GLPK: 6.0
optimal value with GLPK_MI: 6.0

Use the installed_solvers utility function to get a list of the solvers your installation of CVXPY supports.

print installed_solvers()

['CVXOPT', 'GLPK', 'GLPK_MI', 'ECOS_BB', 'ECOS', 'SCS']

Viewing solver output¶

All the solvers can print out information about their progress while solving the problem. This information can be useful in debugging a solver error. To see the output from the solvers, set verbose=True in the solve method.

# Solve with ECOS and display output.
prob.solve(solver=ECOS, verbose=True)
print "optimal value with ECOS:", prob.value

ECOS 1.0.3 - (c) A. Domahidi, Automatic Control Laboratory, ETH Zurich, 2012-2014.

It     pcost         dcost      gap     pres    dres     k/t     mu      step     IR
 0   +0.000e+00   +4.000e+00   +2e+01   2e+00   1e+00   1e+00   3e+00    N/A     1 1 -
 1   +6.451e+00   +8.125e+00   +5e+00   7e-01   5e-01   7e-01   7e-01   0.7857   1 1 1
 2   +6.788e+00   +6.839e+00   +9e-02   1e-02   8e-03   3e-02   2e-02   0.9829   1 1 1
 3   +6.828e+00   +6.829e+00   +1e-03   1e-04   8e-05   3e-04   2e-04   0.9899   1 1 1
 4   +6.828e+00   +6.828e+00   +1e-05   1e-06   8e-07   3e-06   2e-06   0.9899   2 1 1
 5   +6.828e+00   +6.828e+00   +1e-07   1e-08   8e-09   4e-08   2e-08   0.9899   2 1 1

OPTIMAL (within feastol=1.3e-08, reltol=1.5e-08, abstol=1.0e-07).
Runtime: 0.000121 seconds.

optimal value with ECOS: 6.82842708233

Setting solver options¶

The ECOS, ECOS_BB, CVXOPT, and SCS Python interfaces allow you to set solver options such as the maximum number of iterations. You can pass these options along through CVXPY as keyword arguments.

For example, here we tell SCS to use an indirect method for solving linear equations rather than a direct method.

# Solve with SCS, use sparse-indirect method.
prob.solve(solver=SCS, verbose=True, use_indirect=True)
print "optimal value with SCS:", prob.value

----------------------------------------------------------------------------
    SCS v1.0.5 - Splitting Conic Solver
    (c) Brendan O'Donoghue, Stanford University, 2012
----------------------------------------------------------------------------
Lin-sys: sparse-indirect, nnz in A = 13, CG tol ~ 1/iter^(2.00)
EPS = 1.00e-03, ALPHA = 1.80, MAX_ITERS = 2500, NORMALIZE = 1, SCALE = 5.00
Variables n = 5, constraints m = 9
Cones:  linear vars: 6
    soc vars: 3, soc blks: 1
Setup time: 2.78e-04s
----------------------------------------------------------------------------
 Iter | pri res | dua res | rel gap | pri obj | dua obj | kap/tau | time (s)
----------------------------------------------------------------------------
     0| 4.60e+00  5.78e-01       nan      -inf       inf       inf  3.86e-05
    60| 3.92e-05  1.12e-04  6.64e-06  6.83e+00  6.83e+00  1.41e-17  9.51e-05
----------------------------------------------------------------------------
Status: Solved
Timing: Total solve time: 9.76e-05s
    Lin-sys: avg # CG iterations: 1.00, avg solve time: 2.24e-07s
    Cones: avg projection time: 4.90e-08s
----------------------------------------------------------------------------
Error metrics:
|Ax + s - b|_2 / (1 + |b|_2) = 3.9223e-05
|A'y + c|_2 / (1 + |c|_2) = 1.1168e-04
|c'x + b'y| / (1 + |c'x| + |b'y|) = 6.6446e-06
dist(s, K) = 0, dist(y, K*) = 0, s'y = 0
----------------------------------------------------------------------------
c'x = 6.8284, -b'y = 6.8285
============================================================================
optimal value with SCS: 6.82837896975

Here’s the complete list of solver options.

ECOS options:

'max_iters': maximum number of iterations (default: 100).
'abstol': absolute accuracy (default: 1e-7).
'reltol': relative accuracy (default: 1e-6).
'feastol': tolerance for feasibility conditions (default: 1e-7).
'abstol_inacc': absolute accuracy for inaccurate solution (default: 5e-5).
'reltol_inacc': relative accuracy for inaccurate solution (default: 5e-5).
'feastol_inacc': tolerance for feasibility condition for inaccurate solution (default: 1e-4).

ECOS_BB options:

'mi_max_iters': maximum number of branch and bound iterations (default: 1000)
'mi_abs_eps': absolute tolerance between upper and lower bounds (default: 1e-6)
'mi_rel_eps': relative tolerance, (U-L)/L, between upper and lower bounds (default: 1e-3)

CVXOPT options:

'max_iters': maximum number of iterations (default: 100).
'abstol': absolute accuracy (default: 1e-7).
'reltol': relative accuracy (default: 1e-6).
'feastol': tolerance for feasibility conditions (default: 1e-7).
'refinement': number of iterative refinement steps after solving KKT system (default: 1).
'kktsolver': The KKT solver used. The default is a regularized LDL solver. The “chol” solver is faster but requires that A and [A; G] be full rank.

SCS options:

'max_iters': maximum number of iterations (default: 2500).
'eps': convergence tolerance (default: 1e-3).
'alpha': relaxation parameter (default: 1.8).
'normalize': whether to precondition data matrices (default: True).
'use_indirect': whether to use indirect solver for KKT sytem (instead of direct) (default: False).

Getting the standard form¶

If you are interested in getting the standard form that CVXPY produces for a problem, you can use the get_problem_data method. Calling get_problem_data(solver) on a problem object returns a dict of the arguments that CVXPY would pass to that solver. If the solver you choose cannot solve the problem, CVXPY will raise an exception.

# Get ECOS arguments.
data = prob.get_problem_data(ECOS)

# Get ECOS_BB arguments.
data = prob.get_problem_data(ECOS_BB)

# Get CVXOPT arguments.
data = prob.get_problem_data(CVXOPT)

# Get SCS arguments.
data = prob.get_problem_data(SCS)

After you solve the standard conic form problem returned by get_problem_data, you can unpack the raw solver output using the unpack_results method. Calling unpack_results(solver, solver_output) on a problem will update the values of all primal and dual variables as well as the problem value and status.

For example, the following code is equivalent to solving the problem directly with CVXPY:

# Get ECOS arguments.
data = prob.get_problem_data(ECOS)
# Call ECOS solver.
solver_output = ecos.solve(data["c"], data["G"], data["h"],
                           data["dims"], data["A"], data["b"])
# Unpack raw solver output.
prob.unpack_results(ECOS, solver_output)

Examples¶

These examples show many different ways to use CVXPY. The Basic Examples section shows how to solve some common optimization problems in CVXPY. The Advanced Examples section contains more complex examples aimed at experts in convex optimization.

Basic Examples¶

Total variation in-painting

Advanced Examples¶

Citing CVXPY¶

If you use CVXPY for published work, we encourage you to cite the software.

Use the following BibTeX citation:

@misc{cvxpy,
  author       = {Steven Diamond and Eric Chu and Stephen Boyd},
  title        = {{CVXPY}: A {P}ython-Embedded Modeling Language for Convex Optimization, version 0.2},
  howpublished = {\url{http://cvxpy.org/}},
  month        = may,
  year         = 2014
}

How to Help¶

We welcome all contributors to CVXPY. You don’t need to be an expert in convex optimization to help out!

The cvxpy mailing list is for users and developers of CVXPY. Join this mailing list if you’re interested in contributing to CVXPY or would like to track CVXPY’s progress.

We use GitHub to track our source code and for tracking and discussing issues. The open issues are a rough list of what needs to be done in developing CVXPY.

Curvature	Meaning
constant	\(f(x)\) independent of \(x\)
affine	\(f(\theta x + (1-\theta)y) = \theta f(x) + (1-\theta)f(y), \; \forall x, \; y,\; \theta \in [0,1]\)
convex	\(f(\theta x + (1-\theta)y) \leq \theta f(x) + (1-\theta)f(y), \; \forall x, \; y,\; \theta \in [0,1]\)
concave	\(f(\theta x + (1-\theta)y) \geq \theta f(x) + (1-\theta)f(y), \; \forall x, \; y,\; \theta \in [0,1]\)
unknown	DCP analysis cannot determine the curvature

Function	Meaning	Domain	Sign	Curvature	Monotonicity
entr(X)	\(\sum_{ij}-X_{ij} \log (X_{ij})\)	\(X_{ij} > 0\)	unknown	concave	None
geo_mean(x) geo_mean(x, p) \(p \in \mathbf{R}^n_{+}\) \(p \neq 0\)	\(x_1^{1/n} \cdots x_n^{1/n}\) \(\left(x_1^{p_1} \cdots x_n^{p_n}\right)^{\frac{1}{\mathbf{1}^T p}}\)	\(x \in \mathbf{R}^n_{+}\)	positive	concave	incr.
kl_div(X, Y)	\(\sum_{ij} X_{ij} \log(X_{ij}/Y_{ij}) -X_{ij}+Y_{ij}\)	\(X_{ij} > 0\) \(Y_{ij} > 0\)	positive	convex	None
lamdba_max(X)	\(\lambda_{\max}(X)\)	\(X \in \mathbf{S}^n\)	unknown	convex	None
lambda_min(X)	\(\lambda_{\min}(X)\)	\(X \in \mathbf{S}^n\)	unknown	concave	None
lambda_sum_largest(X, k) \(k = 1,\ldots, n\)	\(\text{sum of $k$ largest}\\ \text{eigenvalues of $X$}\)	\(X \in\mathbf{S}^{n}\)	unknown	convex	None
lambda_sum_smallest(X, k) \(k = 1,\ldots, n\)	\(\text{sum of $k$ smallest}\\ \text{eigenvalues of $X$}\)	\(X \in\mathbf{S}^{n}\)	unknown	concave	None
log_det(X)	\(\log \left(\det (X)\right)\)	\(X \in \mathbf{S}^n_+\)	unknown	concave	None
log_sum_exp(X)	\(\log \left(\sum_{ij}e^{X_{ij}}\right)\)	\(X \in\mathbf{R}^{n \times m}\)	unknown	convex	incr.
matrix_frac(x, P)	\(x^T P^{-1} x\)	\(x \in \mathbf{R}^n\) \(P \in\mathbf{S}^n_{++}\)	positive	convex	None
max_entries(X)	\(\max_{ij}\left\{ X_{ij}\right\}\)	\(X \in\mathbf{R}^{n \times m}\)	same as X	convex	incr.
min_entries(X)	\(\min_{ij}\left\{ X_{ij}\right\}\)	\(X \in\mathbf{R}^{n \times m}\)	same as X	concave	incr.
mixed_norm(X, p, q)	\(\left(\sum_k\left(\sum_l\lvert x_{k,l}\rvert^p\right)^{q/p}\right)^{1/q}\)	\(X \in\mathbf{R}^{n \times n}\)	positive	convex	None
norm(x) norm(x, 2)	\(\sqrt{\sum_{i}x_{i}^2 }\)	\(X \in\mathbf{R}^{n}\)	positive	convex	for \(x_{i} \geq 0\) for \(x_{i} \leq 0\)
norm(X, “fro”)	\(\sqrt{\sum_{ij}X_{ij}^2 }\)	\(X \in\mathbf{R}^{n \times m}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
norm(X, 1)	\(\sum_{ij}\lvert X_{ij} \rvert\)	\(X \in\mathbf{R}^{n \times m}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
norm(X, “inf”)	\(\max_{ij} \{\lvert X_{ij} \rvert\}\)	\(X \in\mathbf{R}^{n \times m}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
norm(X, “nuc”)	\(\mathrm{tr}\left(\left(X^T X\right)^{1/2}\right)\)	\(X \in\mathbf{R}^{n \times m}\)	positive	convex	None
norm(X) norm(X, 2)	\(\sqrt{\lambda_{\max}\left(X^T X\right)}\)	\(X \in\mathbf{R}^{n \times m}\)	positive	convex	None
quad_form(x, P) constant \(P \in \mathbf{S}^n_+\)	\(x^T P x\)	\(x \in \mathbf{R}^n\)	positive	convex	for \(x_i \geq 0\) for \(x_i \leq 0\)
quad_form(x, P) constant \(P \in \mathbf{S}^n_-\)	\(x^T P x\)	\(x \in \mathbf{R}^n\)	negative	concave	for \(x_i \geq 0\) for \(x_i \leq 0\)
quad_form(c, X) constant \(c \in \mathbf{R}^n\)	\(c^T X c\)	\(X \in\mathbf{R}^{n \times n}\)	depends on c, X	affine	depends on c
quad_over_lin(X, y)	\(\left(\sum_{ij}X_{ij}^2\right)/y\)	\(x \in \mathbf{R}^n\) \(y > 0\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\) decr. in \(y\)
sum_entries(X)	\(\sum_{ij}X_{ij}\)	\(X \in\mathbf{R}^{n \times m}\)	same as X	affine	incr.
sum_largest(X, k) \(k = 1,2,\ldots\)	\(\text{sum of } k\text{ largest }X_{ij}\)	\(X \in\mathbf{R}^{n \times m}\)	same as X	convex	incr.
sum_smallest(X, k) \(k = 1,2,\ldots\)	\(\text{sum of } k\text{ smallest }X_{ij}\)	\(X \in\mathbf{R}^{n \times m}\)	same as X	concave	incr.
sum_squares(X)	\(\sum_{ij}X_{ij}^2\)	\(X \in\mathbf{R}^{n \times m}\)	positive	convex	for \(X_{ij} \geq 0\) for \(X_{ij} \leq 0\)
trace(X)	\(\mathrm{tr}\left(X \right)\)	\(X \in\mathbf{R}^{n \times n}\)	same as X	affine	incr.

Function	Meaning	Domain	Sign	Curvature	Monotonicity
abs(x)	\(\lvert x \rvert\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
exp(x)	\(e^x\)	\(x \in \mathbf{R}\)	positive	convex	incr.
huber(x, M=1) \(M \geq 0\)	\(\begin{cases}x^2 &\|x\| \leq M \\2M\|x\| - M^2&\|x\| >M\end{cases}\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
inv_pos(x)	\(1/x\)	\(x > 0\)	positive	convex	decr.
log(x)	\(\log(x)\)	\(x > 0\)	unknown	concave	incr.
log1p(x)	\(\log(x+1)\)	\(x > -1\)	sign(x)	concave	incr.
max_elemwise(x1, ..., xk)	\(\max \left\{x_1, \ldots , x_k\right\}\)	\(x_i \in \mathbf{R}\)	max(sign(xi))	convex	incr.
min_elemwise(x1, ..., xk)	\(\min \left\{x_1, \ldots , x_k\right\}\)	\(x_i \in \mathbf{R}\)	min(sign(xi))	concave	incr.
mul_elemwise(c, x) \(c \in \mathbf{R}\)	c*x	\(x \in\mathbf{R}\)	sign(c*x)	affine	depends on c
neg(x)	\(\max \left\{-x, 0 \right\}\)	\(x \in \mathbf{R}\)	positive	convex	decr.
pos(x)	\(\max \left\{x, 0 \right\}\)	\(x \in \mathbf{R}\)	positive	convex	incr.
power(x, 0)	\(1\)	\(x \in \mathbf{R}\)	positive	constant
power(x, 1)	\(x\)	\(x \in \mathbf{R}\)	same as `x`	affine	incr.
power(x, p) \(p = 2, 4, 8, \ldots\)	\(x^p\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
power(x, p) \(p < 0\)	\(x^p\)	\(x > 0\)	positive	convex	decr.
power(x, p) \(0 < p < 1\)	\(x^p\)	\(x \geq 0\)	positive	concave	incr.
power(x, p) \(p > 1,\ p \neq 2, 4, 8, \ldots\)	\(x^p\)	\(x \geq 0\)	positive	convex	incr.
scalene(x, alpha, beta) \(\text{alpha} \geq 0\) \(\text{beta} \geq 0\)	\(\alpha\mathrm{pos}(x)+ \beta\mathrm{neg}(x)\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)
sqrt(x)	\(\sqrt x\)	\(x \geq 0\)	positive	concave	incr.
square(x)	\(x^2\)	\(x \in \mathbf{R}\)	positive	convex	for \(x \geq 0\) for \(x \leq 0\)

Function	Meaning	Domain	Sign	Curvature	Monotonicity
conv(c, x) \(c\in\mathbf{R}^m\)	\(c*x\)	\(x\in \mathbf{R}^n\)	depends on c, x	affine	depends on c
diag(x)	\(\left[\begin{matrix}x_1 & & \\& \ddots & \\& & x_n\end{matrix}\right]\)	\(x \in\mathbf{R}^{n}\)	same as x	affine	incr.
diag(X)	\(\left[\begin{matrix}X_{11} \\\vdots \\X_{nn}\end{matrix}\right]\)	\(X \in\mathbf{R}^{n \times n}\)	same as X	affine	incr.
hstack(X1, ..., Xk)	\(\left[\begin{matrix}X_1 \cdots X_k\end{matrix}\right]\)	\(X_i \in\mathbf{R}^{n \times m_i}\)	sign(sum([x1, ..., xk]))	affine	incr.
reshape(X, n’, m’)	\(X' \in\mathbf{R}^{n' \times m'}\)	\(X \in\mathbf{R}^{n \times m}\) \(n'm' = nm\)	same as X	affine	incr.
vec(X)	\(x' \in\mathbf{R}^{nm}\)	\(X \in\mathbf{R}^{n \times m}\)	same as X	affine	incr.
vstack(X1, ..., Xk)	\(\left[\begin{matrix}X_1 \\ \vdots \\X_k\end{matrix}\right]\)	\(X_i \in\mathbf{R}^{n_i \times m}\)	sign(sum([x1, ..., xk]))	affine	incr.

Welcome to CVXPY¶

Install Guide¶

Mac OS X¶

Ubuntu 14.04¶

Windows¶

Other Platforms¶

Install from source¶

Install with GLPK support¶

Tutorial¶

What is CVXPY?¶

Namespace¶

Changing the problem¶

Infeasible and unbounded problems¶

Other problem statuses¶

Vectors and matrices¶

Constraints¶

Parameters¶

Disciplined Convex Programming¶

Expressions¶

Sign¶

Curvature¶

Curvature rules¶

Infix operators¶

Example 1¶

Example 2¶

DCP problems¶

Functions¶

Operators¶

Indexing and slicing¶

Transpose¶

Scalar functions¶

Clarifications¶

Elementwise functions¶

Vector/Matrix functions¶

Clarifications¶

Advanced Features¶

Dual variables¶

Semidefinite matrices¶

Mixed-integer programs¶

Solve method options¶

Choosing a solver¶

Viewing solver output¶

Setting solver options¶

Getting the standard form¶

Examples¶

Basic Examples¶

Advanced Examples¶

Citing CVXPY¶

How to Help¶