solve eigenvalue problem in python

Python Numerical Methods

This notebook contains an excerpt from the Python Programming and Numerical Methods - A Guide for Engineers and Scientists , the content is also available at Berkeley Python Numerical Methods .

The copyright of the book belongs to Elsevier. We also have this interactive book online for a better learning experience. The code is released under the MIT license . If you find this content useful, please consider supporting the work on Elsevier or Amazon !

< 15.3 The QR Method | Contents | 15.5 Summary and Problems >

Eigenvalues and Eigenvectors in Python ¶

Though the methods we introduced so far look complicated, the actually calculation of the eigenvalues and eigenvectors in Python is fairly easy. The main built-in function in Python to solve the eigenvalue/eigenvector problem for a square array is the eig function in numpy.linalg. Let’s see how we can use it.

TRY IT Calculate the eigenvalues and eigenvectors for matrix \(A = \begin{bmatrix} 0 & 2\\ 2 & 3\\ \end{bmatrix}\) .

TRY IT! Compute the eigenvalues and eigenvectors for matrix \(A = \begin{bmatrix} 2 & 2 & 4\\ 1 & 3 & 5\\ 2 & 3 & 4\\ \end{bmatrix}\) .

numpy.linalg.eig #

Compute the eigenvalues and right eigenvectors of a square array.

Matrices for which the eigenvalues and right eigenvectors will be computed

The eigenvalues, each repeated according to its multiplicity. The eigenvalues are not necessarily ordered. The resulting array will be of complex type, unless the imaginary part is zero in which case it will be cast to a real type. When a is real the resulting eigenvalues will be real (0 imaginary part) or occur in conjugate pairs

The normalized (unit “length”) eigenvectors, such that the column eigenvectors[:,i] is the eigenvector corresponding to the eigenvalue eigenvalues[i] .

If the eigenvalue computation does not converge.

eigenvalues of a non-symmetric array.

eigenvalues and eigenvectors of a real symmetric or complex Hermitian (conjugate symmetric) array.

eigenvalues of a real symmetric or complex Hermitian (conjugate symmetric) array.

Similar function in SciPy that also solves the generalized eigenvalue problem.

Best choice for unitary and other non-Hermitian normal matrices.

New in version 1.8.0.

Broadcasting rules apply, see the numpy.linalg documentation for details.

This is implemented using the _geev LAPACK routines which compute the eigenvalues and eigenvectors of general square arrays.

The number w is an eigenvalue of a if there exists a vector v such that a @ v = w * v . Thus, the arrays a , eigenvalues , and eigenvectors satisfy the equations a @ eigenvectors[:,i] = eigenvalues[i] * eigenvectors[:,i] for \(i \in \{0,...,M-1\}\) .

The array eigenvectors may not be of maximum rank, that is, some of the columns may be linearly dependent, although round-off error may obscure that fact. If the eigenvalues are all different, then theoretically the eigenvectors are linearly independent and a can be diagonalized by a similarity transformation using eigenvectors , i.e, inv(eigenvectors) @ a @ eigenvectors is diagonal.

For non-Hermitian normal matrices the SciPy function scipy.linalg.schur is preferred because the matrix eigenvectors is guaranteed to be unitary, which is not the case when using eig . The Schur factorization produces an upper triangular matrix rather than a diagonal matrix, but for normal matrices only the diagonal of the upper triangular matrix is needed, the rest is roundoff error.

Finally, it is emphasized that eigenvectors consists of the right (as in right-hand side) eigenvectors of a . A vector y satisfying y.T @ a = z * y.T for some number z is called a left eigenvector of a , and, in general, the left and right eigenvectors of a matrix are not necessarily the (perhaps conjugate) transposes of each other.

G. Strang, Linear Algebra and Its Applications , 2nd Ed., Orlando, FL, Academic Press, Inc., 1980, Various pp.

(Almost) trivial example with real eigenvalues and eigenvectors.

Real matrix possessing complex eigenvalues and eigenvectors; note that the eigenvalues are complex conjugates of each other.

Complex-valued matrix with real eigenvalues (but complex-valued eigenvectors); note that a.conj().T == a , i.e., a is Hermitian.

Be careful about round-off error!

• Numerical Differentiation
• Applications

Eigenvalues and Eigenvectors

Let $A$ be a square matrix. A non-zero vector $\mathbf{v}$ is an eigenvector for $A$ with eigenvalue $\lambda$ if

$$ A\mathbf{v} = \lambda \mathbf{v} $$

Rearranging the equation, we see that $\mathbf{v}$ is a solution of the homogeneous system of equations

$$ \left( A - \lambda I \right) \mathbf{v} = \mathbf{0} $$

where $I$ is the identity matrix of size $n$. Non-trivial solutions exist only if the matrix $A - \lambda I$ is singular which means $\mathrm{det}(A - \lambda I) = 0$. Therefore eigenvalues of $A$ are roots of the characteristic polynomial

$$ p(\lambda) = \mathrm{det}(A - \lambda I) $$

scipy.linalg.eig

The function scipy.linalg.eig computes eigenvalues and eigenvectors of a square matrix $A$.

Let's consider a simple example with a diagonal matrix:

The function la.eig returns a tuple (eigvals,eigvecs) where eigvals is a 1D NumPy array of complex numbers giving the eigenvalues of $A$, and eigvecs is a 2D NumPy array with the corresponding eigenvectors in the columns:

The eigenvalues of $A$ are:

The corresponding eigenvectors are:

We can unpack the tuple :

If we know that the eigenvalues are real numbers (ie. if $A$ is symmetric), then we can use the NumPy array method .real to convert the array of eigenvalues to real numbers:

Notice that the position of an eigenvalue in the array eigvals correspond to the column in eigvecs with its eigenvector:

Symmetric Matrices

The eigenvalues of a symmetric matrix are always real and the eigenvectors are always orthogonal! Let's verify these facts with some random matrices:

Create the symmetric matrix $S = P P^T$:

Let's unpack the eigenvalues and eigenvectors of $S$:

The eigenvalues all have zero imaginary part and so they are indeed real numbers:

The corresponding eigenvectors of $A$ are:

Let's check that the eigenvectors are orthogonal to each other:

The dot product of eigenvectors $\mathbf{v}_1$ and $\mathbf{v}_2$ is zero (the number above is very close to zero and is due to rounding errors in the computations) and so they are orthogonal!

Diagonalization

A square matrix $M$ is diagonalizable if it is similar to a diagonal matrix. In other words, $M$ is diagonalizable if there exists an invertible matrix $P$ such that $D = P^{-1}MP$ is a diagonal matrix.

A beautiful result in linear algebra is that a square matrix $M$ of size $n$ is diagonalizable if and only if $M$ has $n$ independent eigevectors. Furthermore, $M = PDP^{-1}$ where the columns of $P$ are the eigenvectors of $M$ and $D$ has corresponding eigenvalues along the diagonal.

Let's use this to construct a matrix with given eigenvalues $\lambda_1 = 3, \lambda_2 = 1$, and eigenvectors $v_1 = [1,1]^T, v_2 = [1,-1]^T$.

Let's verify that the eigenvalues of $M$ are 3 and 1:

Verify the eigenvectors:

Matrix Powers

Let $M$ be a square matrix. Computing powers of $M$ by matrix multiplication

$$ M^k = \underbrace{M M \cdots M}_k $$

is computationally expensive. Instead, let's use diagonalization to compute $M^k$ more efficiently

$$ M^k = \left( P D P^{-1} \right)^k = \underbrace{P D P^{-1} P D P^{-1} \cdots P D P^{-1}}_k = P D^k P^{-1} $$

Let's compute $M^{20}$ both ways and compare execution time.

Let's use diagonalization to do the same computation.

Diagonalization computes $M^{k}$ much faster!

Under construction

Python Scipy Eigenvalues [7 Useful Examples]

In this Python tutorial , we will learn about “ Python Scipy Eigenvalues ” where we will know how to find eigenvalues and eigenvectors of the given matrix. And we will also cover the following topics.

• What is Eigenvalues
• Python Scipy Eigenvalues
• Python Scipy Eigenvalues Hermitian
• Python Scipy Eigenvalues Eigvals_only
• Python Scipy Eigenvalues Subset_by_value
• Python Scipy Eigenvalues Subset_by_index
• Python Scipy Eigenvalues and Eigenvectors

Also, check how to install Scipy in Python: Installation of Scipy

Table of Contents

In the context of the system of linear equations, eigenvalues refer to a unique set of scalars. Most often, matrix equations use it. The word “Eigen” in German implies “proper” or “characteristic.” Eigenvalues can therefore also be referred to as appropriate values, latent roots, characteristic values, and characteristic roots.

• The eigenvalue is a scalar that is used to alter the eigenvector, In simple words. The fundamental formula is Ax = λx , A’s eigenvalue “λ” is an integer or scalar value.

Let’s also know about the term ‘eigenvector’ which is related to eigenvalues.

An eigenvector in mathematics is equivalent to real non-zero eigenvalues that point in the direction extended by the transformation, whereas an eigenvalue is thought of as a factor by which it is stretched. The transformation’s direction is reversed if the eigenvalue is negative.

The non-zero vectors known as eigenvectors remain in the same direction after applying any linear transformation. Only one scalar factor is changed. If A is a linear transformation from vector space V and x is a vector there that is not zero, then v is an eigenvector of A if A(X) is a scalar multiple of x.

An eigenspace for a given vector x is made up of all the eigenvectors that collectively have an identical eigenvalue to the zero vector. The zero vector is not, however, an eigenvector.

• Let’s assume that A is an “nxn” matrix and that is an eigenvalue of matrix A . If x , a non-zero vector, matches the given expression below, it is said to be an eigenvector.

Its eigenvector is x . A is the same as the eigenvalue(λ).

In this tutorial, we will learn about how to use the method of Python Scipy to compute the eigenvalues and eigenvectors of the given array or matrix.

Also, read: Scipy Optimize – Helpful Guide

The method eigvals() of Python Scipy exists in a module scipy.linalg() that Identifies the eigenvalues in a regular or generalized eigenvalue problem.

The syntax is given below.

Where parameters are:

• a(array_data,(M,M)): A real or complex matrix whose eigenvalues and eigenvectors have to be determined.
• b(array_data,(M,M)): Matrix on the right side of a generalised eigenvalue issue. The identity matrix is deemed to exist if removed.
• check_finte(boolean): Whether or not to make sure that the input matrices only have finite numbers. If the inputs actually contain infinities or NaNs, disabling them could improve performance but cause issues (non-termination, crashes).
• overwrite_a(boolean): which data should be overwritten in a.
• homogeneous_eigvals(boolean): In the case when True, give the eigenvalues in homogeneous coordinates.

The method eigvals() returns w (The eigenvalues, which are not in any particular order but are each repeated according to their multiplicity. Unless homogeneous eigvals=True, the shape is (M,)) of type complex ndarray or double.

Let’s take an example by following the below steps:

Import the required libraries using the below python code.

Create an array or matrix using the below code.

Now compute the eigenvalues of the above-created matrix using the below code.

In the above output, the eigenvalues of the matrix are [-1.+0.j, 1.+0.j] .

This is how to compute the eigenvalues from a given matrix using the method eigvals() of Python Scipy.

Read: Scipy Rotate Image + Examples

First, we need to know “What is the Hermitian matrix?” A square matrix, which is the same as its conjugate transpose matrix, is a hermitian matrix. A hermitian matrix’s nondiagonal components are all complex integers. A hermitian matrix’s complex numbers are set up so that the ith row and jth column’s element are the complex conjugates of the jth row and ith column’s element.

If A = A T , then matrix A is a hermitian matrix. Similar to a symmetric matrix, a hermitian matrix differs from one in that the components of its non-principal diagonal are complex numbers.

The Python Scipy has a method eigh() within the module scipy.linalg to deal with standard ordinary eigenvalue problems for real symmetric or Hermitian matrices.

• a(array_data): The computation of the eigenvalues and eigenvectors of a complex Hermitian or real symmetric matrix.
• b(array_data): A real symmetric, complex Hermitian, positive matrix. The identity matrix is presumed if missing.
• lower(bool): Whether the triangles in lower or upper thirds of a, and, if appropriate, b, are where the relevant array data is obtained from. Lower by default
• egvals_only(boolean): If just eigenvalues should be calculated and not eigenvectors. (by default: both are computed)
• overwrite_a(boolean): To overwrite a .
• overwrite_b(boolean): To overwrite b .
• check_finite(boolean): If it is necessary to verify that the input matrices only contain finite numbers. Disabling may improve performance, but if the inputs do contain infinities or NaNs, it may cause issues (crashes, non-termination).
• subset_by_index(iterable): This two-element iterable, if given, specifies the half-open interval (a, b] within which, if any, only the eigenvalues between these values are returned. exclusive to “evr,” “evx,” and “gvx” drivers. For the unconstrained endpoints, use i9 bnp.inf.]-0965\
• subset_by_value(iterable): To define the half interval to get only eigenvalues using a two-element that is iterable.

The method eigh() returns the w (selected eigenvalues) in increasing size of type ndarray.

Let’s understand with an example by following the below steps:

Import the required libraries using the below code.

Create an array of data as a matrix using the below code.

Pass the created matrix data to the method eigh() using the below code.

The output shows both eigenvalues and eigenvectors of the given matrix. What will happen, if we need only eigenvalues and no eigenvectors. The next sub-section is all about it.

Read: Scipy Stats – Complete Guide

The Python Scipy method eigh() returns both eigenvalues and eigenvectors, sometimes we need only one value like eigenvalues. To get only eigenvalues, the method eigh() has a parameter eigvals_only of type boolean or it accepts the True or False value.

If we set the eigvals_only equal to True , then it returns only the eigenvalues, otherwise returns both eigenvalues and eigenvectors.

Now understand with example by following the below steps:

Import the required libraries or methods using the below python code.

Create a matrix containing values by using the below code.

To compute and get only eigenvalues, then set the parameter eigvals_only to True using the below code.

When we pass the matrix to a method eigh() with a parameter eigvals_only equal to True, as we can see in the output, the method returns only the eigenvalues of the matrix.

This is how to compute the eigenvalues of the given matrix using the method eigh() of Python Scipy.

Read: Python Scipy FFT

The subset_by_value is another parameter of method eigh() to inquire about eigenvalues that are under a specific range. For instance, if we need eigenvalues higher than 5, or lower than 8, then the method returns all the eigenvalues higher than 5, or lower than 8.

Let’s see with an example by following the below steps:

Generate a matrix of data using the method np.array() as shown in the below code.

Now pass the above matrix to a method eigh() with a parameter subset_by_value equal to [-np.inf, 5] , to get eigenvalues less than 5 only.

Again change the value of the parameter subset_by_value to [10, 20] , to get the eigenvalues between 10 and 20 using the below code.

This is how to get the specific range of eigenvalues using the method eigh() with parameter subset_by_value of Python Scipy.

Read: Scipy Linalg – Helpful Guide

We already know that method eigh() returns the as ndarray type, and we also know that the array elements or values can be accessed by its index value. So the method eigh() has a parameter subset_by_index that allows us to access the eigenvalues or eigenvectors of the ndarray using its index value.

Now we are going to understand, how we can use the parameter subset_by_index with help of an example.

Now pass the above matrix to a method eigh() with a parameter subset_by_index equal to [0, 2] , to get eigenvalues from index 0 to 2.

Read: Scipy Stats Zscore + Examples

The Python method eig() that exist in a module scipy.linalg identify and resolve a square matrix’s ordinary or generalized eigenvalue problem.

Where parameters are :

• a(array_data): A real or complex matrix whose eigenvalues and eigenvectors have to be determined.
• b(array_data): To input the right-hand side matrix.
• right(boolean): An extended eigenvalue problem’s right-side matrix. The identity matrix is presumed when None is the default.
• left(boolean): Whether to calculate the left eigenvectors and return them. False is the default.
• check_finite(boolean): To check if the provided matrix as input has finite numbers.
• homogeneous_eigvals(boolean): To get the eigenvalues in homogenous coordinates

The method eig() returns the w (the eigenvalues) , vr (the right vector that is normalized) and vl (the left vector that is normalized) of type complex ndarray or double

Generate matrix data using the below code.

Now compute the eigenvalues and eigenvectors of the above-created matrix using the below code.

This is how to compute the eigenvalues and eigenvectors of the given matrix using the method eig() of Python Scipy.

You may also like to read the following Python SciPy tutorials.

• Scipy Ndimage Rotate
• Python Scipy Gamma 
• Python Scipy Stats Norm
• Python Scipy Kdtree
• Scipy Normal Distribution
• Scipy Integrate + Examples
• Python Scipy Stats Mode

So, in this tutorial, we have learned about “ Python Scipy Eigenvalues ” and covered the following topics.

I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile .

• IMSL Library for Python 1.0 documentation
• Eigenvalue Problems ( imsl.eigen )

Previous topic

Error Analysis and Accuracy

imsl.eigen.eig_gen

Reformulating Generalized Eigenvalue Problems ¶

The eigenvalue problem Ax = λ Bx is often difficult for users to analyze because it is frequently ill-conditioned. Occasionally, changes of variables can be performed on the given problem to ease this ill-conditioning. Suppose that B is singular, but A is nonsingular. Define the reciprocal \(\mu = \lambda^{-1}\) . Then assuming A is definite, the roles of A and B are interchanged so that the reformulated problem Bx = μ Ax is solved. Those generalized eigenvalues \(\mu_j=0\) correspond to eigenvalues \(\lambda_j=\infty\) . The remaining \(\lambda_j=\mu_j^{-1}\) . The generalized eigenvectors for \(\lambda_j\) correspond to those for \(\mu_j\) .

Now suppose that B is nonsingular. The user can solve the ordinary eigenvalue problem Cx = λ x, where \(C=B^{-1}A\) . The matrix C is subject to perturbations due to ill-conditioning and rounding errors when computing \(B^{-1}A\) . Computing the condition numbers of the eigenvalues for C may, however, be helpful for analyzing the accuracy of results for the generalized problem.

There is another method that users can consider to reduce the generalized problem to an alternate ordinary problem. This technique is based on first computing a matrix decomposition B=PQ , where both P and Q are matrices that are “simple” to invert. Then, the given generalized problem is equivalent to the ordinary eigenvalue problem Fy = λ y. The matrix \(F=P^{-1}AQ^{-1}\) and the unnormalized eigenvectors of the generalized problem are given by \(x=Q^{-1}y\) .

• © Copyright 2014-2016, Rogue Wave Software.
• Created using Sphinx 1.2.3.

• SciPy v0.19.1 Reference Guide
• Linear algebra ( scipy.linalg )

scipy.linalg.eigh ¶

Solve an ordinary or generalized eigenvalue problem for a complex Hermitian or real symmetric matrix.

Find eigenvalues w and optionally eigenvectors v of matrix a , where b is positive definite:

Previous topic

scipy.linalg.eigvals

scipy.linalg.eigvalsh

• © Copyright 2008-2016, The Scipy community.
• Last updated on Jun 21, 2017.
• Created using Sphinx 1.5.3.