SciPy Sparse Data

Why Sparse Matrices?

Some matrices that show up in real applications - graphs of web pages, user-item rating tables, finite-element meshes - are enormous but almost entirely zeros. Storing every one of those zeros in a regular NumPy array wastes memory and wastes time on arithmetic that always produces zero. scipy.sparse stores only the nonzero values and enough bookkeeping to know where they belong.

CSR: Compressed Sparse Row

A CSR matrix (csr_matrix) organizes its nonzero values row by row. Internally it keeps three arrays: data (the nonzero values, in row order), indices (the column each value belongs to), and indptr (where each row's slice of data starts and ends). This layout makes row access and matrix-vector multiplication fast.

from scipy.sparse import csr_matrix
import numpy as np

dense = np.array([
    [0, 0, 3],
    [4, 0, 0],
    [0, 5, 0]
])

sparse = csr_matrix(dense)

print(sparse.data)     # [3 4 5]
print(sparse.indices)  # [2 0 1]  (column of each value)
print(sparse.indptr)   # [0 1 2 3]  (row boundaries into data/indices)

CSC: Compressed Sparse Column

A CSC matrix (csc_matrix) stores the same kind of information but organized column by column instead of row by row. It uses the same three arrays - data, indices, indptr - except indices now holds row positions and indptr marks column boundaries. CSC tends to be faster for column slicing and is what many sparse linear-algebra solvers expect internally.

from scipy.sparse import csc_matrix

csc = csc_matrix(dense)

print(csc.data)     # [4 5 3]
print(csc.indices)  # [1 2 0]  (row of each value)
print(csc.indptr)   # [0 1 2 3]  (column boundaries into data/indices)

Sparse matrices support many of the same operations as NumPy arrays - addition, multiplication, slicing - but always keep an eye on .count_nonzero() and the overall shape, since some operations (like naive elementwise division) can silently make a sparse matrix dense and undo the memory savings.

from scipy.sparse import csr_matrix

sparse = csr_matrix(dense)

print(sparse.count_nonzero())  # 3
print(sparse.toarray())        # back to a regular NumPy array