Pandas Correlations

What Correlation Measures

A correlation coefficient is a single number between -1 and 1 that summarizes the linear relationship between two numeric variables. A value near 1 means the columns tend to rise together, a value near -1 means one rises as the other falls, and a value near 0 means there is little to no linear relationship between them. Correlation says nothing about the size or units of either variable, only how consistently they move relative to each other.

Computing Correlations with corr()

Calling df.corr() on a DataFrame computes the pairwise correlation between every pair of numeric columns and returns the result as a new DataFrame, commonly called a correlation matrix. Non-numeric columns are excluded automatically (pass numeric_only=True to be explicit on newer pandas versions), and by default pandas uses the Pearson method, which measures straight-line relationships.

import pandas as pd

df = pd.DataFrame({
    'study_hours': [2, 4, 5, 7, 8, 10],
    'sleep_hours': [8, 7, 7, 6, 5, 5],
    'exam_score': [55, 63, 68, 79, 85, 91]
})

corr_matrix = df.corr()
print(corr_matrix)

• Pearson (the default) measures straight-line, linear relationships and assumes roughly continuous numeric data.
• Spearman ranks the values first, so it captures any monotonic relationship (steadily increasing or decreasing), even if it isn't a straight line.
• Kendall also works on ranks but compares concordant and discordant pairs; it tends to be more robust on small samples with many tied values.

import pandas as pd

df = pd.DataFrame({
    'study_hours': [2, 4, 5, 7, 8, 10],
    'sleep_hours': [8, 7, 7, 6, 5, 5],
    'exam_score': [55, 63, 68, 79, 85, 91]
})

# Spearman correlation, useful for monotonic but non-linear relationships
print(df.corr(method='spearman'))

# Correlate every column against exam_score only
print(df.corrwith(df['exam_score']))

Reading a Correlation Matrix

A correlation matrix is always square and symmetric: the value in row A, column B equals the value in row B, column A, and every diagonal entry is exactly 1.0 because a column is perfectly correlated with itself. To read one, scan for cells far from 0 in either direction, ignore the diagonal, and remember that the upper and lower triangles repeat the same information, so you only need to inspect one half.

import pandas as pd
import numpy as np

df = pd.DataFrame({
    'study_hours': [2, 4, 5, 7, 8, 10],
    'sleep_hours': [8, 7, 7, 6, 5, 5],
    'exam_score': [55, 63, 68, 79, 85, 91],
    'coffee_cups': [1, 1, 2, 2, 3, 3]
})

corr_matrix = df.corr().round(2)

# Hide the redundant upper triangle since the matrix is symmetric
mask = np.triu(np.ones(corr_matrix.shape), k=1).astype(bool)
lower_triangle = corr_matrix.mask(mask)
print(lower_triangle)