Pandas Cleaning Wrong Data
Wrong data is technically present but factually impossible or out of range, and loc[] is the standard way to inspect, fix, or remove it.
What Counts as 'Wrong' Data
Wrong data passes every dtype check — an age of 190 is a perfectly valid integer — but it is still impossible in the real world. These values usually come from data entry mistakes, unit mismatches, or sensor glitches, and describe() is often the first place you'll spot them, hiding in an unrealistic min, max, or std.
Example
import pandas as pd
data = {
'Customer': ['Ann', 'Ben', 'Cleo', 'Dev', 'Faye'],
'Age': [34, 29, 190, 41, -5]
}
df = pd.DataFrame(data)
print(df.describe())
# min is -5 and max is 190 -- both are impossible agesFixing Individual Values with loc
Once you know which row and column hold a wrong value, df.loc[row_label, 'column'] lets you read or overwrite that exact cell. loc also accepts a boolean condition in place of a row label, so you can fix every row that matches a rule in a single line instead of looping.
Example
import pandas as pd
data = {
'Customer': ['Ann', 'Ben', 'Cleo', 'Dev', 'Faye'],
'Age': [34, 29, 190, 41, -5]
}
df = pd.DataFrame(data)
# Fix one specific cell
df.loc[2, 'Age'] = 19
# Fix every row that fails a sanity check
df.loc[df['Age'] < 0, 'Age'] = df['Age'].median()
print(df)Removing Rows Instead of Fixing Them
Sometimes there is no reasonable value to substitute — you can't know what a customer's real age was supposed to be if 190 was a random data entry error with no pattern. In that case it is more honest to remove the row than to invent a number, using drop() together with the index of the rows that fail your condition.
Example
import pandas as pd
data = {
'Customer': ['Ann', 'Ben', 'Cleo', 'Dev', 'Faye'],
'Age': [34, 29, 190, 41, -5]
}
df = pd.DataFrame(data)
bad_rows = df[(df['Age'] < 0) | (df['Age'] > 120)].index
df.drop(bad_rows, inplace=True)
print(df)- Fix the value with loc when you know (or can reasonably estimate) what it should have been
- Remove the row when no substitute value would be honest, and the row is a small fraction of the data
- Prefer fixing over removing when the other columns in that row are still valuable
- Always check how many rows are affected before deciding — df[condition].shape[0] tells you the scale of the problem
