Original title: Pandas read_csv: Skip rows contains invalid data that can cause data_type parsing errors
The csv file can contain string values to certain integer columns and i want to ignore/handle via callback if that happens, tried using on_bad_lines=‘skip/warn’ however it gets triggered only on wherever there are parsing issues due to delimiter/num of columns mismatch. Here is the code i am using csv: id,name,age 1,,“25” 2,"",30 jj,John,
import pandas as pd schema = {‘id’:‘int64’,’name’:‘str’,‘age’:‘int’} df = pd.read_csv(‘a.csv’, dtype=schema, on_bad_lines=‘warn’, engine=‘python’)