Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
data = pd.read_csv('file.csv', sep='¬', encoding = 'ISO-8859-1', engine='python', on_bad_lines='warn', nrows=2845060)
Issue Description
Read_csv with on_bad_lines AND nrows at the same time appears to not follow the correct on_bad_lines logic. When I run the above example without the nrows argument, I get the following warning:
Skipping line 2845058: '¬' expected after '"'
But running with nrows argument included, it errors out with the following message: (same happens when I use on_bad_lines='skip' instead of 'warn')
Error: '¬' expected after '"'
Expected Behavior
The on_bad_lines logic should continue to work when using nrows so I should see the 'Skipping line' warning.
Installed Versions
Comment From: lithomas1
Please provide a sample file so that we can reproduce this.
Also, this is probably expected for now. (see #50311). Discussion is probably needed before changing this.
Comment From: phofl
Closing, please ping to reopen when you can provide an example