Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import dask.dataframe as dd
engine = create_engine("mssql+pyodbc://{user}:{pw}@{sv}:1433/{db}?driver=ODBC+DRIVER+17+for+SQL+Server"
.format(user="sa",
pw="sa",
db="table",
sv='server.com'), pool_pre_ping=True)
# Get the list of CSV files in the folder
csv_files = [f for f in os.listdir(dir_folder) if f.endswith('.csv')]
# Loop through the CSV files in the folder
for file in csv_files:
# Read the CSV file into a Dask DataFrame
df = dd.read_csv(f'{dir_folder}/{file}', usecols = cols, dtype = converters)
# Fill empty rows with 0
df = df.fillna(0)
# Rename the columns
df = df.rename(columns = new_column_names)
# Write the DataFrame to the SQL table
df.to_sql(table, engine, if_exists='append', index=False)
Issue Description
Issue related to connection, but as far as I know, I need t create the connection and send it through to_sql.
Expected Behavior
It supposed to connect the database and load the dataframe
Installed Versions
Comment From: phofl
Can you post in #40686?
Comment From: jbrockmendel
isnt this a dask issue?
Comment From: leo-schick
@jbrockmendel no, it is a issue with pandas not supporting SQLAlchemy>=2.0 yet
Comment From: jlonge4
@Edimo05 @phofl does this still exist?
Comment From: phofl
Dask only supports strings as uri, no engines. So this works as expected
Comment From: jlonge4
@phofl thanks! I dug through and couldn't think of a single reason why not so that works 😂