Pandas.errors.mergeerror: incompatible merge dtype, dtype(‘o’) and dtype(‘o’), both sides must have numeric dtype

by

pandas.errors.MergeError: incompatible merge dtype

An incompatible merge dtype error message occurs in Pandas when you try to merge two dataframes with object (dtype(‘o’)) columns that have non-numeric values. In order to perform a successful merge operation in Pandas, both sides of the merge must have matching numeric dtypes, or one side should have a numeric dtype while the other side can be object.

Examples:

Let’s consider two dataframes, df1 and df2, with object columns:

import pandas as pd

df1 = pd.DataFrame({'A': ['foo', 'bar', 'baz'],
                    'B': [1, 2, 3]})

df2 = pd.DataFrame({'A': ['foo', 'bar', 'baz'],
                    'C': ['x', 'y', 'z']})

If you try to merge these two dataframes using the ‘A’ column, which is of dtype object on both sides, you will encounter the incompatible merge dtype error:

merged_df = pd.merge(df1, df2, on='A')

The above merge operation throws the following error:

MergeError: incompatible merge dtype;  and must be the same

To resolve this error, you can convert the object column to a numeric dtype using the appropriate conversion function available in Pandas. For example, if you know that the ‘B’ column in df1 and ‘C’ column in df2 are both numeric, you can convert them to int or float dtype:

df1['B'] = df1['B'].astype(int)
df2['C'] = df2['C'].astype(float)

Once the object columns are converted to numeric dtypes, you can perform the merge without any issues:

merged_df = pd.merge(df1, df2, on='A')

Here, the merge operation will result in a dataframe with columns ‘A’, ‘B’, and ‘C’ merged based on the ‘A’ column.

Leave a comment