8
$\begingroup$

I have column 'ABC' which has 5000 rows. Currently, dtype of column is object. Mostly it has string values but some values dtype is not string, I want to find all those rows and modify those rows. Column is as following:

1 abc
2 def
3 ghi
4 23
5 mno
6 null
7 qwe
8 12-11-2019
...
...
...
4900 ert
5000 tyu

In above case, I can use for loop to find out rows which do not have desired dtype. I just wanted to know, is their better way to solve this issue.

Note: I am using Pandas.

Improve this question
$\endgroup$
3
  • $\begingroup$ In pandas dtypes can be inferred by trying to cast them and making un-castable ones to string dtypes as in object, which means all elements in a single column will be in a same datatype. You cant have two diff. row elements in the same column to be of different datatypes. $\endgroup$
    – Kiritee Gak
    Sep 28, 2019 at 15:45
  • $\begingroup$ @KiriteeGak: I think that is not quite true. You can test that yourself. Create a dataframe, with at least two rows indexed 1 and 2. Then do df.loc[1, 'new_column']= 'my_value'. Then do df['new_column'].map(type). You will see, that all but the first row contain floats. That is because the other rows contain NaN, which is a float and not a str. Likewise you could mix in other object types in your object column if you like (but it is probably not a very good idea). $\endgroup$
    – jottbe
    Sep 28, 2019 at 15:54
  • 1
    $\begingroup$ I stand corrected. Thanks :) $\endgroup$
    – Kiritee Gak
    Sep 28, 2019 at 16:39

1 Answer 1

Reset to default
12
$\begingroup$

You can get the type of the entries of your column with map:

df['ABC'].map(type)

So to filter on all values, which are not stored as str, you can use:

df['ABC'].map(type) != str

If however you just want to check if some of the rows contain a string, that has a special format (like a date), you can check this with a regex like:

df['ABC'].str.match('[0-9]{4}-[0-9]{2}-[0-9]{2}')

But of course, that is no exact date check. E.g. it would also return True for values like 0000-13-91, but this was only meant to give you an idea anyways.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your method is so powerful! I tested with this: r = [pd.to_datetime('12-11-2019'),1.0,'23',1,2,3,'Hello','2019-11-12'] and returns: 0 <class 'pandas._libs.tslibs.timestamps.Timesta... 1 <class 'float'> 2 <class 'str'> 3 <class 'int'> 4 <class 'int'> 5 <class 'int'> 6 <class 'str'> 7 <class 'str'> $\endgroup$
    – rubengavidia0x
    Mar 8, 2022 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.