2
$\begingroup$

I have a 12 GB .csv file and I am trying to filter rows based on some value in a column under "AppName"

Doing this in Pyspark:

display(df.filter(df.AppName=="23954ec32332dfgd"))

only yields 1 row as opposed to at least 1400 rows.

Any ideas ?

$\endgroup$
2
$\begingroup$

Can you try this:

df1 = df.filter(df.AppName=="23954ec32332dfgd")
print df1.count()
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.