I have the following dataset which is a .json file:

enter image description here

and I would like to get the first word for every string inside lista_asm, so I would like to get: jmp,push,uncomisd,...etc

what I am doing to do this is the following:

dataFrame['opcodes'] = dataFrame['lista_asm'].apply(lambda x:[i.split()[0] for i in x])

but it gives me back the following error message:

   3589             else:
   3590                 values = self.astype(object).values
-> 3591                 mapped = lib.map_infer(values, f, convert=convert_dtype)
   3593         if len(mapped) and isinstance(mapped[0], Series):

pandas/_libs/lib.pyx in pandas._libs.lib.map_infer()

<ipython-input-18-5506b5721bf1> in <lambda>(x)
----> 1 dataFrame['opcodes'] = dataFrame['lista_asm'].apply(lambda x:[i.split()[0].strip() for i in x])

IndexError: list index out of range

I don't understand what is wrong. Can somebody please help me?

[EDIT]Trying the code:

dataFrame['opcodes'] = dataFrame['lista_asm'].apply(lambda x:x[0].split(" ",2)[0])

and adding :

df = dataFrame[["opcodes", "semantic"]].copy()

I get:

enter image description here

what I would like to get is a list of the type [push,mov,..] and this for every row.

It seems like when I do x[0] it does not return the first element of the list, but returns the pharentesis, which is weird. Am I doing something wrong which I don't see?

My objective is to pre-process this dataset in order to feed features to my model, but I haveing hard times in doing so.


2 Answers 2


Your question is a bit confusing. So, as much as I understood from your examples, for each sample of list_asm, you want to extract the very first word from the string.

The thing you are doing wrong is treating the string as a list. That is, ['uncomisd xmm2, xmm2', 'jp 0x40', ...] is considered as a string by python, not a list.

Thus, you need to extract the strings from your list first, then you can't take the first words from all these strings.

To achieve that, you can use a regular expression to find all the strings that are inside of quotes '...'.

import pandas as pd
import re

# Read the file into dataframe
dataFrame = pd.read_json("dataset.json", lines=True)

# First extract the strings the take the first word of each string  
dataFrame['opcodes'] = dataFrame['lista_asm'].apply(lambda x: [i.split()[0] for i in re.findall("'([^']*)'", x)])

or modular form of the code would be:

import pandas as pd
import re

# Function to extract the first words from each string
def extractFirstWord(str):
    listOfWords = re.findall("'([^']*)'", str)
    return [i.split()[0] for i in listOfWords]

# Read the file into dataframe
dataFrame = pd.read_json("dataset.json", lines=True)

dataFrame['opcodes'] = dataFrame['lista_asm'].apply(lambda x: extractFirstWord(x))

The result:

enter image description here

  • $\begingroup$ Thank you for your answer. I have tried your suggestion, but I still get only the pharentesis. $\endgroup$
    – J.D.
    Commented Nov 11, 2020 at 11:43
  • $\begingroup$ Can you please provide the data by uploading it somewhere? at least some samples. $\endgroup$ Commented Nov 11, 2020 at 12:07
  • $\begingroup$ I have uploded it here: github.com/JHDOR/try $\endgroup$
    – J.D.
    Commented Nov 11, 2020 at 15:24
  • $\begingroup$ Are you sure, you are checking th right thing? Couse it works in my machine. Check the output and use the exact code I provided. $\endgroup$ Commented Nov 11, 2020 at 17:30
  • 1
    $\begingroup$ you are right. I have tried to rewrite the code from the biginning, and your code works. I don't know why this code is giving me a lot of problems, probably I have a lot to learn. $\endgroup$
    – J.D.
    Commented Nov 11, 2020 at 18:56

The problem is that you say "apply(lambda x:[i.split()[0] for i in x])"

As soon as you say apply, x is your list. So you can say following "apply(lambda x:x[0].split(" ", 2)[0])"

Meaning you say take first element in the list, than split on " " and in two parts. And than take the first word (part) with the last [0]

  • $\begingroup$ Thank you for the answer. But this returns just the first word for the first string of the list. I would like to get the first word for each string of the list for each row of the dataframe. (Sorry, I have posed the question not specifying really well this fact) $\endgroup$
    – J.D.
    Commented Nov 8, 2020 at 13:37
  • $\begingroup$ I see. Can you jsut add list comprehension my code and that should be it? $\endgroup$
    – Noah Weber
    Commented Nov 8, 2020 at 13:38
  • $\begingroup$ Sorry, my bad I misreaded the dataframe before. With your code, it returns only the pharentesis from the list, so '[', sorry I put ampther conlum close to that and misreaded. $\endgroup$
    – J.D.
    Commented Nov 8, 2020 at 14:02
  • $\begingroup$ I will edit the question, so it is clearer. $\endgroup$
    – J.D.
    Commented Nov 8, 2020 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.