# Multiprocessing in KNIME Python Nodes possible?

I am wondering if it is possible to enable multiprocessing in KNIME Python nodes?

This is not working:

import multiprocessing as mp

from multiprocessing import Pool

def parallelize_dataframe(df, func):
df_split = np.array_split(df, num_partitions)
pool = Pool(num_cores)
df = pd.concat(pool.map(func, df_split))
pool.close()
pool.join()
return df

if __name__ == '__main__':
# the preprocess function does some stuff of string manipulation,
# and could be any other function
output_table = parallelize_dataframe(input_table, preprocess)

• Welcome to the site! You showed that you imported the Pool from multiprocessing, but how are you using the Pool? If you include more code in the #code ..... section, we might be able to give you a better answer. Aug 10, 2018 at 18:54
• You are right! I included some code for better understanding of my question. The code above is working in standard python but not in Python KNIME nodes. Any solutions or workarounds available? Aug 11, 2018 at 11:35