I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k or 1 laksh record data set not at all giving at any results in console even though it is taken more than 24 hours time ? please suggest will pipeline model taking to execute on huge data for text classification ? i am not getting that what was the problem with pipeline text classification .

tfidfVector = TfidfVectorizer(tokenizer = textProfilerObj.textTokenizer)

    # Create the  pipeline  model to clean, tokenize, vectorize, and classify using tfidVector

    pipe = Pipeline([("cleaner", TextCleaner()),('vectorizer', tfidfVector),('classifier', modelObj)])

    #fit the data using pipeline  


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.