0
$\begingroup$

I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k or 1 laksh record data set not at all giving at any results in console even though it is taken more than 24 hours time ? please suggest will pipeline model taking to execute on huge data for text classification ? i am not getting that what was the problem with pipeline text classification .

tfidfVector = TfidfVectorizer(tokenizer = textProfilerObj.textTokenizer)

    # Create the  pipeline  model to clean, tokenize, vectorize, and classify using tfidVector

    pipe = Pipeline([("cleaner", TextCleaner()),('vectorizer', tfidfVector),('classifier', modelObj)])

    #fit the data using pipeline  

    baypit=pipe.fit(self.trainX,self.trainY)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.