Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Tensorflow take ages for tf.cond and eval() - python code (sorry but i asked on Stackoverflow but none answer me)

i got a problem with tensorflow and need your help. My need is calculating tensordot between a vector: 1x512 named "face" in my code and a faces data: N x 512 named "input_faces_data&...
0
votes
0answers
4 views

Aggregating embeddings at the token level in BERT

I'm learning BERT and I wonder how to aggregate embeddings on the token level. For each token in the corpus vocabulary, I would like to create a list of all their contextual embeddings and average ...
0
votes
0answers
6 views

Why is the behaviour of groupby command different in R and in python with pandas?

if I execute the following lines of codes in R as well as python for groupby command, the order of the resulting ...
1
vote
0answers
16 views

Using Random Forests as a GAM

This is a regression problem. I have a dataset of sales of various products overtime. I have three kind of feature sets : Price features, ...
0
votes
0answers
20 views

Pandas - Rename all Unnamed columns based on other column names in Excel [closed]

I'd like to rename the 'Unnamed' columns I get by reading and Excel file through Pandas with other names that are found some rows below: I used ...
0
votes
0answers
9 views

how to extract specific phrases using negspacy

When I apply negspacy to my sentence, I want negspacy to consider specific phrase as a single entity and give me the output for it. ...
0
votes
0answers
19 views

ValueError: Data cardinality is ambiguous while adopting Keras Seq 2 Seq model example! [closed]

So basically I am trying to adopt the Keras seq 2 seq model example to my own code.(this can be found at https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html) ...
-3
votes
0answers
28 views

how to train a model using many data sets in machine learning and obtain a final model [closed]

I want to apply auto-encoder for de-noising the time-series data. So I have some doubts in my mind that I need clarifications. I have 100 training data sets and I just want to train a supervised model ...
0
votes
0answers
20 views

Custom Transformers (Class) not working together in ColumnTransformer while do separately [closed]

I'm preprocessing a dataframe of different films. I have the following transformers (scikit-learn classes): ...
1
vote
1answer
20 views

What does it mean when your NN model has “multiple” inputs or outputs in Keras?

I'm reading the guide on Keras website: https://keras.io/guides/sequential_model/ And it says don't use a sequential NN if you have multiple inputs or outputs. What is the difference between multiple ...
0
votes
1answer
19 views

How to play with .cat.codes and get_dummies

I may be lost here and this might be a stupid question. I know that get_dummies returns multiple columns depending upon the number of variables and ...
0
votes
0answers
28 views

Calculating error in each layer of neural network

I am referring Andrew Ng's course to implement neural network https://www.youtube.com/watch?v=x_Eamf8MHwU&t. In this course bias is taken single matrix with weights. I got a error and I am not ...
0
votes
0answers
18 views

How to implement in real life Recommendation Algorithms [closed]

I took a few course online on machine learning, the examples were good I have seeing many tutorials and I think I get the basics. But I have a few doubts on how to implement this in real life. I did ...
0
votes
1answer
17 views

Randomforest parameter tuning R(caret) and Python(scikit learn)?

Question: is it possible, or even necessary, to perform a cross-validation check to tune the parameters of a Python randomforest implementation (eg scikit learn) when training a new model, as can be ...
0
votes
1answer
36 views

How can I do a train test split for an unbalanced panel data set in Python?

I have an unbalanced, panel pandas data frame. I would like to split this data into a training set and a testing set. Python's train_test_split method will not work ...
0
votes
0answers
9 views

Pruning input layers and output layers after training model with shared layers

I'm trying to train a search and item encoder and this is the model I have ...
0
votes
0answers
7 views

Keras Predictions as a list of file names for each class

I've created a model using Keras, I have trained it with a training and validation set, and have used a test set filled with random number of images for each class. My test set consists of one folder ...
0
votes
0answers
38 views

How to use historic data (granularity at day level) for ML modeling?

There is a scenario where I have to use historic data which is at the day level for the past 5 years.Actually it is water flow data, what quantity of water was flown on that particular day. I have to ...
0
votes
0answers
36 views

ConvNet - What to improve regarding architecture, procedure and technique?

I have a dataset of 180k images of license plates (so, not necessary to localize the license plate at first) for which I try to recognize the characters on the images (License plate recognition). All ...
1
vote
0answers
55 views

Assign more importance to recent records during training

My goal is to build a classification model in order to predict if a customer will buy a product or not (binary classification). Since in the last months (let's say 3-4) I know that the advertising of ...
0
votes
0answers
20 views

assigning element to the cluster generating the highest SI score [closed]

let's say we have a dataframe consists of 4 elements stored ([0.85142858], [0.85566274], [0.85364912], [0.81536489]) with their cluster labels (0, 1, 2) and the results of clustering algorithm like ...
0
votes
1answer
33 views

Single image feature reduction at inference time : SVM

I am trying to train a SVM classifier using scikit-learn.. At training time I want to reduce the feature vector dimension. I have used PCA to reduce the dimension. ...
0
votes
1answer
16 views

How to combine the features extracted from different CNN architectures? [closed]

I want to combine the features extracted from different CNN architectures into my fully connected layer. How to proceed?
0
votes
2answers
19 views

Consolidating multivariate time-series information from many data sets

I am having trouble setting up a problem with regards to time series analysis. I have 30 data sets, where each set corresponds to a certain project. Each project has 7 features, and each feature has ...
1
vote
1answer
41 views

Can mini-batch gradient descent outperform batch gradient descent? [duplicate]

As I was reading and going through the second course of Andrew Ng's deep learning course, I came across a sentence that said, With a well-turned mini-batch size, usually it outperforms either ...
0
votes
1answer
17 views

MLP classifier Gridsearch CV parameters to tune?

I'm looking to tune the parameters for sklearn's MLP classifier but don't know which to tune/how many options to give them? Example is learning rate. should i give it[.0001,.001,.01,.1,.2,.3]? or is ...
0
votes
2answers
41 views

Beautifulsoup find_all returns TypeError: list indices must be integers or slices, not str

This should have an easy solution but I can't understand how to avoid it. ...
0
votes
2answers
26 views

Feature scaling for MLP neural network sklearn

I am working with a dataset that has multiple scales for my features. Before running sklearn's MLP neural network I was reading around and found a variety of different opinions for feature scaling. ...
0
votes
1answer
22 views

TensorFlow stopped working after installing opencv

I am working on projects that use TensorFlow and Keras and I did the hello-world MNIST project successfully and later I tried to run some other project and TF suddenly stopped working. Now it doesn't ...
0
votes
0answers
23 views

Entity Recognition to extract question,options and diagrams from a question paper? [duplicate]

Here is a sample question paper. I want to extract questions, their options and diagrams for the corresponding question(if any). Thinking of doing entity recogntion for the same. But I am not sure if ...
1
vote
0answers
21 views

Regarding Keras mean_squared_error losses

I am trying to do a RandomizedSearchCV for a simple regression task. Essentially two inputs to one output. Upon inspecting the resulting models, it appears that the 'best' model has a ...
-1
votes
0answers
32 views
-1
votes
0answers
22 views

Where to learn maths to start learning data science? [closed]

Hope you all are doing great. I would love to do lower-end kaggle competitions in the coming months. In high school we have not learnt any advanced math yet. I know python well. Where would you ...
0
votes
0answers
19 views

Plotting scikit-learn confusion matrix returns no values in the last class

I am attempting to create a confusion matrix using Scikit-Learn for a multiclass classification CNN, and it works well except for the fact that it does not provide ...
0
votes
2answers
37 views

Don't understand why I get an inverse ROC curve for SVM (Python)

I build an SVM classifier but get an inverse ROC curve. The AUC is only 0.08. I've used the same datasets to build a Logistic Regression classifier and a Decision Tree classifier, and the ROC curves ...
-1
votes
0answers
17 views

Increasing accuracy of classification using linear classifier

I have a dataset with around 497k entries. I have to predict the value of 'Level 1' based on 'Product Description'. With the code below, I get 87% accuracy. How can I increase it? I have attached 2 ...
0
votes
1answer
29 views

Wrong result when solving: “chance that two random cards differ in color and value?”

I'm trying to build a simulation for this question: "There are 50 cards of 5 different colors. Each color has cards numbered between 1 to 10. You pick 2 cards at random. What is the probability ...
2
votes
1answer
140 views

Twitter POS and NER: What is state-of-the-art?

What is the current state-of-the-art for pos tagging and named entity recognition for twitter data? Are industrial-strength programs like Spacy and ...
1
vote
1answer
22 views

Population Stability Index(PSI) [closed]

Can we calculate PSI value if there is no training data? if we can please suggest me the steps how can we do it.
0
votes
0answers
37 views

Text classification of an imbalanced dataset

I have a dataset with size ~ 500k entries. There are 2 columns, 'product description' and 'level 1'. I am developing my model such that it learns from a training set of 350k and based on the product ...
0
votes
0answers
28 views

Semantic network using word2vec

I have thousands of headlines and I would like to build a semantic network using word2vec, specifically google news file. My sentences look like ...
0
votes
0answers
22 views

Python Machine learning - Predictions based on several inputs

I have in total three columns of data in a dataframe, lets say column_a, column_b and column_res with 40 datapoints in each column. Is it possible to train a LSTM model based on column_a and column_b ...
0
votes
1answer
29 views

Python writing to Excel file: writerow() takes no keyword arguments

I have this script: ...
0
votes
0answers
27 views

How to use a condition in rolling window function in pandas?

I am trying to calculate number of items sold for a category in the past. I have multiple conditions on which I want to do this: For example: Get the number of items sold in last 10 days, if none ...
0
votes
0answers
21 views

Finding relationships between multiple readings from an IoT sensor network time-series data

I am trying to find possible relationships between a large sensor network (around 100 sensors) where data is recorded in daily intervals. Each sensor has multiple readings simultaneously (Value_X). ...
0
votes
0answers
21 views

Is there a way to calculate the degree of classification of data points in a set?

I'm trying to write a program in python that takes in list of all stocks on different exchanges and filters them out according to multiple criteria (e.g. marketcap has to be larger than 500m, PE ratio ...
0
votes
1answer
16 views

Error when trying to read multiple csv files in jupyter notebook [closed]

I am given a dataset of 1000 csv files (data0,data1,.....,data999) and I tried to read those files and transpose them and then append. But I am getting an error
0
votes
0answers
34 views

Image segmentation network to extract questions from an image of a test paper?

This is the sample document -> I want to extract questions along with the options. There are other question papers as which have questions with diagrams in them. I want to be able to extract them ...
0
votes
0answers
9 views

statsmodels: significance test (claim test), one sample, mean value - any function available?

We sample the bacteria contents of drinking water. Fact: when bacteria contents > 88, water is not safe to drink. These are the samples we've obtained: ...

1
2 3 4 5
94