Questions tagged [python]

Python is a general-purpose, dynamic, strongly typed language with many 3rd-party libraries for data science applications.

0
votes
0answers
6 views

Autoencoder not overfitting data after large number of epochs and small number of samples

I am training a deep autoeocoder on numerical data, with python jupyter notebook. I have 17 samples, each with 534 values, and my auto encoder has all layers to 534, but even after 5,000 epochs, the ...
0
votes
1answer
18 views

in TensorFlow 2.0, what is the different between LSTM and LSTMCell objects?

I am trying to implement an RNN in TensorFlow 2.0 (beta1). Looking at the layer functions (inherited from Keras) I found: tf.keras.layers.LSTM and ...
2
votes
2answers
12 views

Does partial transfer learning require a lot of computer power?

I want to be sure my understanding of the problem is correct. I want to do image classification and current state of the art in my field is achieved by transfer learning with VGG16. Since image on ...
0
votes
0answers
6 views

Monte Carlo Dropout as Uncertainty predection

I am pretty new to Python and this board so I am not sure, if I am at the right place for my question since it doesn't include any code. If not so, please give my a hint for a better way/place to ask. ...
0
votes
0answers
3 views

PREDICTIONS ON ODI MATCHES [on hold]

using the following data I have attached below, how can I predict who will win in the next ODI match? I want to consider the data of last 5 years only i.e.2011 to 2016. https://www.kaggle.com/jaykay12/...
0
votes
0answers
6 views

How to compare genetic profiles or vcf files in Python?

I am wondered how I will compare different vcf files which contain genetic profile in Python? Each vcf file contains profile for a single entity/animal/human etc. I can read each vcf file into ...
0
votes
1answer
34 views

Building resume classifier based on keywords, what would be the steps & what are features and target in this case?

I went through this, as I have a similar requirement (at least what I think, correct me if wrong) of classifying resumes(.doc files) based on the profiles. I have a ...
0
votes
0answers
13 views

PCA giving separated results expected (jupyter sklearn)?

I'm a complete newbie to PCA and I have 3 sets of values which I want to plot with PCA. This is what I am using: ...
0
votes
2answers
21 views

How to fit a model to the V-shaped data?

I have a dataset of chromatic and monochromatic galaxy fluxes which looks like inverted V shape as follows: ...
0
votes
0answers
9 views

Is it possible to run sequential forward selection on a neural network?

I'm testing various models (Logistic regression, Random Forest, KNN, SVC, Neural Network) for a binary classification problem. So far I used both Sequential Forward Selection and Sequential ...
1
vote
1answer
22 views

Multiclass Regression for density prediction

This is my first question in the DS-community, so I'm happily willing to accept any kind of (meta-) advice :). I have time-series data for a set of users (~100), ...
2
votes
2answers
17 views

How do I find the count of a particular column, based on another column(date) using pandas?

I have a dataframe with 3 columns, such as SoldDate,Model and TotalSoldCount. How do I create a new column, 'CountSoldbyMonth' which will give the count of each of the many models sold monthly? <...
0
votes
0answers
13 views

keras custom metric function how to feed 2 model outputs to a single metric evaluation function

I have an CNN object detection model which has two heads(outputs) with tensor names 'classification' and 'regression'. I want ...
1
vote
1answer
24 views

conv net data retrieval on unseen class

I have build a conv net for image classification which work "well" Now I extract features from last fully connected layer and use it for image retrieval (find image most similar to my target image) ...
0
votes
1answer
22 views

ValueError: pos_label=1 is not a valid label: array(['N', 'Y'], dtype='<U1')

X = train_encoded_df.iloc[:, 1: ] y = train_encoded_df["Loan_Status"] print("Precision:",metrics.precision_score(y_test, y_pred)) My training data contains the ...
1
vote
1answer
30 views

Why does np.linalg.eig produce an opposite-signed eigenvector?

I am learning SVD by following this MIT course. In this video, the lecturer is finding the SVD for $$ \begin{pmatrix} 5 & 5 \\ -1 & 7 \end{pmatrix}, $$ which involves finding the ...
0
votes
1answer
14 views

How can I use different loss functions for images with different labels in tensorflow?

I have two types of images, meaning two different sets of possible labels. How can I use two different loss functions based on which image I am currently training on with a batch size of 1? Thanks !
0
votes
1answer
17 views

Predict using a saved regression model

I have trained an ANN model for a regression problem which takes 10 parameters as input and gives 1 output. After training, I saved the model as json and weights as a .h5 file using keras. Now I want ...
1
vote
1answer
75 views

is there a way to normalize [-3,1] to ${\begin{bmatrix} \dfrac{-3}{\sqrt{10}}\\ \dfrac{1}{\sqrt{10}}\\ \end{bmatrix}}$ with python?

I am learning SVD by following this MIT course The lecturer is trying to normalize a vector $${\begin{bmatrix} -3\\ 1\\ \end{bmatrix}}$$ to $${\begin{bmatrix} \dfrac{-3}{\sqrt{10}}\\ \dfrac{1}{\...
0
votes
1answer
23 views

Changing Timestamp format for Date-Time in Excel/Pandas/Python?

I have a excel data with time stamp format like this "2019-06-10T14:05:00+05:30" [YYYY-MM-DDTHH:MM:SS], I want it to be converted into "2019-06-10 14:05" [YYYY-MM-DD HH:MM], Is it possible to convert ...
0
votes
0answers
12 views

How to decode text output in autoencoders?

I have made an autoencoder for text based input, and fitted it to the data. Now I want to see the output text. Is there any way to decode the numbers to text? ...
0
votes
1answer
13 views

Back propagation on matrix of weights

I am trying to implement a Neural Network for binary classification using python and numpy only. My network structure is as follows: input features: 2 [1X2] matrix Hidden layer1: 5 neurons [2X5] ...
0
votes
1answer
26 views

Predicting when component will fail having its parameters data

I have a component and I need to predict when it will wear out and will need replacement. I monitor, let's say 5 parameters of this component, each one is monitored for every run cycle. So, the ...
0
votes
1answer
23 views

Merging multiple duplicate rows based on multiple columns in Pandas Dataframe [on hold]

I would like to merge nearly duplicate rows in a data frame like below: ...
0
votes
1answer
9 views

Reason for Training and test loss sudden increment after some epochs keras

We know that if training and test loss are different from each other, our model is over-fitting. However, if both get high after some epochs, how can we justify it? One way to solve it is to reduce ...
4
votes
3answers
43 views

How can I know if my conv1D model is overfitted or underfitted from loss curve?

I am working on classification of time series multivariate data. By doing PCA, I converted multivariate to uni-variate and fed it into a conv1d in keras. However, I am getting a very high accuracy ...
0
votes
0answers
12 views

What change should I make in the Image Data Generator to solve the error? [closed]

I am trying to perform Mixup Augmentation (details here) but I am getting a value error as follows : ...
0
votes
0answers
12 views

XGBClassifier Feature Order

I'm working with XGBClassifier from the XGBoost Python API and recently found out that the class requires feature columns to be in the same order for predictions as they were used for training. For ...
0
votes
0answers
9 views

Keras model not resuming training well

I tried resuming training with my keras model, to save the model I used model.save('model.h5'). And to load it again I used ...
0
votes
0answers
25 views

Python Machine Learning for Data Generation

I am trying to learn machine learning with Python for a specific application to see if it's doable, right now it's just an idea. The goal is to generate or improve meshes for CFD simulations using ML, ...
1
vote
1answer
28 views

is there a way to customize my loss function to increase recall in one class only?

First, thank you. I'm using Tensorflow 1.13 with Python 3. I'm trying to create a CNN classifier on data with 4 classes, one-hot encoded. However, the insights I need it to produce are really around ...
0
votes
1answer
17 views

Fill authentication login username using speech recognition python [closed]

I would like to fill authentication login using voice in Python.If i am saying 'admin' on microphone then on authentication page,in username textbox 'admin' should type automatically.i dont need to ...
1
vote
0answers
23 views

How to write inputs append with time using panda in python

I have a data csv file include with three inputs temperature, humidity and wind. So in this csv file first input recorded at 6:00:00a.m. But I want to start my time as 0 and then my second time is <...
1
vote
1answer
23 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
0
votes
0answers
7 views

Predicting tool breakage on a CNC/VMC machine using Machine Learning?

I data of a fixture (which holds a component during machining) as vibration in 3 direction, the pressure of hydraulic, and proximity. I would like to know when the tool might get broken in the near ...
-1
votes
2answers
18 views

Clustering of User Behavior in chatbot [closed]

I have a dataset that represents a user's behaviour in my chatbot (userid, last clicked button, session, timestamp). I need to divide the users into clusters based on their behaviour on the chatbot ...
0
votes
0answers
14 views

What are good rare pattern mining codes in python? [closed]

I was initially using FP growth algorithm to find patterns but that are frequent patterns. I’m required to find the rare items ets and pattern in my health data so that I can find the association ...
1
vote
2answers
26 views

Word Embedding for Item Names(integer, one-hot encoding)

I am looking for the way to get the similarity between two item names using integer encoding or one-hot encoding. For example, "lane connector" vs. "a truck crane". I have 100,000 item names ...
1
vote
0answers
16 views

A model that only works by setting all initial weights to zero

In this model from MusicNet, they set the initial weights of their neural network to all zeros. ...
0
votes
0answers
16 views

Which models should I choose in this project? [closed]

I have a project using python which requires to predict their customer retention. The independent variables include order_date, category_name(category of the product the store sold), product_name, ...
0
votes
0answers
21 views

Using bagging and random forests together

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of ...
0
votes
0answers
35 views

how to write start_time= 0 and increment with 60 minutes to display y value (predict) continously using python

Here I have a dataset with three inputs, temperature, humidity, wind. I want to predict temperature value using these three inputs data in future at every 60 mins. For the time code, here my first ...
1
vote
2answers
50 views

Obtaining consistent one-hot encoding of train / production data

I'm building an app that will require user input. Currently, on the training set, I run the following code, in which data is a pandas dataframe with a combination ...
0
votes
0answers
12 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
1
vote
0answers
31 views

Area Under Curve 5 Year Prediction

I have a dataset that includes a ratio(y-axis) and tenure(x-axis). My goal is to take the training set and establish a learning curve by tenure (1 to 15 years) which looks like this: Actual Group ...
2
votes
2answers
52 views

Similarity score: Can Sklearn SVR predict values greater than 1 and less than 0?

I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve a similarity problem. Using GridSearchCV, I am finding the best ...
-3
votes
0answers
35 views

Python sum the specific rows of columns [closed]

I have following data: ...
0
votes
0answers
6 views

Automatically assess training data quality for land cover classification system

I am working on a Land cover classification system, wherein, Sentinel-hub imagery is being used to categorize the land cover by using a time series of multispectral imagery. Training data is being ...
0
votes
1answer
23 views

Clustering with custom criterion (minimum cluster weight)

Edit: following comment from @anony-mousse, I'm changing the question to search for a general clustering approach that matches this criterion (minimum weight per cluster). I am to use a clustering ...