Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Only training for the presence of an object in a convolutional neural network?

I am designing a neural network that is looking to see if an image contains an apple or not. If it contains an apple then it will output 1, if it does not contain an apple it will output 0. Right now ...
2
votes
3answers
27 views

Count the max number of consecutive 1 and 0 in Pandas Dataframe [closed]

Hey I have the following Dataset import pandas as pd df = pd.DataFrame({ 'column1': [0,0,1,0,1,0,0,1,1,0,1,1,1]}) I want to be able to count the number of ...
0
votes
1answer
20 views

Proper way to make Train/test split on Time-Series

I want to create a model with LSTM to predict a user the next purchase value. For this I have used I used a user's purchase history. I have created the model and it works well, but honestly, I don't ...
0
votes
0answers
10 views

How to use WikiData to compare words semantically?

I'm briefly new to WikiData und the SPARQL queries. I want to use it to compare words semantically with Python. For example, I want to archieve something like this: ...
3
votes
0answers
36 views

What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
0
votes
1answer
24 views

Increase Accuracy on Keras Multiclass Classifier

I have a dataset with the shape (430, 13). My output is a single column consisting of floats ranging from 0.2 to 250. I attempted a Keras regression to predict the value, but given the huge range, I ...
1
vote
1answer
16 views

how to find the best parameters to solve a differential equation?

I have a differential equation: def func(Y, t, r, p, K, alpha): return r * (Y ** p) * (1 - (Y / K) ** alpha) and I want to find the best parameters that fit (r,...
0
votes
0answers
5 views

LinAlgError: Singular matrix for finding pvalues in logisticregression [closed]

I am trying to find the probability of default for credit risk modeling, So I tried to find p-values using this code but I'm getting the below error. Could you please check and suggest me. ...
0
votes
0answers
11 views

How to use multiple cross-section observations per subject for churn prediction?

Recently I have started to teach myself about machine learning and I have ran into a dataset, which got me a bit confused. Dataset: The subjects of the dataset are university students (student ID == &...
0
votes
1answer
30 views

Sklearn SVM question classification

So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset: The goal of this task is to ...
1
vote
1answer
33 views

First two principal components explain 100% variance of data set with 300 features

I am trying to do some analysis on my data set with PCA so I can effectively cluster it with kmeans. My preprocessed data is tokenized, filtered (stopwords, punctuation, etc.), POS tagged, and ...
1
vote
1answer
30 views

How to use df.groupby() to select and sum specific columns w/o pandas trimming total number of columns

I got Column1, Column2, Column3, Column4, Column5, Column6 I'd like to group Column1 and get the row sum of Column3,4 and 5 When I apply groupby() and get this that is correct but it's leaving out ...
1
vote
0answers
13 views

Using if statement as a model in orange

I am seeking to evaluate logistic regression against a rule-based (nested if-else statements) classification in Orange. The steps I've taken to investigate this are as follows: I could not identify ...
1
vote
0answers
6 views

Interpreting statsmodel Granger Causality test results

I am trying to use statsmodels Granger Causality test: https://www.statsmodels.org/stable/generated/statsmodels.tsa.stattools.grangercausalitytests.html to assess whether "positivity score" ...
2
votes
2answers
23 views

Resources for text classification algorithms

I'm mining raw Facebook comments (irrelevant) and i am looking for an algorithm that can classify their context as negative/positive/neutral. So you can think of the output in the form of two columns. ...
1
vote
1answer
18 views

parsing dictionary to JSON [closed]

I used the following code to pass a json string as a parameter in requests.post method, ...
1
vote
1answer
13 views

3d plot for clustering

I'm working on a cluster analysis project and was trying out different ways to plot clusters to explain the differences between the clusters. I did plot the different clusters in 2D. Can you please ...
1
vote
1answer
14 views

why does my calibration curve for platts and isotonic have less points than my uncalibrated model?

i train a model using grid search then i use the best parameters from this to define my chosen model. ...
0
votes
0answers
17 views

Maximum of a List (possibly with varying length for each index) and a number [migrated]

Suppose I have a list of length 3. And the entries in the list has dimensions: 5, (5,5), 5 An example can be [[1,23,3,3,2], [[1,2,2,3,4],[1,4,3,1,4],[6,7,8,5,4],[3,5,2,2,1],[5,2,6,7,8]], [1,3,6,4,2]] ...
1
vote
2answers
28 views

How to make a neural network generalizes better?

I designed a neural network model with large number of output predicted by softmax function. However, I want categorize all the outputs into 5 outputs without modifying the architecture of other ...
0
votes
0answers
6 views

Missing UserId values in Stack Exchange Website Data Dumps [closed]

I had downloaded the data from the stack exchange data dump of many websites, but there are missing values in the UsersId column for the ...
1
vote
1answer
35 views

Getting value of 2 in normalized confusion matrix

I have created confusion matrix with python and I have gotten weird results that made me worried: as you can see, I have value of two, and if I understand correct, normalized matrix cannot have value ...
-1
votes
0answers
24 views

Accuracy during the test is very low and the result is similar to a head or tail

First I'm sorry, I don't know Python but I need to get these positive results. I need to find out where I'm going wrong, because such a low accuracy and bad final test. ...
2
votes
1answer
36 views

Computing adjusted p-values in batches

Data For simplicity I propose the following scheme: I have two DataFrames, one with genes, the other with miRNA (it is a simple example, it is not the case what the DF are), the data are continuous: ...
0
votes
2answers
25 views

Is there a method to apply trained weights to a model with the same input shape and model architecture but different output shape?

I am developing a speaker identification model in Keras, and I have saved the weights from a trained custom model. Now, I am looking to use the trained weights to fine tune the model on a new dataset, ...
0
votes
1answer
34 views

Merge two or more ML models with different size of output togetger

I have several models that classify the input (word embedding) into several classes. My problem is that I need to train these models separately and need to merge the output of these models together to ...
0
votes
1answer
28 views

conversion of dictionary to json to be sent to requests.post

I have a data frame like this. df=pd.DataFrame({'sub1':[10,20,30,40],'sub2':[5,10,15,20],'sub3':[1,2,3,4]}) and a dictionary like this. ...
0
votes
0answers
30 views

What metrics to use in regression if variance in output label is very low?

What metrics/method should one adopt in judging the error when variance is low in output variable. To give you an example : the output variable can be stock prices over a month, the variance generally ...
0
votes
0answers
14 views

Get optimizer state tensorflow

I am currently working into a problem that requires to redefine the same model multiple times, just having "logical" variations that do not change the model structure. In doing that, I would ...
0
votes
0answers
19 views

Inverse predict the features from known target with fitted sklearn regressor

I understand that the default way a scikit-learn regressor works is that we fit it to a dataset of features and targets (X_train, ...
1
vote
1answer
14 views

Find the best interpolation method for missing observations

I have a database which has measurements of objects every day every hour. However, some data is missing and I don't have measurements for all the hours. in order to get over this challenge I have ...
0
votes
0answers
14 views

Multi Image to Single Image Assocaition

I have a dataset of drone captured images (non-geotaggged) which I would like to stitch to form a single image. I wanted to know what type of a neural network can I use to associate multiple images to ...
0
votes
0answers
13 views

Understanding Mapper clustering

Is there a way to find the number of clusters in mapper.map? It is a module in kmapper.KeplerMapper. When I plot the graph ...
-1
votes
0answers
13 views

Bottleneck Distance

Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar? Also, does it make sense to compute the bottleneck distance using $H_0$ ...
0
votes
1answer
13 views

Is it valid to use numpy.gradient to find slope of line as well as slope of curve at any point?

what is the difference between slope of line and slope of curve ? Is it valid to use numpy.gradient to find slope of line and slope of curve at any point ? ...
1
vote
1answer
20 views

How to find slope of curve at certain points

how to find slope at certain points circled in blue in below curve ? Are these below 2 approaches valid ? though they give different results . How to automatically find the points where the slope ...
0
votes
0answers
14 views

Fixing autocorrelation not caused by time series

I have data with features that looks like this I am using ML regression in Sklearn to predict a final cost (in a separate df). Before fitting a linear regression I went to test the assumptions of ...
0
votes
0answers
6 views

Automatically find an optimal threshold for multiple datasets

I am running an unsupervised model on data with no labels to find outliers, and have 100 datasets in total. The way I do this is, first I run it using a default threshold on each dataset, and second, ...
0
votes
1answer
17 views

How to improve pandas DataFrame lookup [closed]

Need help to optimize this code, just for 256553 records and about 100 MB DataFrame size, this code takes 25-30 minute to execute. Basically what I am doing here ...
4
votes
0answers
55 views

ground truth fit is worse than cross validated fit on noisy data?

I am having these weird results when playing around with cross validation that I would greatly appreciate to have any comments. Briefly, I have a lower mean squared error (MSE) when doing regression (...
0
votes
1answer
15 views

How to Transpose dataframe with custom output? [closed]

I have a data like this: I want to transpose it like this in python:
-1
votes
1answer
15 views

How to run Neural Net on GPU without python frameworks?

I coded a deep learning model from scratch in python(using numPy) without using any frameworks like keras or tensorflow. So far my model works fine but it runs on CPU. How should i modify my code so ...
0
votes
0answers
23 views

How to set colored text on ipywidgets.Textarea? [closed]

I have a code that using defined keywords extracts sentences from text. ...
1
vote
1answer
29 views

Pandas - Sum of multiple specific columns [closed]

I created this script: ...
0
votes
1answer
23 views

How to give some conditions in an array

...
1
vote
1answer
56 views

Choosing a distance metric and measuring similarity

I am trying to decide which particular algorithm would be most appropriate for my use-case. I have dataset of about 1000 physical buildings in a city with feature space such as location, distance, ...
0
votes
1answer
30 views

No gradients provided for any variable: ['Variable:0', 'Variable:0']

I am using Python 3.6 and Tensorflow 2.0 for the following code for linear regression: ...
1
vote
0answers
19 views

Predicting Item Ratings with the Log Likelihood Ratio

I'm trying to infer prediction ratings from an item-item similarity matrix where the similarity score is calculated via the log-likelihood ratio (LLR). I'm using this code snippet to calculate the LLR ...
1
vote
0answers
11 views

how to build lstm with functinal api?

I am having a time series prediction problem and the data set has 4 variables. My data set is like below: ...
0
votes
1answer
21 views

Unhashable type 'list' while looping through dataframe in Python [closed]

I have the following dataframe comments. I have segregated a list of users based on certain conditions. I want to get the count of words based on those users which ...

1
2 3 4 5
93