Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

Forecasting - 3 independent variables

I have 3 independent input variables: time, temperature and pressure, and one output - usefulness. Example: Number of the sample; Time; Temperature; Pressure; Usefulness 1;1;30;50;9% 2;3;50;50;80% ...
0
votes
1answer
15 views

Trying to return more than just the top result from sklearn NearestNeighbors

I'm trying to compare a list of names (duplicated into a clean file and a messy file). I then compare the files against each other. My problem is that it returns only the top 1 result for each, ...
0
votes
1answer
32 views

Why do probabilities sum to one and how can I set optimal threshold level?

I am working on a text classification use case. The training data has two classes, so the XBBoostClassifier and onevsrest model is classifying the test data into either of the two classes. But my ...
0
votes
0answers
15 views

How to validate regex based Resume parser efficiently

I am using rule based logic to extract features from resume. Basically I am trying to find if the candidate switched the company in less than 1 year. So I have the code in place to find it using ...
0
votes
0answers
26 views

Can anyone direct me on how to got to [closed]

I am a newbie to python. I have uploaded about 2000 images into my google drive, assigned a variable to all content in the folder and tried to split the folder's content into training and testing, ...
1
vote
1answer
17 views

Feature extraction from Resume using Python without rule based logic

I am working on Resume parser project. Currently I am using rule based regex to extract features like University, Experience, Large Companies etc. So Basically I have a set of Universities name in a ...
1
vote
0answers
21 views

Need help regarding my thesis project related to Data Analysis [closed]

I am working on a project with a company that manufactures Injection molding machines. I am to perform data analysis on some selected parameters as a part of my thesis to tell the company if from ...
2
votes
1answer
26 views

Why does MAE differ after prediction (Neural Network)?

I'm having trouble understanding what's happening in the following code. I already have defined x_train, y_train, x_val, y_val and x_test which define my training, validation and test sets. I'm using ...
1
vote
0answers
38 views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
2
votes
1answer
30 views

How can we extract fields from images?

I am making an document parser which extracts data fields from the documents and store them in a structured way. Each field in my dataset is horizontal which is easy to extract but the model fails on ...
1
vote
1answer
18 views

multiclass classification

I want to build an ml model, which can when given a text input, can predict predefined tags or labels for the text. I already built one such model, but the problem with that is that it only predicts ...
1
vote
0answers
18 views

How to classify unseen text data?

I am training an text classifier for addresses such that if given sentence is an address or not. ...
1
vote
1answer
18 views

Passing the output of a model as input to an other model , computer vision?

i want to try my hands at a project where i have to detect the brands + the models of sportswears worn by persons participating to a marathon , the project is quite complex , so my first approach ...
3
votes
1answer
30 views

What is a good method for detecting local minims and maxims?

I'm using kernel density estimation in order to compute probability density function for item (triangles in the figure) occurrence. Using this output, i want to find all the local minims and maxims. ...
2
votes
1answer
34 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
1
vote
0answers
19 views

Organize the Dataset

I am working on a neural network(Variational Autoencoder) to separate music instruments I don't know how to organize my data I have a folder full of spectrograms of the songs as a mix and the ...
1
vote
0answers
12 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python?
2
votes
0answers
21 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
1
vote
2answers
36 views

Predict the average temperature for next 30 years

Objective: I want to predict the average temperature for next 30 years. Q1: What type of dataset is suitable for this (what columns should it contain) Q2: What are the independent variables for ...
1
vote
1answer
26 views

Implementing “full convolution” to find gradient w.r.t the convolution layer inputs

I've been trying to implement "full convolution" w.r.t to convolution layer inputs. According to this article, it looks like this: So, I wrote this function: ...
0
votes
0answers
11 views

ERROR: torch-1.0.1-cp37-cp37m-win_amd64.whl is not a supported wheel on this platform [closed]

I need to install pytorch in my pycharm on windows, I tried this code: pip install https://download.pytorch.org/whl/cpu/torch-1.0.1-cp37-cp37m-win_amd64.whl and I got this error: ERROR: torch-1.0.1-...
0
votes
0answers
26 views

What is the difference between “fit_transform” and “transform” methods when using “SimpleImputer”? [duplicate]

I have following code, I am not able to understand the difference between use of fit_transform() and transform() method in this ...
1
vote
1answer
18 views

(pre-trained) python package for semantic word similarity

I am searching for a python package that calculates the semantic similarity between words. I do not want to train a model (what most packages seem to offer) - the package should have been pre-trained ...
3
votes
0answers
27 views

Sources to learn Anomaly Detection

Project Goal- Create an Anomaly Detection tool. As the first step I worked on some python code that would parse my XML files into a Dataframe in csv file. The next step is to go into machine ...
-2
votes
0answers
9 views

Looping over lists [closed]

I have list values like this ['01000000111011100010001100101001', '01000110111010010010011000101100', '01110100000011101100111000101000', '01110101000010001111011010000111'] I want to count ...
0
votes
0answers
15 views

detect table of content in PDFs

I have PDF text data that I want to use for NLP tasks. Now during preprocessing, I came to the conclusion that it is not worth analyzing the table of content. It merely generates a bunch of ...
1
vote
1answer
28 views

PCA Regression Problem

I have a regression problem whereby my data has 21 features and I wish to apply dimensionality reduction using PCA. As far as I know, all the tutorials I have seen so far use PCA for classification ...
0
votes
0answers
28 views

Power BI or Dash? [closed]

I'd like to know opinions regarding Dash (python) and Power BI. For context, I am currently working on producing several dashboards and I am not sure which one I should work with. I've never used ...
3
votes
0answers
19 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
3
votes
1answer
51 views

Best anomaly detection algorithm based on two conditions

Choosing the right anomaly detection algorithm seems quite hard at the moment. It might be because I am bombarded with so many alternatives likes clustering, K-Means DBSCAN and so many others. On my ...
3
votes
1answer
26 views

How to properly store a list of pandas dataframes, so that I can import them in a new python script?

I'm trying to make a GUI in PyQt5 to present the results of my research. The results are around 800 pandas dataframes, stored in lists (so that I can call them individually according to the list index)...
2
votes
1answer
23 views

Ridge regression model creation using grid-search and cross validation

I created python code for ridge regression.For that I used cross validation and grid-search technique in together. i got output result. I want check whether my regression model building steps correct ...
1
vote
0answers
21 views

What is the formula to calculate the precision, recall, f-measure with macro, micro, none for multi-label classification in sklearn metrics?

I am working in the problem of multi-label classification tasks. But I would not able to understand the formula for calculating the precision, recall, and f-measure with macro, micro, and none. ...
2
votes
1answer
26 views

Multiclassification Error: NotFittedError: This MultiLabelBinarizer instance is not fitted yet

After picking the model, when I try to use it, I am getting error - "NotFittedError: This MultiLabelBinarizer instance is not fitted yet. Call 'fit' with appropriate arguments before using this ...
3
votes
4answers
73 views

Machine Learning methods for finding outliers

I have a csv files of thousands of lines, the data is put down into a Dataframe of columns. Some of the column have text information while others might have numbers. I want to detect anomalies or ...
1
vote
0answers
11 views

Plotly and altair [closed]

I have a question about these two bookstores: Plotly and Altair. They are both used for data visualization and interaction. However, when I use them with a 300,000 line dataset, my notebook doesn't ...
1
vote
1answer
15 views

find bigrams in pandas

I have a DataFrame with 4 columns: 'Headline', 'Body_ID', 'Stance', 'articleBody', with 'Headline' and 'articleBody containing cleaned and tokenized words. I want to find bi-grams using nltk and have ...
1
vote
0answers
24 views

Use trained Tensorflow model to predict on different dataset

I have few datasets, I've trained model on the biggest one and now want to see how it will predict values for different set of data. For saving the model, I've used ...
2
votes
0answers
18 views

Is there any solution for modified t-test in python for spatial data?

I am searching for a solution to conduct a modified t-test or any other possibility in order to compute significant or non-significant relations in a spatial dataset (which can be a stack of rasters ...
0
votes
1answer
24 views

KeyError: 'i' and <listcomp> Error

I am implementing the codes for https://www.kaggle.com/eray1yildiz/using-lstms-with-attention-for-emotion-recognition which is an emotion analysis from the text. I am having some errors when I am ...
1
vote
1answer
23 views

Calculate Similarity using User's Personal Data?

I want to find out which users are similar to each other using their personal/organisational data, such as department, company, site, etc. I have this data in a boolean format, as shown below: ...
1
vote
1answer
15 views

Need some info regarding string matching algorithms?

Let me explain a scenario to better explain my question, Assume I am working in a credit-card related company in which people uploads their receipts every month, I want to check if that person bought ...
5
votes
4answers
573 views

Convert Numpy array with 'n' and 'y' into integer array of 0 and 1

I have a NumPy array of strings: 'n', 'y', wanna convert it into integer array of 0, 1, how to convert it? ...
0
votes
1answer
16 views
2
votes
0answers
15 views

Is it possible to get prediction intervals in sequenced data RNN forecasting?

Is it possible to get prediction intervals in sequenced data RNN (keras+python)forecasting? For example: predicts your car sales or new purchase The question is: UserId 1 will change his car in the ...
3
votes
5answers
279 views

GridSearch without CV

I create a Random Forest and Gradient Boosting Regressor by using GridSearchCV. For the Gradient Boosting Regressor it takes too long for me. But i need to know which are the best Parameter for the ...
2
votes
2answers
28 views

Method to assess text credibility

I am searching for an automated method (ideally a python package) that produces a score to assess the credibility of a given text (e.g. from a webpage). I am not searching for: text complexity ...
-2
votes
0answers
13 views

index 46 is out of bounds for axis 0 with size 40 [closed]

from sklearn.model_selection import train_test_split train_data,test_data=train_test_split(df1,test_size=0.2,random_state=0) import statsmodels.formula.api as sf m1=sf.ols("Profit~RnD+Administration+...
1
vote
1answer
27 views

Python package to assess text coherence

I am looking for a python package that calculates how well one sentence of a natural text follows the next. One could simply count how many identical words are in the next sentence but the better ...
0
votes
0answers
15 views

Correlation between numeric variables of different meaning

I have a data set containing information about buildings, such as the area code, the neighborhood code, the number of floors, the year in which the house was built and the year in which restorations ...