Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1
vote
1answer
51 views

Unsupervised Text Classification with Python: Kmeans

I am working on a project to build a text classifier of questions being asked. There are no labels provided in my data so I have chosen to go with an unsupervised approach. This solution needs to read ...
0
votes
0answers
15 views

U-Net doesn't work with images different from the dataset

I have implemented a very similar U-Net code from github, but for a different dataset, this one, to segment roads, it works fine using the test folder images, but when i for example, pick a print from ...
0
votes
0answers
37 views

Extracting possible relationships and insights from a large dataset

I have a dataset with about 26'000 variables and 33'000 continuous observations. My task is to extract any interesting insights/possible relationships between various variables. I am looking for any ...
0
votes
1answer
22 views

Pandas - avoid iterrows() assembling a multi-index data frame from another time-series multi-index data frame

For analyzing pairs of currencies, I have two dataframes. The first one contains information about the symbols constituting every unique pair (as in "pair metadata"): ...
1
vote
1answer
27 views

Adjusting imbalance in classification problem reduce precision, accuracy but increase recall

I've learned that adjusting imbalanced data when training a CNN affects model performance which got me thinking "what about in ML?" so I've done some testing on my own, you can check it out ...
0
votes
1answer
29 views

How to specify different columns stacked vertically within CSV using pandas?

I have a CSV file that looks like below with columns labeled below data, and I'm have trouble trying to retrieve said data with pandas. I'm only able to address the columns at the top with the ...
2
votes
1answer
47 views

Why do my target labels need to begin at 0 for sparse categorical cross entropy to work?

I'm following a guide here to implement image segmentation in Keras. One thing I'm confused about are these lines: ...
2
votes
1answer
74 views

TypeError: Expected int32, got None of type 'NoneType' instead

I want my model batch size to be a dynamic shape, and I've assigned none as batch size, but that's causing an error. Here, in the first line, I specified batch size as None: ...
0
votes
1answer
34 views

Imputing missing value based on filtering result of another column [closed]

C1 C2 A x A y A z A x A NaN A x A x A x B y B y B z B y B NaN B y B x B x I have to impute missing values in C2 , the imputation should be such that if the missing values ...
1
vote
2answers
17 views

Normalizing a list with both positve and negative numbers to the range [-1 1] [closed]

For example, I have a list [-518.8134, 480.1884,160.4761] How to normalize this to [-1 1] range?
3
votes
0answers
11 views

Running DenseNet from cmd line and jupyter but vastly Different loss and accuracy [closed]

I’m running a DenseNet121 from Pytorch with the same exact code, same exact hyper parameters and same exact image sizes, once from a jupyter notebook and once using the command line via a python ...
1
vote
1answer
532 views

TypeError: Image data of dtype object cannot be converted to float [closed]

I am trying to convert files stored in a CSV file dataset to grayscale. I run the code which worked on my previous dataset but I now get this error: TypeError: Image data of dtype object cannot be ...
0
votes
1answer
36 views

Random Sequence Predictions

Can machine learning algorithms predict random number generators. key A= 1 2 3 B= 4 5 6 C= 7 8 9 Example to catch a number sequence 4 8 8 I would select B C C That would give me 27 number combinations ...
2
votes
1answer
44 views

How to compare 10000 data frames in Python?

I have 10000 data frames like this: Each data frame corresponds to a different card game (and so different numbers in the table) and I want to compare these data frames all together. For example, I ...
1
vote
0answers
32 views

Text preprocessing on corpus in pipeline before Gensim word2vec training

I have a large compressed corpus, about 30gb in .txt.gz format. In raw format it can be used as input to word2vec like this: ...
1
vote
0answers
7 views

Cleaning error text strings inside Categorical Variable Columns [closed]

I have a DataFrame (see below - the real dataset is about 10,000 rows, 26 columns) with errors like '?' or '&#-81;' where there should be a string corresponding to a value like a Premise or a ...
1
vote
0answers
27 views

Can we combine decision tree , xgboost , lightgbm to make one hybrid classifier? [closed]

Can we combine three different classifier to make one hybrid classifier ?
0
votes
0answers
23 views

Scipy kstest problem

I am fitting mixture models to data and assessing how mixtures with more or less components will fit the data. To do this, I am going to plot the cdf of the empirical data and the cdf of my mixture ...
1
vote
0answers
18 views

Serving feature pipeline [closed]

Imagine, there is a service, providing credit history for customers in form of list of his loans. Let's call it my-loan-service. For the sake of simplicity - I can ...
1
vote
0answers
27 views

Time Series Classification & Seasonal Data [closed]

I am looking to predict values based on seasonal data. Bonuses are paid quarterly/ annually/ monthly and amount usually goes up after couple of time periods. Data is given below. I have converted the ...
0
votes
0answers
13 views

how to detect bifurcation, trifurcation, and crossover on image with python opencv

how to detect bifurcation, trifurcation, and crossover from the branch point with OpenCV when the input like this Expect output : I'm trying to use binary_hit_or_miss but I don't know how to display ...
2
votes
0answers
28 views

unsupervised learning time series datasets

I experiment on building electricity power consumption datasets and try to see relationships of the power consumption with weather data and dummy variables that represent time-of-week. The only thing ...
1
vote
0answers
21 views

Padding before or after truncation for stateful LSTM in Keras

I am training an LSTM on a dataset with variable timesteps (between 10 and 6000). Using the truncated backpropagation through time (TBPTT) technique, I am truncating the sequences to windows of 128 ...
0
votes
1answer
14 views

Correct LSTM model to predict shuffled data

For a year I've been collecting data from my RPi: [0 core load, 1 core load, 2 core load, 3 core load, environment temperature, fan speed, CPU temperature] Now I ...
0
votes
1answer
28 views

Help improving time series prediction with LSTM on PyTorch

So, I am trying to use a LSTM model to forecast temperature data on PyTorch. I am relatively new to both PyTorch and the use of recurrent networks so I took a model I found on the internet to start. ...
1
vote
1answer
33 views

How to interpret the Mean squared error value in a regression model?

I'm working on a simple linear regression model to predict 'Label' based on 'feature'. The two variables seems to be highly correlate corr=0.99. After splitting the data sample for to training and ...
1
vote
0answers
19 views

What is the meaning of, or explanation for, having multiple tags in a Doc2Vec model's TaggedDocuments?

I've tried reading the other answers on this topic but I'm unsure if I understand completely. For my dataset, I have a series of tagged documents, "good" or "bad." Each document ...
0
votes
1answer
16 views

How do i get the loss function graph?

I used Mini-batch gradient descent to train the model, but i am unable to get the proper loss graph. The loss graph is always showed as a straight line. I know there is something wrong but would ...
0
votes
1answer
21 views

Vertical concatenation in a df based on column value_python

Is there any way to concatenate vertically a specific column from my df (concat1), considering/filtering values from another column (col_value)? My df looks like this: ...
1
vote
0answers
17 views

MCMC algorithm — understanding some paremeters

I am trying to understand an MCMC program. I manage to run it, but I am trying to understand the meaning of the some parameters in the analysis. The code is something like this ...
1
vote
1answer
79 views

Python: SARIMAX Model Fits too slow

I have a time series data with the date and temperature records of a city. Following are my observations from the time series ...
0
votes
1answer
45 views

Implementing the Trapezoid rule without the formula for the curve

I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows: ...
0
votes
0answers
23 views

Appropriate Approaches for Unsupervised Textual Classification

Let's say I take 1000 dissertations from a variety of different fields, and thereafter extract the top 100 words from each dissertation. What would be the best approach to categorize them into STEM/...
0
votes
1answer
29 views

What is “Missing” in output of plot_tree API of XGBoost

What this "Missing" term means here at each node after split in Image? and also what is at leaf, is this means prediction value? I converted Output variable to 1 and 0. I tried searching on ...
0
votes
1answer
82 views

numpy array center of mass

I'm wondering how does ndimage.measurements.center_of_mass calculate the center of mass values for a two dimensional numpy array. For example let's say we have an ...
0
votes
1answer
43 views

Model Dump Parser (like XGBFI) for LightGBM and CatBoost

Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
0
votes
0answers
18 views

How to determine my pacf and acf in ARIMA with daily time series data?

I have daily time series data for revenue sales ranging from 2018-04-01 to 2021-02-19. I want to implement an ARIMA model and the plot of the data looks like this: Based on my augmented-dickey fuller ...
1
vote
1answer
17 views

Replace part column value with value from another column of same dataframe

I have a dataframe with two columns: Name DATE Name1 20200126 Name2 20200127 Name#DATE# 20200210 I need to replace all the #DATE# with ...
0
votes
1answer
18 views

Is LSTM or pretrained BERTForMasked LM usable for predicting changed word in a sentence using a small dataset? (2000 samples)

I have a small (2000 samples) dataset of newspaper headlines and their humorous conterparts where only one word is changed to sound silly, for example: Original headline: Police <officer> ...
0
votes
0answers
17 views

Predictions receiving in Unknown (8)[e,e,e,e,e,e,e,e] format from TensorflowJS Mobilenet

I have trained a mobilenet on diamond images to count diamonds in broswer. I then converted the SavedModel format to TFJS format. I have the following code in my JS file. ...
1
vote
1answer
17 views

LDA topic model has 0-weight topics, is that normal?

While experimenting with different number of topics for the Gensim implementation of LDA, I found that for a high number of topics, the output often consists of topics with all weights equal to zero. ...
0
votes
0answers
8 views

Loading data directly from BigQuery to Orange

I would like to ask you if there is possibility in Orange to load to it data directly eg. from BigQuery? I added block "Python script" to the flow and my script looks like that: ...
0
votes
1answer
15 views

ship class Object detection with custom model weights

I have made a model to classify different categories of ships(yacht,catamaran,rubber boat) in Python and i hit a 70% accuracy in training so now i have my weights.hdf5 file.Now i need to object ...
1
vote
1answer
24 views

Set value for column based on two other columns in pandas dataframe

I have a dataframe that has contracts with different order dates and I need to create a new column that assign a number to each contract if it has more than one order date. For example my sample ...
0
votes
0answers
15 views

How to implement co-occurance matrix in Python which only takes into account next word relationships

I've been trying to create a dictionary version of a co-occurence matrix in Python, such that it only counts instances where one word follows another in a set of sentences. I found this implementation ...
0
votes
0answers
20 views

Why is doing a ShuffleSplit with GridSearchCV decreasing the performance of my model?

I'm using the Boston Housing dataset in Keras, and neural networks for this regression problem. The following is the code I use to prepare the data, build the model, and fit it with GridSearchCV. <...
0
votes
0answers
35 views

Failed to convert a NumPy array to a Tensor (Unsupported object type float)

I'm using Tensorflow 2 and using model.fit for training the model. but when I try to run it, it says Failed to convert a NumPy array to a Tensor (Unsupported object type float). Where I'm making ...
0
votes
0answers
15 views

Measurement error in mixture density network input

I am fairly new to keras / tensorflow but it looks like a mixture density network can solve my problem. I’d like to predict a continuous variable using, in a simple case, 3 features. All features and ...
1
vote
1answer
38 views

How to select the best parameters for GridSearchCV?

I've created a couple of models during some assignments and hackathons using algorithms such as Random Forest and XGBoost and used GridSearchCV to find the best combination of parameters. But what I'm ...