Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).

Filter by
Sorted by
Tagged with
1 vote
0 answers
49 views

Highlight specific paragraphs from documents

I have a bunch of documents in which I want to highlight certain paragraphs/keyphrases. I have a list of the most frequently appearing sentences and I want to search for these paragraphs/keyphrases in ...
spectre's user avatar
  • 1,941
0 votes
0 answers
22 views

Should I remove noise by fitting a curve in my case?

I have an object that is moving towards target point and I want to determine when it will reach or pass through it. I receive points in real time, which represent the current location of the object. ...
theateist's user avatar
  • 141
0 votes
0 answers
24 views

s3 object as corpus_file for gensim Doc2Vec

Is it possible to use a txt or jsonl file in an s3 bucket as the corpus_file input for a gensim Doc2Vec model? I am looking for something of the form: ...
Mutasim Mim's user avatar
0 votes
0 answers
7 views

I am not able to change the model family of my GLM using statsmodels

Here are my lines (in python) for my GLM: ...
chris.rechenauer's user avatar
1 vote
1 answer
124 views

Funny looking learning curve

I am training an XGBoost for binary classification. I am getting a very odd-looking learning curve. Any idea what is going on and how I might fix this? Thanks
pyassign67's user avatar
0 votes
2 answers
61 views

How do I separate periodic data from time series data?

I am currently working on a classification task of gym exercises based on accelerometer data. I am trying modularize window extraction so I can train my model based on metrics within a window (which ...
David Chen's user avatar
0 votes
1 answer
273 views

Implementation of Graph Neural Network for Image Classification

I'm currently working on a project where I want to utilize Graph Neural Networks (GNNs) for image classification tasks. However, I'm facing difficulties in understanding how to implement GNNs ...
Rezuana Haque's user avatar
0 votes
0 answers
26 views

Help with running topology GAN (TopoGAN)

Link: https://github.com/TopoXLab/TopoGAN-ECCV2020 Sorry if this is the wrong place to ask, but I've been looking for help on how to work this out for a long time as I don't have much experience in ...
fds-mqdwqmqkdwl's user avatar
1 vote
1 answer
37 views

Should I normalize entire dataset or normalize every x_train and every y_train for timeseries forecasting LSTM?

Suppose I have list like this: ...
Muhammad Ikhwan Perwira's user avatar
1 vote
0 answers
20 views

How to find own way in data field? [closed]

I need help finding my path in the data field. I just completed my MSc in Big Data at university. During my studies, I gained approximately Junior level Python and SQL programming skills, worked with ...
Gleb's user avatar
  • 11
1 vote
0 answers
37 views

ML with Python - extract information from user text

I'm a web developer, got a little experience with Python but none on ML. Tour operator customer want to introduce AI/ML on his website, the goal is to have a single text input where user can prompt ...
fudo's user avatar
  • 111
1 vote
1 answer
103 views

How can someone evaluate llama index model?

I have built a openAI llama index based model which takes multiple pdf and able to give chatbot based response. I want to evaluate the llm for accuracy. I already know method such as Rouge and Bleu. ...
Prashant Pandya's user avatar
0 votes
1 answer
51 views

RMSE too high when trying to create a machine learning model in Python

I am new using Python/ML. Right now, I am trying to create a model to forecast the expected call volume for a company. However, the RMSE that I am getting is higher than expected Here is my code. I ...
LatverianKid's user avatar
0 votes
2 answers
154 views

KNN Accuracy training

After I developed my model using KNN I get the following accuracy: Train Accuracy :: 1 Test Accuracy :: 0.24 What is the accuracy of my model?
user150859's user avatar
0 votes
0 answers
23 views

Varying the axes of a seaborn displot to save space

I used the code below to generate the image attached to this question: ...
Zorgoth's user avatar
  • 133
0 votes
0 answers
175 views

Seaborn displot with labels on every set of axes?

I used the code below to create the image attached to this question. There are a couple things I don't like about the result. One is that it consumes a huge amount of vertical space, all because a ...
Zorgoth's user avatar
  • 133
0 votes
0 answers
401 views

CUDA error: "no kernel image is available for execution on the device" - which PyTorch version to use?

I'm running some ML python code on Amazon's EC2 - the machine has a GPU: NVIDIA A10G. The ML code is written using PyTorch. When I run the code I get error: ...
Danijel's user avatar
  • 173
0 votes
0 answers
42 views

I have a strange loss behavior in Tensorflow/Keras. It goes smooth towards better values, and then suddenly it increases rapidly before falling again

I have a model that trains on a dataset of 100000 data points. The model is a single dense layer with a single neuron. During training, I use a batch size of 1000 and train for 250 epochs. After ...
Runei's user avatar
  • 1
0 votes
1 answer
325 views

Parsing Python List values from column in pandas data frame??? Data cleaning help me please

I am new to the community and data science. I have found this hub to be of extreme value and hope to be an additive contributor one day. VS Code: Python: Pandas Question I am working with a data set ...
Justin White's user avatar
0 votes
0 answers
50 views

how to predict the arpu for a monthly cohort dynamically?

The main idea of this project, is to predict the ARPU (Average Revenue Per User) 11 month after subscription of a cohort with a monthly subscription, using minimum number of delays (a delay is a month ...
wassimdiai's user avatar
0 votes
0 answers
152 views

Augmentation and transformations in Detectron2

I'm working on a custom Faster RCNN with Detectron2 framework and I have a doubt about transformation during training and inference. I created a custom Trainer ...
Agostino Dorano's user avatar
1 vote
1 answer
209 views

Why COST FUNCTION AND MSE IS CALLED THE SAME?

Why are the cost function and mean squared errors called the same thing? WHEN THE COST FUNCTION IS 1/2M AND THE MSE IS 1/N. AND M=N
Rubayet Alam's user avatar
0 votes
0 answers
16 views

Gaussian 'around' a given distribution

I want to find the data points within 0.2 dex of the Main Sequence Relation: (The main sequence relation is what you can observe in this figure) Dex is an astronomical jargon which signifies scatter, ...
Ambica Govind's user avatar
1 vote
1 answer
40 views

LogisticRegression Loading problem

Saving Model Part ...
Micheal T. Atienza's user avatar
0 votes
1 answer
28 views

How should I proceed with this type of dataset and requirement?

Suppose I have a dataset where I want to give input of up-to 30 lines (can be variable with min 5 available) these lines are timestamps of every minute and want my model to predict output of next 5 ...
Pratik Bhadane's user avatar
-1 votes
2 answers
141 views

How to generate synthetic dataset with dependent and independent features?

I want to generate a synthetic dataset, in python, composed of 10 features. 5 are dependent among each others and 5 are independent. Generate dependent features. ...
I Sui's user avatar
  • 57
3 votes
0 answers
64 views

Where can I find implementation of the various improvements of K-nearest neighbors (KNN)?

I have been facing some challenges where traditional KNN algorithm perform well. I'd like to explore more advanced knn solutions. While researching possible solutions, I came across a paper titled <...
Lucas Morin's user avatar
  • 2,577
1 vote
1 answer
50 views

How to proceed when correlation plot doesn't show as much multicorrelation as is seen by statsmodels variance_inflation_factor in a regression task?

I am working with the kaggle Blueberry Yield prediction dataset. There are 17 columns including the target variable. Below is the correlation heat map: It can be seen that multiple features are ...
forest's user avatar
  • 111
0 votes
0 answers
21 views

RFECV giving same test scores for every feature size

I am trying to reduce the feature space of my training dataset down from the 16000 it currently is before engaging in Random Forest Classification. Initially I removed all features in less that 5% of ...
RMM's user avatar
  • 101
0 votes
0 answers
23 views

Tips regarding using a CNN to teach a model car (JetRacer) to park

I'm using a model car which can be remotely controlled and which has a front facing camera. My goal is to train a CNN which would be able to efficiently park this car in a simple environment (pictured ...
Jamess11's user avatar
0 votes
0 answers
20 views

object detection neural network's bounding box error converging instantly during training

I tried to make an SSD neural network for object detection. For now, i'm using 600 (700x700) training examples, i'm planning of using 1000 (I only have one class) or more if needed. However, there ...
KarimCool's user avatar
0 votes
0 answers
16 views

How do I get ktrain to return a multilabel output in tabular regression?

So the output vector I'm training a ktrain model on is 2 dimensional (the outputs are ['Temp', 'Velo']. And ktrain is even able to compute these when I use predictor.predict (see attached image), but ...
Sisyphus's user avatar
0 votes
0 answers
50 views

SVM taking too much time to train

I'm trying to train my ML model with Svm.svc from sklearn, but it is taking so much time, it won't even train for once. This happens only when kernel function is used. Currently i selected 10 Features ...
OctoCat's user avatar
1 vote
1 answer
511 views

LLAMA MODEL WITHOUT USING HUGGINGFACE API

Is it possible to obtain the llama model alone as open source code without using the Huggingface API so that it can be hosted on our server?
Anagha M P's user avatar
0 votes
0 answers
40 views

Clustering 2 lists

I have a dataset that contains 2 columns based on experimental data. Both columns contain lists of the same length. One column equals to the argument, where the list is the same for every sample. The ...
NND's user avatar
  • 1
0 votes
0 answers
32 views

Using model objects created using different scikit-learn versions

I have a project where I would like to make predictions using previously created model objects. The model objects where both build using scikit-learn but on different library versions (0.22.1 and 1.2....
magladde's user avatar
  • 101
0 votes
1 answer
37 views

How do I implement convolution partially on my dataset?

I'm training a neural network on the results of a CFD simulation (or rather, 300-ish simulations with different initial conditions). The dataset contains the values for temperature, density, velocity, ...
Sisyphus's user avatar
0 votes
1 answer
374 views

How to Implement padding and masking sequences for RNN

As an exercise, I'm building a network for binary classification of sequences (whether a sequence belongs to type A or type B). The network consists of an RNN with one LSTM layer, and on top of it an ...
kodkod's user avatar
  • 103
-2 votes
1 answer
828 views

ValueError: in python

How to solve this Error?---ValueError: Invalid classes inferred from unique values of y. Expected: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13], got ['Afghanistan' '...
S Ma's user avatar
  • 3
0 votes
0 answers
21 views

pytorch: implementing logistic regression: input dimension of torch.nn.Linear is input.flatten(start_dim=1)

I tried to implement a logistic regression class using pytorch. The following implementation worked. ...
thingy's user avatar
  • 1
6 votes
2 answers
1k views

How to remove the hotspots from given image by using Python and opencv?

In the picture below there are some regions which are very bright (i.e. more white). Some bright regions are wide and some are narrow or thin. The red box covers one such wide bright spot, and blue ...
S. M.'s user avatar
  • 173
0 votes
0 answers
43 views

Issues with sklearn.svm.SVC

I am trying to use the sklearn.svm.SVC on a relatively big dataset, 1.5k test/train samples, 512 features each, one sample per class (so, 1.5k classes). I know that SVC doesn't scale well, so at first ...
Ilya Kuleshov's user avatar
0 votes
2 answers
39 views

Data Analysis Process

How do you feed data from a data warehouse to Python for ad-hoc analysis? My day-to-day work is to answer ad-hoc questions, and 95% of the data I need is in our data warehouse. I often query data from ...
Bunny Eat Data's user avatar
0 votes
1 answer
33 views

Struggling with understanding RandomForest model with SMOTE

From what I understand my code is telling me that my base model is performing at 96% on it's training data, 55% on it's test data. And my SMOTE model is performing at ~96% on both. From my ...
GroupTheory14's user avatar
1 vote
1 answer
80 views

How do people usually handle creating an embedding vector of longer texts (32000 characters?

I have a set of podcast episode transcriptions in Arabic. I wish to convert these to embedding vectors so I can run a similarity comparison of them. Here's the summary statistics on the episodes: ...
Stan Shunpike's user avatar
0 votes
1 answer
114 views

Extract phrases/keywords that are SIMILAR to a python list of keyword/phrases, from a document

EDIT : If I had to match single worded phrases, I could first tokenize the text from the document and then calculate the cosine similarity of all the tokens with all the keywords from the ...
spectre's user avatar
  • 1,941
0 votes
0 answers
126 views

How do the predict() and predict_proba() for xgbClassifier() work?

I am using xgboost classifier for a binary classification problem. I used the predict() to get the class predictions (0/1) but I also used the predict_proba() method to get the class probabilities (my ...
user62198's user avatar
  • 1,091
0 votes
1 answer
48 views

What is the minimum ratio to consider the data set as balanced for the classification algorithm?

Can I consider 20% / 80% as a balanced dataset? My target variable ratio is 62% and 37%. I hope It is a balanced dataset. Please let me know if I am wrong. However, I would like to know, what is the ...
Iynga Iyngaran Iyathurai's user avatar
0 votes
0 answers
10 views

H20 AutoML Parallelism

Does H2O AutoML parallelize different models when launched via train ? (or you should specify it somehow?) If so, can you show an example ?
Master_Sniffer's user avatar
1 vote
1 answer
47 views

Convolution neural network loss increasing instead of decreasing

I am working on a binary image classification task in which I have greyscale images of size (1, 224, 224) (all normalized between 0 and 1) and a set of labels (0 or 1). I have around 2.6k images with ...
Akshit Sharma's user avatar

1
3 4
5
6 7
134