All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

What should I do with a dataset where the number of targets in regression is greater than the number of samples in feature?

Suppose that I have a dataset for regression. This dataset has a feature vector with 2000 samples. However, 200 of them have the target value. I am wondering how I can use the other 1800 samples in ...
0 votes
1 answer
77 views

Can I use regression to solve a multiple equation problem

I'm working on a problem which is a multiple equation. I have a group of people and each person in the group is working on different tasks (e.g. n tasks in total). Each person in this group is working ...
0 votes
1 answer
17 views

tf.keras.layers.LSTM: where did all these parameters came from?

Why does this code... model = tf.keras.Sequential() model.add(tf.keras.layers.LSTM(9,input_shape=(6,1),activation='relu')) print(model.summary()) ... print 396 ...
2 votes
1 answer
113 views

Regression problem - too complex for gradient descent

I try to predict temperatures values as function of time and different parameters. The temperature curve look like a "ramp" with some "gauss peaks" on regular intervals. So, I try to build a ...
0 votes
0 answers
2 views

Variational AutoEncoder using a matrix as dataset with a time component or ...?

The project is to repair data/smoothing. I would like to use a matrix of shape (5,13,3922): 5 differents types 13 features, one of them is the Times series, dates in format yyyy-mm-dd that I turned ...
0 votes
1 answer
91 views

Trimming "unused" neurons from the bottleneck of an autoencoder

I'm working with autoencoding data in segments, and working with the latent space afterwards (I am also working on VAEs, but this segment of the project concerns deterministic AEs). I've noticed that ...
1 vote
1 answer
10 views

Chi-square test results interpretation

I am comparing with Chi Square the distributions of two categorical variables. Both have the same number of classes. After counting each class per variable, I obtain very similar counts but the p-...
0 votes
0 answers
7 views

Autoencoders are fitting anomalies too good

I have a set of ~ 5000 greyscale images with resolution of 64x128. I want to do an unsupervised anomaly detection. As a first try, I chose convolutional autoencoders (AE) and trained an AE model. I ...
1 vote
1 answer
13 views

how can I fix my model to accurately assess zero values?

I'm building a random forest model that is designed to predict tree canopy cover. I'm using an R software environment and satellite remote sensing data as my predictors. I'm okay with under or over ...
4 votes
1 answer
72 views

Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

My questions follow the below page 4 excerpt from Hochreiter's LSTM paper: If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
0 votes
1 answer
51 views

Survival analysis on time series data for predictive maintenance

I want to train a survival analysis model for predictive maintenance on an asset (confidential, let's say it's a motor). The dataset consists of hourly readings of multiple sensors, the type of motor, ...
1 vote
1 answer
3k views

ValueError: cannot reshape array of size 36276416 into shape (96,227,227,1)

I am running my LeNet code with LFW, but when I run it, I am getting the following error message: Here is the code that it is getting the error ...
0 votes
0 answers
8 views

Need malformed csv files

Is there a resource one can use to find malformed cvs files? I tried creating my own but I can only come up with so many errors. It is for a project I am working on to read and visualise malformed csv ...
1 vote
1 answer
348 views

Is it a good practice to evaluate model performance by comparing the metrics of rescaled (inverse transformed) predictions and true target values?

I am now working with a Linear Regression for a time-series regression problem (I am sorry that I cannot say too much about the problem and feature vector due to NDA). I scaled both the input values ...
-3 votes
0 answers
7 views

How python help in healthcare

Python, the god of all programming languages; has emerged as the savior for staggering growth in the Healthcare revolution. Global charts have been smashed with the best Python applications being ...
-1 votes
0 answers
15 views

What are tools and lesson do I need to make an AI drone?

Recently, I've been exploring AI drones online, and I've conceptualized an exciting idea: creating an advanced automated AI drone. This drone would be capable of flight controlled through text ...
1 vote
1 answer
127 views

Transfer learning with many small datasets

Context I am working on a NLP-model that can classify documents into one of N categories. I have document data from a number of different customers. The document topics are similar across customers ...
1 vote
1 answer
4k views

Error with pandas dataframe (needs to be 1-dimensional)

I am trying to determine the conformal predictions for my model with my data. But it gives me following error that occurs at icp.calibrate(X_cal, y_cal) : ...
0 votes
1 answer
4k views

cuDNN isn't found FWD algo for convolution. How to TRAIN DARKNET ON GE FORCE GTX 1650

ISSUE: while training Darknet with GE FORCE GTX 1650 using following: CUDA 11.0 cuDNN 8.0.5 OPENCV 4.5 Model starts training with config file details as below for [net] section: ...
2 votes
1 answer
991 views

How to classify neutral sentiments using BERT

We can do text classification as positive and negative as mentioned in below notebook. But is there any way to classify neutral sentiment also? https://colab.research.google.com/github/google-...
1 vote
1 answer
238 views

What is the difference between kdeplot's "cut" and "clip" parameters?

In Seaborn's kdeplot, is there a semantic difference between the cut and the clip parameters?...
0 votes
1 answer
887 views

bad regression performance on imbalanced dataset

My current dataset has a shape of 5300 rows by 160 columns with a numeric target variable range=[641, 3001]. That’s no big dataset, but should in general be enough for decent regression quality. The ...
0 votes
0 answers
30 views

When is sampling bias acceptable?

Overview: Dataset is small and a bit messy and the task is to classify 5 classes wherein the targets are ordinal. Feature Engineering and Selection, Model Tuning, etc. did not produce acceptable ...
0 votes
1 answer
366 views

When will the mlp give constant prediction?

I have a regression task (to predict price for finanical market.) I build a mlp to do the regression. I found mlp will stop at giving a constant prediction, which i think it's useless. Does this mean, ...
2 votes
2 answers
358 views

How to implement Semantic Search in R or Python

I have a task to provide semantic searching capabilities. For example, if I have a dataset of resume and if I search for "machine learning" than it should return me all resumes which have data science-...
3 votes
2 answers
183 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
1 vote
1 answer
162 views

A dataset has skewness = 1 with missing data. Standard deviation around median is 1.5. How much data will be unaffected?

There is no other description about the data, if it is univariate, bivariate, etc. neither the type of distribution is given. I recently came across this question, I would like to know how skewness ...
1 vote
2 answers
4k views

Adding a new token to a transformer model without breaking tokenization of subwords

I'm running an experiment investigating the internal structure of large pre-trained models (BERT and RoBERTa, to be specific). Part of this experiment involves fine-tuning the models on a made-up new ...
0 votes
0 answers
52 views

what is the genesis of the term - the clinical significance?

The literature suggests that p-value represents "clinical" significance. (p-value, is a number whose value can range from zero to one.)To me,p-value does not measure clinical significance....
0 votes
1 answer
112 views

How to identify new potential customers from existing customers profile?

I have firmographic data of all the possible customers. Data includes sales, profitability, capital, organisation size, geographic location, industry, etc. What is the best way of identifying new ...
0 votes
1 answer
400 views

How to model predict on Tensorflow model that has feature columns?

So far all the coding examples using feature columns do not have examples of how to format their model.predict(...). I tried using raw string and also putting them ...
0 votes
1 answer
12 views

XGB predict_proba estimates don't match sum of leaves

When using an XGB model in the context of binary classification, I observed that the test estimates given by predict_proba were close but not equal to the results I ...
0 votes
1 answer
131 views

Scaling and handling highly correlated features in tabular data for regression

I am working on a regression problem trying to predict a target variable with seven predictor variables. I have a tabular dataset of 1400 rows. Before delving into the machine learning to build a ...
0 votes
1 answer
63 views

Hadoop, Spark and Cloud

It seems Hadoop, Spark, and different versions of Clouds offer facilities to store and analyze big data. There are some articles comparing Hadoop and Spark (for example, this article). There are also ...
1 vote
3 answers
334 views

Forecasting/predicting techniques for qualitative data?

I have a food alert dataset composed of nominal qualitative variables, such as type of alert, country of origin, action taken, etc. as well as the date on which the alert was recorded. What ...
3 votes
2 answers
772 views

Gensim doc2vec error: KeyError: "word 'senseless' not in vocabulary"

I am new to machine learning and tried doc2vec on quora duplicate dataset. new_dfx has columns 'question1' and 'question2' which has preprocessed questions in each row. Following is the tagged ...
0 votes
0 answers
7 views

What is the reference time point relative to which the capital-gain and capital-loss features of the UC Irvine Adults dataset are measured?

The Adults dataset available in the UC Irvine Machine Learning Repository is based on the 1994 census data (USA census, I presume). The dataset has two features named ...
1 vote
0 answers
10 views

Are there any public medical image databases?

I'm not sure if I'm asking this in the right place, please direct me to the right place for me to ask this if I am not. I'm working on a project to look at images of the inner ear from an otoscope and ...
2 votes
2 answers
1k views

What are the audio features to best describe a music?

I'm working on the content-based filtering part of a recommender system for an audio streaming project. I firstly used the k-mean algorithm with music genres and one-hot encoding to classify musics ...
2 votes
2 answers
1k views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
1 vote
1 answer
2k views

How to quantize weights in forward pass during training in Keras?

In Keras, I would like to train a network with binary weights in the manner of Coubariaux, et al., but I cannot figure out where the quantization (binarization) should occur within the code. A core ...
1 vote
0 answers
251 views

Matching financial reconciliation data / matching multiple rows based on column values

I'm working with financial reconciliation data and the ask is to train the algorithm to match transactions (that are otherwise manually matched if the existing application didn't because not all the ...
3 votes
1 answer
659 views

What is the state of the art method for synonym detection?

In Natural Language Processing and Computational Linguistic what methods are deemed as SOA for similar word extraction? Can anyone direct me to those resources?
4 votes
1 answer
626 views

How does a Neural Net handle an unseen class for a Categorical Feature?

Let's say I train a Neural Net, and I have a Categorical Feature X. During training, there are only 3 classes seen in feature X; A, B, C. Now, let's say I want to make predictions from this trained ...
1 vote
1 answer
18 views

How can a citation dataset (like Cora) have strongly connected components?

This website https://snap.stanford.edu/data/cit-HepPh.html shows that the High-energy physics citation network has strongly connected components and it's driving me crazy. A SCC would mean that you ...
1 vote
1 answer
9 views

Visualizing clusters of 3D images / volumes

I have a dataset of 3D (medical) images / volumes, think CT/MRI recordings. In other words, each of my samples consists of a 3D array of grayscale values, typically in the range of e.g. 1000x1000x500. ...
0 votes
0 answers
15 views

How can I know the expected inputs to a huggingface Trainer?

I'n a Colab notebook by huggingface here they print the raw dataset keys as such: ...
1 vote
0 answers
5 views

Problem with HDI dataset

I am new to Orange3. I am trying to follow the same steps in the following article https://orangedatamining.com/blog/data-mining-covid-19-epidemics-part-1/ But unfortunately the HDI dataset that is ...
0 votes
2 answers
881 views

The actual results and results from pickle files are not matching in pandas for DBSCAN clustering

I've built a DBSCAN clustering model. The output result and the result after using the pickle files are not matching. Based on HD and MC column, I am clustering WT column. ...
0 votes
1 answer
12 views

Understanding batching in pytorch models

I have following model which forms one of the step in my overall model pipeline: ...

15 30 50 per page
1
2 3 4 5
738