All Questions

27,534 questions
Filter by
Sorted by
Tagged with
5 views

How to interpret training and testing accuracy which are almost the same?

Note - I have read this post but still don't understand I have a Naive Bayes classifier, when I input my training data to test the accuracy, I get 63.05%. When I input my test data, the accuracy is 65....
5 views

Error : query data dimension must match training data dimension ,

Héy Guys new here and new to Data science in general , i'm trying to do a project for my uni where i use streamlit , the project is a disease detector and it's as folow : The user will put any data he ...
5 views

Hierarchical clustering with the consensus matrix as similarity matrix

I'm following this article on consensus clustering in Python programming. On page 7 the authors state that "The consensus matrix lends itself naturally to be used as a visualization tool to help ...
6 views

transformers require a large d_model even when the input cardinality is low?

I'm training a transformer encoder for an NLP task over character data, so the cardinality of my input is 26. I've noticed that if I want to create a strong model, I need make $x$ == my embedding ...
11 views

How to learn suggestion from drop-down queries?

I have a drop-down that suggests 10 out of 1500 classes based on the similarity of the classes to the query of the drop-down. I also record every key the user types in and the finally selected class. ...
6 views

For RNN's to work efficiently we vectorize the problem which results in an input matrix of shape (m, max_seq_len) where m is the number of examples, e.g. ...
4 views

SelectKBest for regression f_regression behaves weird when changing the random_state parameter when splitting

I am working on a regression project using the Audi dataset from Kaggle. I have looked at other notebooks and i saw that people use SelectKbest. I tried using the same thing, but when I was splitting ...
10 views

Hypertune xgboost to dealing with imbalanced dataset

My training data has extremely class imbalanced {0:872525,1:3335} with 100 features. I use xgboost to build classification model with bayessian optimisation to hypertune the model in range {learning ...
6 views

Person names dataset

Is there publicly available dataset containing real world people names? I’m looking for dataset containing names from different regions and cultures. I want to use it for research on name ...
6 views

Feature vector representation

I have a clarification. I have to create a classification model for certain set of documents. We are supposed to flag it anamoly or not based on certain terms in the document. My question is the terms ...
11 views

is this problem a multiclass case?

I'm trying to classify my textile design patterns (let's just think of it as medieval painting) what I understand of "multilabel classification" is like this: it outputs multiple possible ...
9 views

Is there any problem with the following Python+TF+Keras code for a custom loss function and network?

I am trying to code a custom loss function for variational autoencoder. I am not using mse for reconstruction loss since I am not learning p(x|z) ~ N(mu,I). Instead ...
7 views

Structuring extensive medical histories + demographic information for prediciting future medical outcomes

I'm looking for advice structuring extensive medical histories for predicting future outcomes, specifically hospital admissions. Let's say I want to predict the whether or not someone will be admitted ...
18 views

Calculation of PCA

Consider the following data set : Now we need to calculate the principal component analysis for this data. Here are the eigenvalues and eigenvectors calculated for the covariance matrix of this data :...
6 views

UNK in ludwig learnning model

I am using ludwig to train and test on the dataset. There is one independent variable that has a text data type. I used 'Parallel_CNN' as a text encoder for the independent variable. My dependent ...
10 views

How to specify output_shape parameter in Lambda layer in Keras

I don't understand how to specify the output_shape parameter in the Lambda layer in Keras/Tensorflow. The documentation says: ...
5 views

ImportError: DLL load failed while importing _pywrap_tfe when running exe file created by Pyinstaller

I have created the exe file of my code which uses tensorflow on gpu. It seems that building process using Pyinstaller goes fine. However, when I run the exe file in a system without GPU, I get the ...
11 views

Pre-processing data - Dataframe manipulation Time Series

I have a question in which I'm not entirely sure in which path to take. I'd appreciate if you could point me in the right direction. Below a screenshot of the a few records of my dataset. As you can ...
7 views

Trying to input a list of strings into my LSTM model

I'm training a model for dialogue act classification. I'm trying to write it so that I can enter a singular list of strings and receive a prediction for each of the strings. I've come to understand ...
16 views

Validation loss diverging away from the training loss

I used the XLNET for a sentiment classifier in determining whether a comment is positive or negative. I was able to get good results But when I plotted the validation and training losses I saw this ...
3 views

Is adding geo information to zip codes redundant in feature Engineering?

I was wondering if it is redundant to add geo information like elevation and distance between two points (between supplier and purchaser) as features to a model, if you already have country code and ...
5 views

Metaforecasting regression with an unbalanced dataset

I am trying to produce a metaforecast for the production of RES powerplants. I have the spec sheets of 100 plants, I have two forecasts for 95 of them and historical data for 83 of them, plus 2 extra ...
21 views

Does a confusion matrix have to sum to 100% for each class?

Does my confusion matrix looks correct ?
94 views

What is the advantage of a tensorflow.data.Dataset over a tensorflow.Tensor?

I have my own input data class. It has x and y as well as test and train values (1 Tensor for each combination). I noticed there is a Dataset class built in to TensorFlow. What is the advantage of ...
16 views

Feature extraction from sequence of images with Siamese Neural Network

I am trying to train a neural network to recognize certain actions in short movies. Each such movie consists of a fixed number of frames, each frame - the image is of course the same size, after ...
5 views

Clustering Analysis with Categorical Variables of Limited Range

I've been recently been implementing some k-modes/k-medoids type procedures in R, such as k-prototypes, and the daisy package. The output is fine. Yet, I wondered if anyone was aware of clustering ...
19 views

What is the opposite of baseline?

I have created a prediction model and on the one hand I have to compare it with other baseline models, and on the other hand, I have to compare it with the ideal approach (supported by additional data)...
10 views

DeepTaylor and LRP

I'm studying explainable AI. Is it possible to apply DeepTaylor or Layer-wise Relevance Propagation that was made for NNs with ...
14 views

14 views

Unsynchronized time series visualization

I would like to visualize a large amount of events composed of time serie windows. A typical event would be: Problem is, my events are not synchronized, and so if I plot them all, it would look like: ...
13 views

Best Feature extraction for at the end retrieving audio

I work on a machine learning algo, which basically learns sequences in an audio .wav and generates the most “logical” sequences. The algorithm learns features, so I generate MFCCs from the audio file. ...
19 views

I am trying to score CNN for packet data [closed]

The current error I am getting is:Shape mismatch: The shape of labels (received (32,)) should equal the shape of logits except for the last dimension (received (192, 3)).
8 views

Layer-wise Relevance Propagation with the swish activations

I'm studying approaches to explain DNN. I've found a lot of papers on LRP (and Deep Taylor decomposition), but they all explain NNs with the ReLU activations. I'm wondering why nobody applied LRP to ...
7 views

Can distortion be derived from inertia rather than recalculating it from scratch in case of kmeans?

I got this definitional difference between distortion and inertia from here: Two values are of importance here — distortion and inertia. Distortion is the average of the euclidean squared distance ...
40 views

How to Prepare data for LSTM

I'm having difficulties to wrap my head around how I can prepare my dataset to train an LSTM. Below is a screenshot of a subset representation of my dataset. There are several other feature not ...
24 views

When the shallow network has a small number of neurons, how it will need more sum-product?

I am reading a paper, where I am not able to understand the meaning of the following statement, can someone please help. a shallow network requires exponentially many more sum-product hidden units ...
13 views

The notation of $splits(label)$ under Random Forest

On the "Fair Forests: Regularized Tree Induction to Minimize Model Bias", it is written that We propose a simple regularization approach to constructing a fair decision tree induction ...
13 views

Interpretation of Autocorrelation plot

I am trying to understand better how to read the autocorrelation plot here for a timeseries data. I ran the following code and got the output as a chart show below. ...
19 views

How do we train a model to predict the relation between 2 samples?

I have two ddbb with around 60.000 samples each. Both ddbb have the same features, with values that can be numbers representing qualitative values of a category (for example 1 for wood, 2 for metal, 3 ...
28 views

Reduce multiclass classification targets to binary classification targets in scikit-learn

I would like to reduce multiclass classification targets to binary classification targets. Ideally, this mapping would happen within scikit-learn so the same transformation applies during both ...
11 views

Optimizing Tensorflow Graph Trace for Ram

I'm implementing an attention model in Tensorflow 2.0. One 'issue' I've run into is that building the graph for a very long decoder can use a lot of ram and take a long time to compute. That is, if ...
22 views

Solve for the set of coordinates that reduces the average distances between request and server in half

I generate a DataFrame with coordinates and distances to 3 servers. ...
25 views

Multiple Regression, Classification and Boundary Poins

I have two gangs which are doing crimes. And i want to classify them. Lets say I'm looking for a regression function: M(x1, x2) = w1x1 + w2x2 + w3 Now I have ...
8 views

I keep getting an error message in R while using twitteR

here is what I am doing: tweets=searchTwitter("walmart",n=3000, lang="en",since="2021-01-08",until="2021-01-10") And this ...
13 views

LinearRegression with fixed slope parameter

I have some data $(x_{1},y_{1}), (x_{2},y_{2}), ..., (x_{n},y_{n})$, where both $x$ and $y$ represent real numbers (float). I want use Scikit-learns LinearRegression model to fit a model of the form: \$...
4 views

How to calculate an average “will not close” date on opportunity data?

Please let me know if this question belongs elsewhere. In my simple data set focused on sales pursuit opportunities, I have the following columns available. Pursuit Name Status(Open, Won, Lost) Date ...
25 views

Take several points for each period in a machine learning model

Problem presentation I am working on a prediction model where I must find out if a boat will go back to the same offshore workplace after spending time in a port. When a boat is in a port, it can stay ...
6 views

Material Science dataset with feature-dependent inputs

I'm dealing with a material science/chemistry dataset where I have a bunch of duplicates inputs formulas corresponding to different values of a specific features like temperature. It looks something ...