All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

LSTM Feature engineering: using different Knowledge Graph data types

For a research project, I'm planning to use an LSTM to learn from sequences of KG entities. However, I have little experience using LSTMs or RNNs in general. During planning, a few questions ...
  • 1
0 votes
0 answers
3 views

As a newbie in Data Science, should I learn the main python language first or should I learn it's libraries

I'm currently learning as a beginner and I need guidance. Thank you
0 votes
0 answers
3 views

How to find significance for Gini coefficient changes?

I'm using the Gini coefficient to evaluate the performance of a model. Making some changes (feature selection, hyperparameter tuning, etc.) I created variant models with different Gini coefficients. ...
  • 594
0 votes
0 answers
8 views

How does batch normalization make a model less sensitive to hyperparameter tuning?

Question 22 of 100+ Data Science Interview Questions and Answers for 2022 asks What is the benefit of batch normalization? The first bullet of the answers to this is The model is less sensitive to ...
  • 162
3 votes
1 answer
43 views

Time series data for prophet model

I have a time series data that has some missing dates. For example, in the dataset below '2017-08-06' is missing. To enter this data in the Prophet model, do I have to create the data frame with all ...
  • 133
1 vote
0 answers
7 views

Algorithm suggestion for correlated models

I'm looking for suggestions on how to proceed with predicting on separate but correlated models. The example I will use is housing data. I have three inputs: Latitude Longitude 1-Google Street View ...
  • 495
2 votes
1 answer
283 views

URGENT: who knows how many matrix combinations this is

Input Matrix: A B C D E A 1 - - - - B - 1 - - - C - - 1 - - D - - - 1 - E - - - - 1 ...
1 vote
1 answer
11 views

Building a Time Series Model

I am working on building a time series model, but the dataset I have only has date features for the year; the month and date are not available. What would be a suitable model to use and is it even ...
0 votes
0 answers
5 views

How can SHAP feature importance be greater than 1 for a binary classification problem?

Let's say I build a binary classification model to predict survival on the Titanic. I then use SHAP to get feature importance for each feature. I see that the SHAP importance for the top feature, <...
0 votes
0 answers
17 views

Pre-trained BERT model running slow on CPU

I'm running a inference model using a pre-trained BERT model (BERTikal). The model works but is not fast enought running on CPU. It's taking about 5 minuts to classify a batch of 300 phrases. The the ...
0 votes
0 answers
10 views

Input size vs hidden state in RNNs

Im using PyTorch to implement RNNs on univariate time series data. This is the documentation for the RNN class: link I think I'm understanding the math behind an RNN cell. But I have an specific ...
  • 101
0 votes
1 answer
10 views

Preserve relations between data points when preprocessing

I am tasked with a project that aims to predict the probability of a product being returned before the product is even ordered. I have an excel containing a bunch of orders. In order to make ...
0 votes
1 answer
13 views

Proof of perpendicular distance of an observation from the Maximal Margin Hyperplane

I was reading about Maximal Margin Classifiers in "Introduction to Statistical Learning" and could not understand how is the perpendicular distance of an observation (which is a vector) from ...
0 votes
1 answer
25 views

Advantages of different tokenizers for NLP (specifically text generation)

What are the advantages of using different tokenizers? For example, let's take the sentence: "In Düsseldorf I took my hat off. But I can't put it back on." The treebank tokenizer yields: &...
0 votes
0 answers
11 views

Incremental semantic similarity with sentence embedding using sentence_transformers

I'm trying to find similar sentences to a given query sentence from a corpus. Also, I want to incrementally add new sentences to that corpus for future prediction without retraining the whole corpus ...
  • 1
0 votes
1 answer
29 views

Draw a decision tree with depth 2 that is consistent with the data

I am trying to come up with a solution to this for an exam preparation but cant come up with anything, dont know how to tackle it... if i use information gain the depth increases beyond 2. What would ...
0 votes
0 answers
10 views

Distance between deterministic policies that are not probability distributions

This question asks if there is a way to measure distance between policies that are in fact probability distributions. In the case of continuous control with deterministic policies where they take a ...
0 votes
0 answers
11 views

Predicted images are quite good with loss=0.20 while are black with loss=0.02

I'm trying to train a U-net with VGG16 as a backbone in order to recognize 4 classes: sky, rocks, trees and background in a dataset of about 10000 images. I'm using categorical crossentropy as a loss ...
  • 111
0 votes
0 answers
8 views

Time-Series Similarity & Clustering

I need to investigate Python-based tools for time series clustering and / or similarity matching on specific dataset and evaluate different approaches. Could you please suggest approaches that could ...
0 votes
0 answers
14 views

VC- dimension calculate

Let X = {1, 2, 3, ... , 100}. Let H be the class of all subsets of X that contain at least 20 and at most 80 elements. What is the VC-dimension of H?
0 votes
0 answers
13 views

- Models to rank sentences

I am working with tasks made by some occupations and am trying to find out the importance of these tasks within the occupation. My solution was to use tf-idf and then text rank and use word2vec and ...
0 votes
1 answer
13 views

Training XGBoost on time series features of varying sample length

I have some time series data that contain features that that go back anywhere from 5 to 50 years. I've considered imputation (e.g. taking the mean), but I'm not sure it's feasible to impute such large ...
0 votes
0 answers
14 views

Multiclass classification (gradient boosted trees) predictions distribution using softmax()

Let's consider a multiclass problem where the target is composed by 20% class 'A', 50% of class 'B' and 30% of class 'C'. The model is trained and then the class predictions are obtained via the ...
  • 133
3 votes
1 answer
30 views

Logistic Regression using Logisticregression() class

In the documentation of Logisticregression() offered by sklearn library, it states the following note: The underlying C implementation uses a random number ...
0 votes
1 answer
18 views

Kernel ridge regression (KRR), accuracy scale?

What does a good range for the accuracy score look like for the KRR model? For example, RMSE produces a value between 0 and 1, where values closer to 0 represent better fitting models. What's the ...
  • 101
0 votes
1 answer
19 views

Adding another 'hue' to a pairplot

I have plotted a pairplot in Seaborn with a hue, similar to the one shown below. I would like to add another hue by changing the shape of the markers based on another categorical feature. E.g., the ...
  • 1
0 votes
1 answer
14 views

Extract the embedding from a specific layer of MarianModel

I am using using MarianModel from the hub of HuggingFace for a translation task. Now I want to extract the embedding from the output of the last MarianEncoderLayer ...
0 votes
0 answers
9 views

How to calculate total observation time per focal individual across entire observation period?

I have a large amount of data for a set focal individuals that were under observation on a daily basis over three years. I am trying to calculate total times visible doing an activity and not oos (out ...
0 votes
1 answer
21 views

Transform dataset to regression problem by sorting?

I have a raw unlabeled dataset, and I want to design a model to perform a regression. In my dataset, it does not make sense to give each observation a value, but it does make sense to sort them. Can I ...
0 votes
0 answers
3 views

using xmgrace in batch mode

I am using xmgrace to plot 2D bar plot from the following input data: ...
1 vote
1 answer
24 views

Combine datasets of different domains to ehance generalizibility

so I try to implement an Emotion Classifier, which should detect several emotions from a text. There are several datasets for this (ISear, GoEmotions, etc.). However, a lot of them come from different ...
  • 13
1 vote
1 answer
24 views

Nearest Neighbor Recommendation System w/ categorical variables

I would like to build a recommendation system: no ratings are available at the time of recommendation, therefore only a purely context-based recommendation system is needed as input features answers ...
0 votes
1 answer
16 views

Likert Scale Target Variable

I have a case study where the target variable (a single factor) gauged through multiple items. the items are measured using 5-Likert scale (Never, Seldom, Sometimes, Often, Very often, Always) since ...
0 votes
0 answers
33 views

clustering customer base purchase behavior

I have a set of data and I want to know that whether they are necessary to add in the clustering analysis. Like ONEOFF_PURCHASES_FREQUENCY, I am not sure it is wether helpful in doing cluster analysis....
0 votes
0 answers
20 views

Why am I seeing these spikes in model loss curve

I am training an image classifier on 1152 images in 4 classes , I have used data augmentation too . ...
0 votes
1 answer
24 views

Feature scaling in Linear Regression

I always use Linearregression() class in sklearn library for creating a linear regression model. According to my understanding, we need feature scaling in linear ...
  • 23
0 votes
0 answers
21 views

How can create deliberately biased models?

I deal with an image classification problem with 3-class. I want to create a model which takes side to one specific class. I mean, while the model predicts a sample, if it is hesitant between class-1 ...
  • 101
3 votes
3 answers
437 views

How to train deep learning model on high dimensional dataset with limited memory and disk

For large datasets in terms of rows, usually it is handled by splitting data into pieces and feeding them into the model one at a time using tf.datasets or custom generator. However what if number of ...
  • 385
0 votes
0 answers
13 views

pix2pix TensorFlow Tutorials

I am trying to follow this tutorial, but I want to use my own dataset. The problem I am having is that in this tutorial they merge the realimage and the label image together in to one image( which ...
0 votes
0 answers
16 views

is it possible to turn a list of sentences into paragraph?

I have a problem and seeks advise, I have a couple of sentences like: ...
  • 1
0 votes
0 answers
6 views

What is the space and runtime complexity of sklearn MiniBatchNMF

I am trying to scale my virtual machine instance to be the necessary size but not waste extra space. What is the space and runtime complexity for sklearn MiniBatchNMF?
  • 1
2 votes
1 answer
29 views

Why is it an advantage "that Markov chains are never needed" to obtain gradients?

In the original GAN (Generative Adversarial Network) paper, Generative adversarial networks by I. Goodfellow, J. Pouget-Abadie, M. Mirza et. al. they state an advantage of the GAN is "that Markov ...
  • 21
0 votes
0 answers
18 views

Border values at which timeseries decrease and increase

I have a timeseries data of signals in stock market ([-1, 1]) and I want to find mean values at which I have down trend and upwards trend. I already used Moving ...
  • 31
0 votes
1 answer
22 views

How do I know that my weights optimizer have found the best weights?

I am new to deep learning and my understanding of how optimizers work might be slightly off. Also, sorry for a third-grader quality of images. For example if we have simple task our loss to weight ...
  • 3
1 vote
1 answer
23 views

How do you train an ML algorithm to achieve a desirable clustering?

Most clustering examples on the net are unsupervised learning. There is a given vectorization into a 2D space and the algorithm discovers clusters. However, what if the input data that I want it to ...
  • 117
0 votes
1 answer
19 views

Testing the impact of events on time series

Context I am working with product data for a retail company. I have the daily impressions (number of times it was viewed online) for all products over a 30 day period (can get more data). Here is the ...
0 votes
0 answers
29 views

What is the best practice for combining cross-validation with hyperparameter tuning and comparing preprocessing methods

The Goal Compare several preprocessing methods and models - while tuning hyperparameters for each model - without leaking information into the final generalization estimate, applying cross-validation (...
0 votes
1 answer
49 views

Binary classification performance difference between 0 and 1 class

I have trained a binary Random Forest classifier on a dataset containing 7M rows. I also set aside a holdout validation set of 1M rows that the training pipeline never sees. The dataset consists of ...
0 votes
0 answers
9 views

Embeded Google Maps map has delay on Oracle Analytics Cloud

I've been using Oracle Analytics Cloud for a while and wondered how to embed the Google Maps map (Dynamic Javascript Map) instead of using the default map. I finally made it work (using an API key ...
1 vote
1 answer
17 views

Newbie questions: real-time clustering of messages

I'm very much a newbie in NLP, so please accept my apologies if this is an obvious question, the wrong place to ask it or any other error I could be making. I am considering using NLP for some subset ...
  • 111

15 30 50 per page
1
2 3 4 5
690