Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

Filter by
Sorted by
Tagged with
1
vote
1answer
15 views

How does joint probability generate new data in generative model

Learning about different Machine Learning concepts, I came across Generative and Discriminative model. To infer from what I have studied, generative model is based on P(x,y)(Joint probability ...
1
vote
0answers
8 views

How to estimate total correlation KL[q(z)||Πjq(zj)] of VAE after training (useful for latents disentanglement evaluation)

FactorVAE and β-TCVAE both use total correlation (TC) batch estimation for their objectives. Where TC is: $$ KL\bigl( q(z)||\prod\nolimits_{j} q(z_{j})\bigr) $$ both estimates are applied to $q(z|x)$...
1
vote
0answers
19 views

How to deal with time-series imbalance classification data?

I want to predict the user to buy or not a product in next month in the e-comercial site. I mainly using the past 1-year data to predict it. But I found the training data is imbalanced, and the buy(1)...
1
vote
0answers
10 views

Deep Q Learning - training slows down significantly

I'm trying to build a deep Q network to play snake. I designed the game so that the window is 600 by 600 and the snake's head moves 30 pixels each tick. I implemented the DQN algorithm with memory ...
1
vote
1answer
21 views

How to use Inception v3 in Tensorflow

I am trying to import Inception v3 in TensorFlow. I wish to apply it after reading this tutorial on object detection.
0
votes
0answers
7 views

should I re-initialize my optimizer and my scheduler before I try to fine tune my neural network on the different dataset?

I am doing NLP, and I have this block of Transformer body that was already trained on dataset A. Now I am interested in fine tuning this same Transformer on a new dataset B. In my Python code, should ...
2
votes
4answers
23 views

Image Classification using Single Class Dataset using Transfer Learning [closed]

I only have around 1000 images of vehicle. I need to train a model that can identify if the image is vehicle or not-vehicle. I do not have a dataset for not-vehicle, as it could be anything besides ...
1
vote
4answers
68 views
+50

scaling images for image classification

I am trying to do image classificaition with a dataset that contains images of different sizes. The images are in a folder called Train, which contains 4 subfolders callsed HAZE,RAINY,SNOWY and SUNNY. ...
0
votes
0answers
23 views

Algorithms and Techniques used for Route Optimization

What are the algorithms/techniques used for route optimization problems like VRP in Data Science? Vehicle Routing Problem (VRP) can be described as the problem of creating a set of optimal routes ...
0
votes
1answer
27 views

bad input shape (5634, 2)

I tried everything and I am not sure how to resolve the following error: "ValueError: bad input shape (5634, 2)" This is my first machine learning example so please bear with me. This is the python ...
1
vote
1answer
17 views

why is MSE of prediction way different from loss over batches

I am new to machine learning so forgive me if i ask stupid question. I have a time series data and i split it into training and test set. This is my code: ...
1
vote
1answer
14 views

NoSQL Comparison - Is this part of my Job?

For the more experienced Data Scientists here, i was asked to perform a case study on how Redis / HBase etc performs, compared to each other.How does data science play a role into this? Note that ...
0
votes
1answer
16 views

increase performances in neural networks

I am starting being interested in neural networks, and I am writing some code about it. But, differently from methods like support vector machines, random forests,..etc., to me it seems more like a ...
1
vote
1answer
21 views

How regularization helps to get rid of outliers?

I have heard regularization helps to get rid of outliers, how so? 'My intuition is, regularization shrinks parameter or even make it zero, and hence large value will have less effect on overall result'...
1
vote
1answer
18 views

Given n ordered sets, each containing 6 numbers, generate the next set in the sequence

I am facing the following problem in machine learning. Given n ordered sets, each containing 6 numbers, generate the next set in the sequence. The numbers in a set ...
0
votes
0answers
9 views

Using neural networks and user feedback for finding similar documents

I have a collection of (tens of) thousands of short documents, each containing a title and a short description of around 10-40 words. I am developing a method that finds similar/relevant documents ...
0
votes
2answers
45 views

Classification accuracy based on top 3 most likely classifications

My goal is to recommend jobs to job seekers based on their skill set. Currently I'm using an SVM for this, which is outputting one prediction, e.g. "software engineer at Microsoft". However, consider ...
1
vote
1answer
12 views

Lime Explainer: ValueError: training data did not have the following fields

I'm attempting to gather ID level drivers from my XGBoost classification model using LIME and I'm running into some odd errors. I'm using this link as a reference. Here is the overall code that I'm ...
1
vote
0answers
21 views

Hypothesis Testing- Independent variable Importance

I am learning Data Science and I have a confusion in one topic. I would like to describe my approach. I have understood the problem statement. I have used snowflake schema to join tables that have ...
0
votes
1answer
17 views

Classification of images of different size

I am doing image classification using Convolutional neural networks, but I have a problem, because the images I want to classify are all of different sizes. My code is the following: ...
0
votes
0answers
8 views

Load in many files from google drive into colab

I am trying to load in 30k images (600mb) from Google drive into Google Colaboratory to further process them with Keras/PyTorch. Therefore I have first mounted my Google drive using: ...
0
votes
0answers
11 views

How to change classification model architecture for a new target application

I'm new to Deep Learning with Keras. With some tutorials online for cat vs non-cat classification, I was able to compile this simple architecture for my own ...
0
votes
0answers
8 views

back propagation through time derivation issue

I read several posts about BPTT for RNN, but I am actually a bit confused about one step in the derivation. Given $$h_t=f(b+Wh_{t-1}+Ux_t)$$ when we compute $\frac{\partial h_t}{\partial W}$, does ...
0
votes
0answers
15 views

Why does gradient descent fail training a network for predicting times table?

Cross posted I am training a feedforwardnet with gradient descent traingd as backpropagation algorithm to predict times table. ...
0
votes
2answers
36 views

How to handle “year” variable for Machine Learning models

I have a "year" variable but I don't know which is the best way to handle it for a ML model, as it is a numerical variable, giving some sequence. Should I treat it as a categorical variable? Thanks ...
0
votes
0answers
27 views

Logistic Regression Plotting Learning Curve and Decision Boundary with Python

I already trained a dataset with Logistic Regression. However , I could not find any plotting code blocks of learning curve and decision boundary of my trained data. I put my codes at below. So , how ...
2
votes
1answer
31 views

Need explanation of a matrix multiplication

I'm reading the Deep Learning book by MIT. On the page 172, there's a part like this: $$ f^{(1)}(x)=h=W^Tx \tag{1} $$ $$ f^{(2)}(h)=h^Tw \tag{2} $$ Substitute (1) into (2), they got: $$ f(x)=w^TW^Tx $...
0
votes
0answers
9 views

Conceptual help generating a text adventure game using a GAN

I have built a playable dungeon crawler game that lets a character progress through a series of randomly generated rooms filled with doors, chests, stairs, etc. Ideally, I would be able to display a ...
0
votes
0answers
42 views

Pytorch CrossEntropyLoss expected long but got float

have you done some research before asking the question? Yes. I have done a lot of online searching, and others had similar problems. There solution was to use .float() when entering into the loss ...
4
votes
1answer
59 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
6 views

Is there a dataset that contains images of talking people with information about their mouth state ( open, closed, teeth showing, etc.)?

Is there a dataset that contains images of talking people with information about their mouth state ( open, closed, teeth showing, etc.)? I would like to classify images based on whether or not a ...
8
votes
1answer
147 views

Why does frequency encoding work?

Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't ...
0
votes
0answers
10 views

Need of value vector in transformers

I am reading the paper "Attention is all you need" (https://arxiv.org/abs/1706.03762). In transformer architecture, we have 3 vectors(key,value and query) for each word. I don't understand the need of ...
1
vote
1answer
60 views

NLP Transformers: How to get a fixed sentences embedding vectors size?

I'm loading a language model from torch hub (CamemBERT a French RoBERTa-based model) and using it do embed some sentences: ...
0
votes
4answers
72 views

What do you call the ratio of positive to negative samples?

I am working with a binary classifier and I want to express the "balance" or "skewness" of the training data using a metric. I want to reflect this ratio in a report, like this: ...
0
votes
2answers
30 views

Fine Tuning the Neural Nets

I have recently read about Fine Tuning, and what I want to know is, when we are fine-tuning our model is it necessary to Freeze the model and train only the top part of the model and then unfreeze ...
-1
votes
2answers
61 views

What is the approx minimum size of dataset required to build 90% correct model?

I am working with a financial dataset size which is around 3000. I have attempted the supervised-learning regression techniques and not able to go beyond 70% accuracy. Features: 10 Data size:3700 ...
0
votes
0answers
13 views

Actor Critic Model implementation

I am going to work on a project which requires implementation of A2C model using Tensorflow 2.0. I am new in the Machine Learning field and also in Python. These are topics which I have covered ...
1
vote
0answers
14 views

Is kFoldLoss simply the average classification error?

I'm using a kNN classifier in MATLAB to classify ECG signals, with 5 fold cross validation I get almost 97% accuracy. However, I'm not entirely sure if this is truly the accuracy value. By default the ...
0
votes
2answers
30 views

Binary Classification - One Hot Encoding preventing me using Test Set [duplicate]

I have a preprocessing pipeline that includes replacing missing values and onehotencoding for the categorical variables. When I try to use my model on the test set, it explains that the number of ...
0
votes
1answer
7 views

Face dataset organized by folder [closed]

I'm looking for a quite little/medium dataset (from 50MB to 500MB) that contains photos of famous people organized by folder. The tree structure have to bee something like this: ...
0
votes
0answers
14 views

How to Write a Tensor to PNG Image File? [migrated]

I am trying to write a tensor with the following properties to a PNG file. ...
0
votes
1answer
15 views

Calculating possible number of configuration

I am wondering how did they get the $19200$ possible configurations? Like, $5^6 = 15625$, where $6$ is the number of hyper-parameters:
1
vote
0answers
21 views

Predict with some probability the day of the month paycheck received through daily transactions

I am using R to do machine learning. I have daily shopping expenditure data of individuals over a couple of months and my goal is to be able to identify through their shopping pattern the day of the ...
0
votes
2answers
27 views

Difference between a parameter and a weight?

Is there any pedantic difference between referring to a specific "parameter" of a model as $\theta_1$ or referring to a specific "weight" of that model as $w_1$? Are they always the exact same thing?
1
vote
1answer
7 views

Parameter Adjustment based only on tagged predictions

not sure that this is the best place to post this but if not, please let me know if there is a better stack community. I have an anomaly detection method which has some parameters. I have some data ...
1
vote
0answers
20 views

types of metrics to evaluate the performance of regression models?

I have a conceptual question related to types of performance analysis of regression models. In general we have - R Square, Root Mean Squared Error and Mean squared Error to evaluate Regression ...
1
vote
2answers
24 views

Universal function approximation with fixed values (as vector or matrix)

I was thinking about way to represent/approximate universal function and came up with the idea that a plain fixed numbers could be used to represent pretty much any function on a fixed interval. I ...
0
votes
0answers
14 views

How to use a unique machine learning model for different types of inpute?

This question is generic because I am in the middle of pseudo code and sorry for not sharing the example of main code. The question is that suppose I have a neural network (or any other type of) model ...
0
votes
1answer
26 views

Neural network is not giving the expected output after training in Python

My neural network is not giving the expected output after training in Python. Is there any error in the code? Is there any way to reduce the mean squared error (MSE)? I tried to train (Run the ...