Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
0 votes
2 answers
87 views

Sourcing (discounted) products customers want

Goal: Generate a list of 100 products per vertical (e.g. fashion, electronics) that the teams should source, discount, and list on the website over a specific period. You may assume all customers are ...
1 vote
1 answer
55 views

Do I need to standardize time series data in change point detection?

I have process data in time series data(0min, 1min, ... 999min). I don't know what does the variables mean. They are just written in X1, X2, ... X52. Each row means the data at the time. At certain ...
3 votes
1 answer
440 views

Segmentation Network produces noisy output

I've implemented a SegNet and SegNet ReLU variant in PyTorch. I'm using it as a proof-of-concept for now, but what really bothers me is the noise produced by the network. With ADAM I seem to get ...
0 votes
1 answer
161 views

Shape of Flattened Layer in CNN

If I have a convolutional layer with dimension (5,5,4), (i.e, 4 no. of 5x5x1 feature maps), what will be the dimension of the flattened layer, if I apply flattening ...
0 votes
1 answer
58 views

Machine Learning in Tensorflow

I am doing a work that is based on analyzing different Python libraries for Machine Learning. I chose to analyze Scikit-Learn, Keras, Tensorflow and Pytorch for being the most known ones. The idea was ...
0 votes
1 answer
2k views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where I am trying to predict if the data returns a 1 or 0. So your classic binary classification. I have my set of data that I have split into the dependent variables (...
5 votes
1 answer
443 views

Can I turn any binary classification algorithms into multiclass algorithms using softmax and cross-entropy loss?

Softmax + cross-entropy loss for multiclass classification is used in ML algorithms such as softmax regression and (last layer of) neural networks. I wonder if this method could turn any binary ...
0 votes
1 answer
755 views

ImportError: cannot import name 'Settings' from 'pandas_profiling.config' (/usr/local/lib/python3.7/dist-packages/pandas_profiling/config.py)

I'm trying to import pandas profiling on goggle colab. !pip install https://github.com/ydataai/pandas-profiling/archive/master.zip after succesfully installing ...
0 votes
1 answer
18 views

How to explain missing dates to a model?

I have this dataset that I'm trying to train a neural network on. The problem is that since weekend dates are not available, I am not confident in whether the model is able to account for that. ...
0 votes
1 answer
81 views

Algorithms for Vertex or Node Correspondence

Given a graph G, and another graph with the same number of vertices G’, one can define a vertex correspondence function f, from the vertex set of G to the vertex set of G’. The correspondence function ...
1 vote
2 answers
2k views

Can Precision-Recall be improved for imbalanced sample?

I tried out a few models on a highly imbalanced sample (~2:100) where I can get decent AUC from ROC (test sample). But when I plot precision-recall (test sample), it looks horrible. Kind of like the ...
3 votes
2 answers
518 views

How to treat the undefined values which make sense?

I'm currently trying to create a few features to improve the performances of a model. One of those features that I would like to create corresponds to the difference in days between a customer's ...
1 vote
1 answer
2k views

Input shape of dense layer in keras

I'm trying to build a new model ...
1 vote
2 answers
1k views

How best to embed large and noisy documents

I have a large corpus of documents (web pages) collected from various sites of around 10k-30k chars each, I am processing them to extract relevant text as much as possible, but they are never perfect. ...
1 vote
1 answer
70 views

How to deal with data (specially image data) where input and output both are images?

I have input dataset in the form of images and output data is also an images insteade of being labeled data. So it looks neither classification problem nor regression problem. Input and output iamges ...
5 votes
1 answer
435 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...
-2 votes
2 answers
42 views

Determining the threshold value for the neural network

I have a dataset with last name, first name, middle name of people participating in sporting events. I need to train a neural network that will match similar surnames, first names and patronymics. But ...
0 votes
0 answers
13 views

Feature Engineering a Recency feature

I have a customer scoring problem I'm working on specifically on predicting conversion and coming up with a probability score on conversion (using xgboost classifier atm). There's a feature I want to ...
0 votes
1 answer
124 views

Comparing RMSEs of multiple test sets having different sizes

The data I have is a time series data (stock returns), and I am training a Random Forest Regressor on it. Total observations = 2499 To better evaluate the performance, I have implemented rolling ...
0 votes
1 answer
351 views

Keras model is not learning - Validation loss does not decrease

I am training a regression model for crypto prediction and the model is not learning. When I train my model over say the period of 01-01-2021 till 01-01-2022 and I split the dataset into a train and ...
0 votes
1 answer
284 views

Support Vector Machine (SVM) for classification problem based on Earth Mover's Distance (EMD)

I would like to run SVM for my classification problem using the Earth Mover's Distance (EMD) as a distance measurement. As I understood the documentation for Python scikit-learn (https://scikit-learn....
0 votes
1 answer
399 views

What are the theoretical differences of multitask learning vs fine tuning based transfer learning?

Suppose, I have the following scenarios: I have a bunch of fruits, i.e., apple, orange, and banana. I simply made a multitask model, where my network first tell me which fruit it is, and then telling ...
6 votes
2 answers
3k views

Why do we need to concatenate in a U-Net?

You might be familiar with the U-Net, a machine learning network deceived for image segmentation. It's basically an encoder/decoder network with some direct links between encoder and decoder segments: ...
0 votes
1 answer
297 views

Data Augmentation for Regression ANN with low Sample Size

There is a Dataset of 65 tuples. I want to Augment new Data from this set and validate my ANN on the original Data. Is there a possibility, that my ANN already overfits on the augmentet Data. For ...
0 votes
1 answer
32 views

Can anyone help me understand this problem in my data?

I tried making a model using the autoTS library but the thing is in the result it gives me the following results. I checked everything there is no missing data but the original data had a missing ...
0 votes
1 answer
117 views

"Invalid value" in RMSprop implementation from scratch in Python

Edit 2: The regularization term (reg_term) is sometimes negative due negatative parameters. Hence S[f"dW{l}"] contains some negative values. I realize the reg_term has to be added before ...
0 votes
1 answer
16 views

Improving GPU Utilization in LLM Inference System

I´m trying to build a distributed LLM inference platform with Huggingface support. The implementation involves utilizing Python for model processing and Java for interfacing with external systems. ...
1 vote
1 answer
34 views

How to measure different models' feature importance using a generic and common standard?

I want to measure the feature importance of a series of models after training them. Most models have some built-in APIs that allow me to access their feature importance, but as far as I know, these ...
0 votes
1 answer
187 views

Clustering a variable based on another variable or set of variables

df11[['COMPONENT_ID','FIRMWARE','SERIAL','CRP0_VDDN']].head() Consider I have these four columns to analyse. I want to form say 3-5 clusters of COMPONENT_IDs with ...
0 votes
0 answers
21 views

Fuzzy Name Matching with Machine Learning. Input data encoding

I have a huge amount of data in my dataset: Last name, first name, date of birth of Indian residents and I need to match them for similarity. The matching is fuzzy, the data looks like this (names are ...
0 votes
1 answer
57 views

Test Error is extremely higher than Training error after gridsearch and crossvalidation

I'm currently working on a machine learning project. It's a supervised learning problem. My goal is to predict for given data of an animal(keeping,size,weight,...) ingredients(energy,vitamine etc..). ...
0 votes
1 answer
58 views

Sale Forecasting Problem -- Is it legit to use inventory level as a feature?

I'm working on a project to predict future sales for our company's products so that the supply chain can have better idea how much to restock. Detail about the model I'm working on: Model: LGBM (from ...
1 vote
1 answer
157 views

What model should I use for multiple time series input

I want to predict bacteria plate count in the water from time series(around 10000 values in a row) of water temperature on a one minute granularity, and other daily climate data including min and max ...
1 vote
1 answer
89 views

Using sensor data and a know reference point infer the position of a moving robot

Say, the robot is starting at a known position and I've data coming off of the robot as it traverses the grid layout. Exploiting the nuances captured in the data - like the implication of unequal rpm ...
1 vote
2 answers
3k views

Dealing with multiple distinct-value categorical variables

So, I've got a dataset with almost all of its columns are categorical variables. Problem is that most of the categorical variables have so many distinct values. For instance, one column have more ...
1 vote
2 answers
406 views

How interpret keras training loss without compare with validation loss?

I have several implementation of the same neural network, but each one with different starting parameter. This is one of my plot comparing the training loss of the base experiment with the training ...
1 vote
1 answer
59 views

How do I interpret probability results in conjunction with my known precision/accuracy/recall scores?

I have a Random Forest Classifier (trained with sklearn) modeling a binary data set. Here's what the configuration looks like (I've tuned it for precision intentionally): ...
1 vote
1 answer
65 views

Ngram based Langauge Models learned using an Encoder-Decoder Model

I have been going through a Ngram based Langauge Model learned using an Encoder-Decoder Model for Email smart compose. The program output only 1 prediction for given input. I want to know how to ...
-1 votes
0 answers
29 views

What two different formulas in SVC minimization problem means?

Im studying a Support Vectors Machine and for soft margin I found minimization problem in form like this: $$\min_{w,b} \frac{1}{2} \|w\|^2 + C \sum_{i=1}^l \xi_i$$ And this this formula seems pretty ...
0 votes
1 answer
143 views

For feature selection, do we use Chi-squared with Mutual Information together?

Or do we only choose one out of two for categorical data.
1 vote
0 answers
27 views

recognition of names, surnames and patronymics

is there an example of neural networks on Github or Kaggle that perform the task of recognizing identical surnames, first names and patronymics? I'm just learning neural networks so it's interesting ...
0 votes
0 answers
5 views

Does Factorization Machines accept continuous variables?

Most of the implementations I have seen of FM rely on an Embedding lookup matrix, restricting the variables that can be used to some categorical variable. Is there a way to use FM with both ...
3 votes
1 answer
531 views

Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way. The way I decided to go about this is template-based: each data type has a template ...
0 votes
2 answers
107 views

feature normalisation problem

I am very new to ML and have limited knowledge about it. I am having issue in feature normalization process. I have understood from the post that we need to normalize the training features and scale ...
0 votes
0 answers
20 views

Diffusion Models: Conditioning on Time vs. Noise Level

I am new to SE-Data Science, therefore I hope this is the right place to ask this rather theoretical question. In diffusion models we usually have a time variable which determines the noise schedule (...
1 vote
1 answer
119 views

Compressing profiles with a large number of dimensions

I 'think' this is a related question, but not sure how to apply it. I'm trying to build out a very crude recommendation system using Amazon ML, Facebook likes, and historical actions. So lets say we ...
1 vote
3 answers
705 views

How to deal with highly skewed (on counts) dependent variables?

I am working on a binary classification problem and the dataset consists of several variables which are count variables. For example, how many times a customer defaulted on a broadband bill payment in ...
1 vote
0 answers
46 views

Using SMOTE Train Model and Optimal Cutoff on Unbalanced Test Data

My original dataset has a binary dependent variable with 3% of the values being one. First, I split the original dataset into training and testing sets using an 80-20 split. Since it includes both ...
0 votes
1 answer
54 views

dolly 2.3b machine laerning using trainer.train

first time poster here so please forgive me and correct me on my posting mistakes... Im trying to teach the databricks/dolly-v2-3b llm, some data, which is just one sentence. In the future I would ...
3 votes
1 answer
80 views

Detect Missing Records in Dataset

I have a dataset that contains several measures from various widgets on a daily basis. While the widgets remain relatively stable over time, sometimes there are legitimate reasons for one to disappear ...

1
2 3 4 5
229