Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

How to do a batch trainning of Pytorch model without using Dataloader?

I am doing a time series data training. I have to pad 0s to the data so the sequences have the same length. Because of 0s are padded, I have to mask them during the training, for Keras, it is simply ...
user avatar
  • 1
0 votes
0 answers
10 views

Creating tables from unstructured texts about stock market

I am trying to extract information such as profits, revenues and others along with their corresponding dates and quarters from an unstructured text about stock market and convert it into a report in ...
user avatar
  • 11
0 votes
4 answers
37 views

Where to learn which ML task is most appropriate for a problem?

There is now tons of material available on how to do certain (most popular) ML tasks and what kind of output you can expect. However I found that resources on how to select appropriate ML task/...
user avatar
1 vote
1 answer
14 views

Certain Image Augmentation Prevent Unet Model from Learning

I am training a Unet model for cell image segmentation from microscopy images. In order to help the model generalize better to different microscopes, I attempted to apply brightness augmentation to ...
user avatar
2 votes
2 answers
475 views

Suggestions for guided NLP online courses - Beginner 101

I would like to know from the data science community here for suggestions on nlp courses. I am new to NLP area and would like to take up a course which covers from basic to advanced concepts such as ...
user avatar
  • 2,275
0 votes
0 answers
10 views

Fine-tune network with fewer labels to increase accuracy

I'm trying to detect just dogs in input images. Would a pre-trained network on COCO significantly perform better if it was fine-tuned using COCO again (or another dataset) where all non-dog instances ...
user avatar
  • 293
2 votes
1 answer
24 views

How to build single pipeline with multiple estimators supporting fit and predict?

I have a ML problem where I want to divide the prediction task into subproblems (where I believe specialized models will do better). All these predictions tasks operate independently and will use the ...
user avatar
0 votes
0 answers
10 views

Can anyone help me with this error. I did the following code but it does not work and I am getting the following error

...
user avatar
2 votes
1 answer
22 views

What justifies feature scaling?

Although I can understand the significance of feature scaling in some cases (e.g. when gradient descent is involved), I don't feel I understand the necessity of this process in general. But there a ...
user avatar
  • 151
0 votes
1 answer
16 views

'NoneType' object has no attribute 'get_shape' in standard AdamOptimizer Initialization

I'm trying to construct a basic neural network in TensorFlow by following an example in Hands-On Machine Learning by Aurelian. The following code ...
user avatar
2 votes
2 answers
146 views

Usage of Word2Vec

Sorry for the basic doubt, I would like to know if I can use my Word2Vec straight for classification without using LSTM. My assumption is it’s not possible because the ordering of the words will not ...
user avatar
  • 109
0 votes
1 answer
26 views

Is gradient descent useful to get the least mean squared error in linear regression?

I am new to machine learning. I have read about the linear regression where-in the ideal model is a line which has the least mean squared error. In multi-variable linear regression we would have a ...
user avatar
  • 187
0 votes
0 answers
3 views

Predicting exponentially distributed output with multi inputs with the help of neural networks

I want to construct a Neural network regression model for data (8 inputs and 1 output). My problem is that my output has exponential distribution. When I normalized it in 'log scale' or 'min-max scale'...
user avatar
0 votes
1 answer
27 views

Transformers vs RNN basic doubt

I have a basic doubt. Kindly clarify this. My doubt is, When we are using LSTM's, We pass the words sequentially and get some hidden representations. Now transformers also does the same thing except ...
user avatar
  • 109
1 vote
1 answer
23 views

Getting equal distributions of data from different input sets

I am new to ML. I am trying to create a training dataset that is equally distributed between multiple lists, each of which have a different kind of data. How can I do this? I looked into ...
user avatar
0 votes
0 answers
8 views

How to perform a 0 mask for RNN in Pytorch

I have some time series data padded with 0s in the shape of (Batch, length, features). For more detail, I extracted MFCCs from audio files with (60,40), 60 frames, and 40 MFCCs for each audio file ...
user avatar
  • 1
1 vote
1 answer
32 views

What is the best method for learning a non-linear function $f(X)$ to predict multiple outputs?

I have feature data $X$ and four predictions $ (u_1, u_2, u_3, u_4) = f(X)$ with $u_1, u_2, u_3, u_4 \in \mathbb{R^+}$. $f$ is an unknown function (no assumptions on its properties) that needs to be ...
user avatar
0 votes
1 answer
23 views

ImportError: cannot import name 'Settings' from 'pandas_profiling.config' (/usr/local/lib/python3.7/dist-packages/pandas_profiling/config.py)

I'm trying to import pandas profiling on goggle colab. !pip install https://github.com/ydataai/pandas-profiling/archive/master.zip after succesfully installing ...
user avatar
0 votes
0 answers
13 views

Keras Tensorflow - CNN training performs well on gaming workstation (Windows) but not on high performance Nvidia DGX (Ubuntu)

I've been training on a local machine with Windows 11 (Version 10.0.22) with a 3070 Ti and recently have been able to access a DGX (Ubuntu 20.04.3) with 4 x V100s. Despite numerous re-runs the model ...
user avatar
  • 1
1 vote
0 answers
8 views

Good starting point for Natural Language Processing thesis

I want to do a masters thesis on Natural Language Processing, where I want to evaluate if given definitions meet certain criterias. Problem is, I'm new to NLP and I don't know where to start. I need a ...
user avatar
  • 11
0 votes
0 answers
8 views

How regularization relates to mean and median?

Recently read this in a blog. Blockquote Where L1 regularization attempts to estimate the median of data, L2 regularization makes estimation for the mean of the data in order to evade overfitting. ...
user avatar
0 votes
0 answers
21 views

Which model is best for the generating accurate answers of the Boolean questions?

I am trying to generate the question using T5 transformer answer of the questions but I am getting the error like below. here is the code. ...
user avatar
0 votes
1 answer
16 views

Clustering unknown product names

I have a parser that reads messages that contain product names. I would like to automatically cluster product names in clusters where each cluster would be one product and all the ways it can be ...
user avatar
  • 133
0 votes
2 answers
33 views

What is the implication of having features with less variation in a tree based model?

I'm training a tree-based model (e.g. xgb). I have some features with more than 90% values constant. Does it add value to the model since the variation in the data is minimal?. What would be the ...
user avatar
  • 231
0 votes
1 answer
27 views

Can ML or DL Predict output vector target?

I have output data as follows: Then I encode into : Then I convert into vector: The input of model is word embedding of sentence. My question is that: Can ML or DL return a vector output above? If ...
user avatar
  • 21
1 vote
0 answers
15 views

predict next purchase time of an item

I have a bunch of timestamps (purchase date from history), that looks like: [1658753101, 1658760061, 1658824861, 1658846461, 1658853961, etc] What I want is to based on that list predict next item ...
user avatar
  • 11
0 votes
0 answers
12 views

Feature selection with GridsearchCV [migrated]

I am trying to use GridSearchCV to optimize a pipeline that does feature selection in the beginning and classification using KNN at the end. I have fitted the model using my data set but when I see ...
user avatar
0 votes
1 answer
19 views

Method of accuracy improve for binary classification imbalanced dataset

I have an imbalance data set where the imbalance ratio No: Yes is 8:1. If I run classifiers on the groundtruth dataset I got recall and F2 measure for Naive bayes, Logistic regression, random forest. ...
user avatar
  • 137
0 votes
1 answer
23 views

does "unravelling" lstm units still mean one unit

I have seen images of lst and rnn units online, where they "unravel" the unit. Is this only one, singular, unit? If you have multiple units in a cell (layer), are both the cell state and ...
user avatar
  • 103
2 votes
2 answers
61 views

Predicting deterioration of equipment on a production line

Background There is a production line where there is a machine that interacts with some tools. The process goes as such. Machine makes a product Product is moved into the tool When the product is ...
user avatar
  • 123
0 votes
1 answer
22 views

Training on multiple timeseries at different locations

I have timeseries data that comes from a few locations. Location is not thought to be major factor, and although it might have some influence, details of locations aren't precise enough to be ...
user avatar
  • 71
2 votes
2 answers
85 views

Are all problems solvable using machine learning?

I am confronted with a relatively original problem which consists in predicting on which floor of a building audio recordings have been made. I have tried many machine learning approaches but none of ...
user avatar
0 votes
0 answers
32 views

Word stemming effect on dictionary-based Sentiment analysis

I am currently building a Farsi dictionary-based sentiment analysis model, based on thousands of Farsi tweets. Our team's approach has been as follows: ...
user avatar
1 vote
0 answers
39 views

How to guide exploration in reinforcement leanring/model predictive control/dual control problem

Consider the following optimization/control problem: We aim to maximize the cumulative reward R during the horizon H by every day allocating a portion of total budget B to our two different investment ...
user avatar
0 votes
0 answers
17 views

Point regression in ultra high resolution images using heatmaps

I'm looking for input on regressing the position of the corners of football pitch from a 5k image. There is severe fish-eye distortion and the angle is quite low so the far side corners of the pitch ...
user avatar
0 votes
0 answers
25 views

Priming a reinforcement learning agent with non-observation space data

Let's say I have a large, external source of information relevant to the domain I'm trying to train a reinforcement learning agent in. For example, in an environment where either observations or ...
user avatar
  • 101
1 vote
1 answer
29 views

How do I specify encoding in scikit-learn OrdinalEncoder?

Scikit-learn object OrdinalEncoder() allows the user to create a lineary based encoding principle for ordinal data, however the the codes are encoded randomly. Is there any way I can specify how the ...
user avatar
  • 11
0 votes
0 answers
20 views

How to evaluate Light Graph Convolutional Networks (LightGCN) correctly on sparse binary data?

I implemented the LightGCN at work to recommend k items to users according to the TensorFlow implementation of Microsoft: https://github.com/microsoft/recommenders/blob/main/examples/...
user avatar
  • 109
0 votes
0 answers
7 views

Local Deployment/Installation of Kubeflow on Windows

I am facing a major problem in installing Kubeflow locally on my Windows 10 Machine. Machine Specs - OS: Windows 10, RAM: 16GB Approaches Tried To Install Microk8s - Not Successful I get Cannot ...
user avatar
  • 101
0 votes
0 answers
21 views

how to handle categorical data that has two or more columns with unique values?

...
user avatar
0 votes
2 answers
35 views

Method of calculating sample size for a machine learning model [duplicate]

Is there any method of calculating sample size for a machine learning model/problem?
user avatar
  • 101
0 votes
1 answer
19 views

why we have problem with gradients when feature values are of different range?

A blog below mentioned. " Because different features do not have similar ranges of values, gradients may take a long time, oscillate back and forth, and take a long time before they can finally ...
user avatar
1 vote
1 answer
22 views

Found input variables with inconsistent numbers of samples: [908, 9080]

I have a dataset, I have reconfigured my tensors as a single 3072 sized line array. I have reconfigured the valid dataset and training dataset. You can find all of the information about my train, ...
user avatar
  • 11
0 votes
1 answer
31 views

Limitations of NLP BERT model for sentiment analysis

I am reading a paper, where the authors assess online public sentiment in China in response tot the government's policies during Covid-19, using a Chinese BERT model. The author's objective is not ...
user avatar
1 vote
1 answer
76 views

What is the best machine learning technique to fuse two spatial data sets?

I have two data sets, containing points geometry (X,Y) and a recorded car exhaust parameter (let's say, RP value), of an area of ...
user avatar
1 vote
0 answers
19 views

How to obtain data for warehouse throughput forecasting project?

I am trying to build a machine learning model to predict warehouse throughput. I do not have any domain information or data since I am supposed to build a generic prototype for my clients. I am going ...
user avatar
  • 11
0 votes
0 answers
10 views

Debug and address potential sources of errors in an xgboost ML model

I am training an xgboost ML model on dataset of shape ~3500x27. I had previously trained a model on about 1500 sample for market ...
user avatar
  • 302
0 votes
0 answers
12 views

Is it okay to use Least Squares Regression when comparing neural network accuracy?

I am comparing graphs of neural network accuracy for a study I am doing. I know you could use 5x2 cv and McNemars t-test however I already did the training without folding and this is not a binary ...
user avatar
  • 1
0 votes
1 answer
20 views

Detecting ellipses in an image

I’m trying to use CNNs to detect ellipses (of different shapes, sizes, orientation) in an image. How do I do this? I’ve tried using many centered ellipses as training (positive) examples and noise as ...
user avatar
0 votes
0 answers
34 views

Will time-series analysis not work for this?

Example: Let's say I'm running an experiment which takes 10 days and I take a measurement each day. So my "batch" consists of 10 rows of data, and let's say I've performed this experiment ...
user avatar
  • 1