All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

Data access to SAS

I hope this question is not off-topic, and apologize if it is. I'm not a data scientist per se, but as the person in charge of data architecture in my organization, I'd like to check my understanding ...
mrgou's user avatar
  • 133
0 votes
0 answers
6 views

How to work with temperature data

So I have some equipment temperature and i have outside temperature (both are collected daily) and I want to predict the equipment temperature. However, I'm new to this and unsure about which model to ...
Ria's user avatar
  • 1
0 votes
0 answers
6 views

Model evaluation approach allowing manual experimentation without data leakage

In supervised machine learning, are there any evaluation approaches beside using a fixed holdout test dataset, which allow me as a scientist to manually compare preprocessing approaches, without ...
thomas8wp's user avatar
  • 101
0 votes
1 answer
7 views

How to choose a loss function and how to calculate the loss for Text Generation in Generative AI?

For the classification problems, what loss functions can I choose ? For the translation problem how do I decide whether the translation is good and how to choose a loss function? And what about the ...
Qiulang's user avatar
  • 123
0 votes
1 answer
15 views

Data binning for interval data

I am trying to create a ML model for salary classification into 5 categories (0-90k, 90-120k, 120-180k and so on). The problem is that in my dataset almost all salary data is presented in intervals. ...
pinkkdeerr's user avatar
0 votes
1 answer
15 views

Custom loss function in python

I am trying to implement a custom loss function inspired by https://arxiv.org/pdf/2305.10464.pdf. That is: $ L(\mathbf{x}) = (1-y) \left\lVert \mathbf{x_{true} - \mathbf{x_{pred}}} \right\rVert^2 + y \...
Gst's user avatar
  • 1
0 votes
0 answers
3 views

How can i retrieve incorrect labelled predictions on new unseen data on image classification when i want to retrain my model later on?

I've recently made a binary image classification model with transfer learning. The model is used on an api and the predictions gets saved into a database. The problem is that it predicts images ...
Enes Aygun's user avatar
0 votes
0 answers
8 views

How to ensure that the output of my generative model is uniform?

Generative models transform noise to data. However, is there a way to ensure the output of my generative model follows a specific distribution, say uniform? More concretely, applying a scalar function ...
Butters's user avatar
  • 11
0 votes
0 answers
8 views

Tips for pre-processing long unstructured text?

I'm working on a React-Node project where I'm trying to pre-process some text before passing it to GPT-4 to perform information extraction out of it, the flow of the project is: User uploads document ...
Abdul Munem's user avatar
0 votes
0 answers
15 views

Unordered Set Classification Problem

In my setup I have one feature which is a sparse list representing categories. For example, let's say that we have M categories in the interval ...
dpalma's user avatar
  • 101
0 votes
0 answers
12 views

Machine Learning Predictive Machine

I am looking to hire someone to build a machine learning predictive machine. I have a dataset of 1000 data pts. Day 1000 = 0 Day 1001 = 1 Day 1002 = 1 Day 1003 =?
Russell lopez's user avatar
0 votes
0 answers
6 views

Transfer Learning - GoogLeNet - Training Times || Loss not converging || Pytorch

Hi Community and thanks in advance for the help. I am working on transfer learning - specifically GoogLeNet model with the Food101 Dataset. Code is below. I think everything is in order from data ...
James's user avatar
  • 1
0 votes
0 answers
6 views

Hyperparameter Widget

Could you please inform me if there exists a widget designed for the purpose of conducting hyperparameter optimization? I attempted to locate such a tool, but regrettably, I was unable to find one.
Gerardo's user avatar
1 vote
0 answers
16 views

How to derive at the expectation equation given in the paper "Video Diffusion Models"?

In the paper Video Diffusion Models, Section 3.1 mentions the following equation: $$ E_q[x^b|,z_t,x^a] = E_q[x^b|z_t] + (\frac{{\sigma}_t^2}{{\alpha}_t})\nabla_{z_t^b}\log q(x^a|z_t)$$, where $x^a, x^...
p1p13 's user avatar
  • 21
0 votes
0 answers
5 views

How to identify outliers on a box and whisker plot that seems to be compressed?

I have plotted box plots for the features of an ML problem, to identify outliers. I have scaled the data using a MinMaxScaler so that the scaled data is in the range [0,1]. For some columns, the two ...
Susmitha Acharya's user avatar
2 votes
1 answer
59 views

How does dependency information impact binary classification in multi-label prediction models?

TL;DR: I don't understand the dependency issue with binary classification (binary relevance) compared to multi-label prediction models. I often read in papers that some kind of "dependency ...
malisokan's user avatar
  • 123
0 votes
0 answers
8 views

training data of stable cascade vs stable diffusion

So according to this: https://waxy.org/2022/08/exploring-12-million-of-the-images-used-to-train-stable-diffusions-image-generator/ Stable Diffusion was trained on data from CommonCrawler. I believe ...
x89's user avatar
  • 191
0 votes
0 answers
11 views

I need help with asr_evaluation library

I have an assignment for which i need to find out the word error rate. Now asr_evaluation is supposed to be the library that i use but i am not able to use it. Help. Link: https://github.com/belambert/...
Push29's user avatar
  • 1
-2 votes
0 answers
11 views

Orange Report file [closed]

Report file created by an application (.ows) and saved as .report file type does not get associated with the application when application is saved and closed, and reloaded. Any clue?
Subramanian M R's user avatar
0 votes
0 answers
9 views

Implementing softmax regression using numpy

I was trying to implement softmax regression but my model failed why? ...
samsamradas's user avatar
-1 votes
0 answers
25 views

Best way to deal with a large dataset?

I am working on a project with a team of 4 people, on a relatively large dataset (around 500Gb). I cleaned the data and did some feature engineering, and now I want everyone to have simple access to ...
Mordechai's user avatar
1 vote
0 answers
18 views

Use text embeddings to map job descriptions to ESCO occupations

I'm trying to build a model to map job descriptions to ESCO occupations which is a taxonomy for job titles. Every ESCO occupations have a title, a description and some essential skills. Ideally I ...
GanaelD's user avatar
  • 11
0 votes
0 answers
11 views

Question regarding model inference consistency

I am doing some research on gaze estimation model L2CS provided in here. When you look at the inference predictions on provided image, when face/eye direction barely changes, inference predictions ...
JohnDoe's user avatar
0 votes
0 answers
5 views

HCC-TACE-datase

Is anyone worked with CT images. I am having problem in dicom images. I want to make a similar slice for all my segmentation image and liver image. Is there any tutorial or suggestion for that? Thank ...
Sumaiya's user avatar
1 vote
0 answers
22 views
+100

Using UMAP on text data (euclidean distance on jaccard distance matrix)

I am checking the capabilities of the UMAP dimensionality reduction algorithm, I am not sure whether the approach I am using is valid and does not violate the rules/limitations of this algorithm. ...
rkabuk's user avatar
  • 21
0 votes
0 answers
22 views

What is the Understanding level required in mathematics to become a great data scientist?

I am a civil engineer and i got some undergraduate courses in linear algebra, probability and statistics. I am pursuing some ML/DS courses in udemy so I'm just a beginner and I would like to become a ...
Camer Guy's user avatar
0 votes
1 answer
14 views

How to train a recommender based on user_features, item_features and likes?

It is my first time dealing with recommender systems, so I don't understand which algorithm should I use. I am given dataset with columns "user_id", "item_id", "like" (0 ...
GulNkt's user avatar
  • 1
1 vote
1 answer
20 views

Text Classification with unlimited labels, Text Extraction?

I'm looking to use ML to read in a blob of text, and extract a name from that text blob. (The blob is from an OCR result from an iPhone) The text blob varies in size, but the name is always present in ...
Matthew Knippen's user avatar
0 votes
0 answers
9 views

saving ML models with pickle to be deployed using Flask

I trained some ensemble Ml to predict, I needed to save with pickle so as to be able to deploy using Flask. To save with pickle I have tried several methods and read several articles but could not get ...
Kehinde Olatunji's user avatar
1 vote
0 answers
18 views

Multiclass matrix loss function in scikit-learn / xgboost / lightgbm

I have data with 4 classes: $c_1, c_2, c_3, c_4$. I'd like to create a classifier which has different scaling for the loss function per class combination: $$ \begin{bmatrix} 0 & l \left( \hat{c}_{...
Avi T's user avatar
  • 11
0 votes
0 answers
9 views

How will weights learn in CNN for multi class classification?

How are the weights of filters in a CNN can learn meaningful features in multiclasses classification if they keep changing as different images are passed through the network during training.Say we are ...
Jai's user avatar
  • 1
0 votes
0 answers
13 views

Recreating results from Research Paper

so I have been trying to recreate the results from this particular paper (Neural Collaborative Filtering). The dataset I use closely resembles this . I understand that I should my data into train and ...
Panos_42's user avatar
0 votes
0 answers
9 views

Elastic Net alpha value using GLMNET 4.1-8

Is it a valid method to “brute force” the alpha value for an elastic net? What I mean is trying alpha = .1, .2, .3, .4 and so on to 1.0 and looking at the highest R-squared value of each and choosing ...
user162172's user avatar
0 votes
0 answers
33 views

improve my f1_score for classification - pandas/sklearn

I would like advice on how to improve my f1_score for classification. I currently have something around 0.57. Dataset: lotWaferDie - lot, board and chip on which defects were measured string values ...
user162343's user avatar
2 votes
1 answer
41 views
+50

What type of technique can be used to solve this question?

Apology for the ambiguous title, I do not know the term. I have data of some products which a few variables: origin, weight, brand. For example: Product A = "China, 100g, Brand X" Product ...
lpounng's user avatar
  • 998
0 votes
0 answers
6 views

Can GraphSAGE be applied to characterize vrp data?

My scene is as follows: (1) Suppose there are 10000 vrp graph, each graph contains 100 nodes and each node is featured by its coordinates, i.e. (x,y) (2) My goal is to use these data to train a ...
jinkun Dong's user avatar
2 votes
1 answer
146 views

AutoDiff on different operations?

How it is possible to use automative differentiation (computational graph) on operations like - convolution? I know that 2d convolution can be represented by matrix multiplication. But what about 3d ...
Тима 's user avatar
0 votes
0 answers
23 views

Why the f1 score on validation dataset significantly higher than f1 score on testing dataset?

I'm using a TensorFlow model that look likes this: ...
Furno's user avatar
  • 1
0 votes
1 answer
13 views

Struggling with normalization/Standardisation for machine learning dataset

Sorry for what is probably a very obvious/rookie question. I'm currently doing a data science module for my degree and making very slow progress with the work. The case study i'm doing is around HR ...
Alex Ferry's user avatar
0 votes
1 answer
31 views

Build a Neural Network for Multi-output Regression

I have a network model that accepts about 25 inputs and outputs 3 actions. The outputs are: delta X and delta Y of the robot and the angle of the robot. After I enter the data into the model, I get ...
May's user avatar
  • 1
0 votes
0 answers
7 views

How to organize multi-layer data in Orange Data Mining?

I have data in the form of a MATLAB cell array in which: Rows are individual ROIs columns are image channels But each element of each column stores not only the mean intensity value of the ROI, but ...
DopeOmics's user avatar
0 votes
0 answers
4 views

Video / image recognition: Featurize movement

How would it be best to featurize movement of objects between video frames? Let's say I have a footage of a wolf pack and I am trying to understand where the Alpha is there. The best is actually not ...
Tagar's user avatar
  • 198
0 votes
0 answers
4 views

stabilityai/stable-cascade vs runwayml/stable-diffusion-v1-5

What are the major differences in these text-to-image AI models: stabilityai/stable-cascade runwayml/stable-diffusion-v1-5 in terms of architecture and performance?
x89's user avatar
  • 191
0 votes
1 answer
18 views

Cross validation

I do not get why in For cross validation should I use training set, or whole dataset? the responses say that cross validation must be done exclusively on training set. Doesn't the methods (for example ...
Curious student's user avatar
0 votes
0 answers
11 views

What are the most important evaluation metrics for anomaly segmentation?

When people talk about anomaly segmentation models, they often mention evaluation metrics like F1 score, AP, AUROC, and AUPRO. But which one really matters most when comparing models, and why? I'm ...
Mosh Geb's user avatar
0 votes
0 answers
14 views

Convolutional Network not Fully Learning MNIST (60% accuracy)

I have been working on a self-made Conv Net for some time. In addition to my network taking multiple epochs to increase the test-set accuracy, the accuracy flatlines at about 60% around 30 Epochs. ...
Mce Bab's user avatar
-1 votes
0 answers
6 views

How to get pmi score and count for NLTK everygrams / pmi and count all possible sizes of phrases in my text . Same as below for everygrams

...
98fly's user avatar
  • 1
0 votes
0 answers
9 views

Feedback on this plan to make a data pipeline

I am picturing something like the following: List internet sources you want to acquire data from. Study the source code structure for each site. Try to build an “extractor” for each individual site ...
Julius Hamilton's user avatar
0 votes
0 answers
19 views

Model improvement

Whenever I try to solve some ML problem I get stuck on the first model I choose. I understand the bias-variance tradeoff, but I think it is not the only way to debug a model. Are there any tools to ...
DimitrijeCiric's user avatar
1 vote
0 answers
17 views

Train a LLM to learn the entropy of the use case

I want to train a LLM (prefered Llama-2-13b) to learn the entropy of german texts - to be specific sports news. I use perplexity as training metric and want to check the training success after the ...
Christian01's user avatar

15 30 50 per page
1
2 3 4 5
738