All Questions
35,926
questions
0
votes
0
answers
6
views
Problem of extreamly varied reward value in DDQN
I am trying to train my model by DDQN agent after creating a customized environment in gym. I am stating my hyper-parameters and other details here.
state shape = 5
action space = 0,1,2, ..., 100
...
- 103
0
votes
0
answers
16
views
How to create a custom labelled dataset for self-supervised learning on images
[SOLVED] The code has been updated: I wish to create an image dataset for self supervised learning, where I have a dataset of 1000 unlabelled images (.jpg files). I wish to create 4000 labelled images ...
0
votes
0
answers
4
views
Is there a way for CTC to output different types of blanks?
I am using a CTC loss for math handwriting recognition in Tensorflow/Keras. The output consists of a sequence of symbol ids, with a spatial relationship between every pair of consecutive symbols. For ...
0
votes
0
answers
14
views
LDA calculations manually
my question: has anyone ever done LDA calculations manually? I have difficulty in manual calculation. can someone help me to teach me for lda calculations manually.
- 1
0
votes
0
answers
7
views
Sklean :: NotOneValueFound: Expected one value, found 0 Why is existing working code now broker with Sklearn 1.2.2?
Sklean Problem
What is the compiler complaining regarding 'A task has failed to un-serialize'?
Entire new models to be defined for OneVsRestClassifier and OneVsOneClassifier. Planning to test my SVC ...
0
votes
0
answers
16
views
Which ML model should I use to reorder a schedule
I have a list of performers (up to 800 elements) and some requirements about distance, age and other stuff, like:
Performer should have 30 min to change outfit before appearing again in another group
...
- 101
1
vote
1
answer
10
views
How to identify certain term in a long document with NLP?
Given different long documents of the same type, e.g. certain type of report, I need to identify certain items within the report, such as certain item's amount, the name of the certain person etc. How ...
- 81
0
votes
0
answers
4
views
how do i compute the predictive covariance matrix from the posterior samples?
I have generated with EMCEE some posterior samples from a statistical model whose likelihood is a multivariate gaussian. it's a regression problem.
can you explain me how I can use these samples to ...
- 101
0
votes
1
answer
19
views
Training a network on time-series data with very long data
This question has come up a few times, but I haven't seen a lot of good answers. I have data where I have about 1000 samples and 3 time-series data for each sample. The time-series data is extremely ...
- 161
0
votes
1
answer
21
views
How Naive Bayes makes prediction based on scikit-learn?
I need to understand, how multinomial-naive-bayes can make prediction based on scikit-learn implementation.
I saw the source code but I want to understand the math behind it. Could you please explain ...
- 253
1
vote
0
answers
10
views
Generating artificial training data with encoder and classical algorithm
I would like to know if this idea has been tried before, and if so, where I can find more information about it.
This is an approach to generating artificial training data for segmentation tasks using ...
- 11
0
votes
0
answers
7
views
Statsmodel VAR - add future external data - forecastr
Currently I am forecasting future values using following code:
...
- 125
0
votes
0
answers
11
views
Perhitungan LDA
pertanyaan saya : apakah ada yang sudah pernah melakukan perhitungan LDA secara manual? saya mengalami kesulitan dalam perhitungan manualnya
- 1
0
votes
0
answers
16
views
Selecting an element in a sequence with self-attention networks
I have a doubt on I should set up the following problem:
Data:
My data is a tensor with shape (N, J, F) where N is the batch size, J is the sequence length, and F is the number of features of each ...
0
votes
0
answers
8
views
Faster R-CNN: are the proposal coordinates predicted in stage 1 fed as input to the bbox regressor of stage 2?
If I understand correctly, stage 2 of Faster R-CNN "refines" the proposals predicted by stage 1. However, this would require providing the coordinates from stage 1 as input to the bbox ...
- 347
0
votes
0
answers
7
views
Implementing a dataset to Computer Vision Article
I want to implement the PIE dataset in the AgentFormer arch.
AgentFormer uses ETH and nuScene datasets. I successfully run these datasets on this arch. However, I couldn't take a good way with the PIE ...
- 1
0
votes
0
answers
12
views
the accuracy of a random baseline
Hi everyone I am new in machine learning and deep learning field can someone explaining to me, What is the accuracy of a random baseline ?
0
votes
1
answer
32
views
Is there a machine learning model I should look into to predict the effect land topology will have on prevailing wind direction near bodies of water
I'd like to predict the change in wind velocity due to land near bodies of water. Warmer or colder land should change the wind velocity of nearshore breeze. I'd also like to predict wind shadows that ...
- 101
0
votes
0
answers
6
views
When evaluating an ML model, does it usually occure that some changes on the data should be made? made some changes,but not sure, if its correct to do [closed]
from sklearn.metrics import accuracy_score, precision_score, f1_score
import numpy as np
from tensorflow import keras
outs_all_flat = outs_all.reshape((71716*60,))
words_all_flat = words_all.reshape((...
0
votes
0
answers
13
views
find the parameter that minimize a multivariate distribution
i was trying to use scipy's minimize scalar to find the value of the parameter T that minimize the negative log-likelihood of a multivariate distribution with covariance matrix C. if i understood ...
- 101
0
votes
0
answers
10
views
Install package from conda-forge
I am trying to install the TRIQS package from conda-forge. As mentioned here, I used conda install -c conda-forge triqs on ...
- 101
0
votes
1
answer
35
views
How do GPT models go from token probabilities to textual outputs?
Suppose GPT-2 or GPT-3 is trying to generate the next token, and it has a probability distribution (after applying softmax to some output logits) for the different possible next tokens. How does it ...
- 103
0
votes
2
answers
39
views
Why is cycle consistency loss alone not sufficient to produce meaningful output?
Imagine an adaptation of CycleGAN, in which the discriminators were removed in lieu of using only cycle consistency loss. Well, it turns out that the original authors of Cycle Consistent Adversarial ...
0
votes
1
answer
20
views
Is there a standard order of operations for data preparation?
In what order should I do the following given a dataset:
(E)ncoding of Categorical Variables
(N)ormalization
(B)alancing of data
(I)mputation of Missing Values
(R)emoval of Duplicates/Infinity/...
- 15
1
vote
1
answer
24
views
Do you have to use clustering with SciKit-Learn's Mutual Information metric?
I'd like to calculate the mutual information between two datasets, but I'd prefer not to cluster them first.
I'm thinking of using SciKit-Learn's mutual_info_score ...
- 523
0
votes
1
answer
31
views
Single model or multiple models for predicting at each level in a multi-level classification problem
Given a flat structured data with features that can be considered hierarchical, where each feature is at a different level (e.g., Brand at the top level, Product, Color, and Size at different levels), ...
0
votes
0
answers
13
views
Accelerated learning when wrapping layers in a class
I am implementing a VGG-like network using Pytorch 1.13.1 (python=3.7.12) for image classification on the CINIC-10 dataset. The following two implementations turn out to have very different training ...
- 1
0
votes
0
answers
16
views
how to define search space over hyperparameters automatically?
I'm trying to automate retrain steps of our ML models. My aim is hyper-parameter tuning with current data (newer performance window) using same algorithm and features on production environment. ...
- 1
1
vote
1
answer
27
views
I used SMOTE-ENN to balance my dataset and it improved the performance metrics, but how can I be sure it's not overfitting?
The models were evaluated using 10-fold cross validation.
foldCount = StratifiedKFold(10, shuffle=True, random_state=1)
The models in question are XGBoost.
...
- 15
0
votes
1
answer
22
views
How many parameters does the vanilla Transformer have?
The original Transformer paper (Vaswani et al; 2017 NeurIPS) describes the model architecture and the hyperparameters in quite some detail, but it misses to provide the exact (or even rough) model ...
- 163
0
votes
0
answers
5
views
Couldn't get the correct values after denormalizing LSTM Model
I couldn't get the correct value of the actual stock price after I denormalized the prediction value. The actual stock price should be around 150+-, however, after denormalizing it only shows around ...
0
votes
0
answers
5
views
Optuina pruning during CrossValidation, does it make sense?
I'm currently trying to build a model using CatBoost. For my parameter tuning, I'm using optuna and cross-validation and pruning the trial checking on the intermediate cross-validation scores. Here ...
1
vote
2
answers
42
views
How to boost the performance of a single decision tree by adding additional trees?
I have a binary classification task and the data has imbalance issue (99% is negative and 1% is positive). I am able to build a decision tree that is carefully tuned, weighted, and post-pruned. Take ...
- 11
0
votes
0
answers
17
views
how each tree in random forest structured/built?
I'm new to machine learning and I want to use random forest for the problem I have. What I have done so far is I did the 80/20 split of the original data set.
I need to understand what will happen ...
- 1
0
votes
0
answers
17
views
GitHub Archived Repositories
I'm trying to build a model that observes patterns of source control usage, from how many files are changed per commit, how many contributors there are, even semantic analysis on the commit messages.
...
- 101
0
votes
0
answers
12
views
How to fit datasets accounting for standard deviation in Python?
I have multiple datasets of measurements with standard deviation and I would like to fit all the data (with a non-linear regression model) accounting for SD or SEM (it is quite similar to let the ...
0
votes
0
answers
11
views
fine tuning open ai model with historical data
i'm trying to understand more about training models and unsure how to approach this problem.
I have a bunch of historical financial data that I would like open ai to use as additional context when ...
- 101
0
votes
1
answer
13
views
moments of weight vectors in Adam
When performing backpropagation with Adam algorithm, are the moment and the second moment of the weight vectors calculated also for the weights in hidden layers?
- 124
0
votes
0
answers
22
views
Where can I find gpt-35-turbo in the Microsoft Pricing calculator?
Where can I find gpt-35-turbo in the Microsoft Pricing calculator? I don't see gpt-35-turbo in the model list:
https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/ ...
- 5,488
0
votes
0
answers
7
views
Feature selection in high-dimensional datasets with sparse features
What are the most effective techniques for feature selection in high-dimensional datasets with sparse features in the field of natural language processing?
- 101
0
votes
0
answers
10
views
Word2Vec Data Leak
I want to train a machine learning model that can determine the sentiment of tweets about different stocks.
To do this I have a dataset, lets call it A. For dataset A about 30% of the data is labelled....
0
votes
1
answer
9
views
Unsupervised ranking of samples
Say I have a dataset of n samples. I want to maximize every feature’s value. I’m not sure if feature 1 is more important than feature 2, etc. Are there any methods of ranking my samples out there? If ...
- 1
0
votes
1
answer
19
views
How to cluster based on sensor data? - My first data science job
I'm on my first (real), data, programming job. As everyone can imagine, this can be quite hard and I learn a lot from it, given I am a data science student in university. However, I am completely ...
- 1
0
votes
0
answers
10
views
How to get same output with reference model
I have a tflite model which was working with a random script on swift and I decided to add some more classes on an object detection aim. Training with the new classes didn’t work. The first model was ...
- 1
0
votes
1
answer
11
views
Integrating time context in a machine learning model
Basically, what I'm curious about, are there any methods in machine learning to make the model take into account events that happen in real time that affect the data points during that time period. ...
1
vote
1
answer
12
views
Filling na values with condition from other column
I am working on famous titanic dataset and I want to replace na values in Age column but on such a condition that these people whose Pclass=3 receive 25, Pclass=2 29 and Pclass=1 38.
I was trying to ...
- 11
0
votes
0
answers
15
views
Why facenet works better than siamese network
I have been reading about face recognition literature. I stumbled upon siamese networks with contrastive loss and the facenet paper. Both approaches use metric learning. The difference is mainly in ...
0
votes
1
answer
173
views
ChatGPT: How to use long texts in prompt?
I like the website chatpdf.com a lot. You can upload a PDF file and then discuss the textual content of the file with the file "itself". It uses ChatGPT.
I would like to program something ...
- 23
0
votes
0
answers
21
views
General question on the test and train data
Currently I'm working on a Kaggle problem. I have to predict an outcome by the given information. There are few metafiles for training a model with lots of features (>30). However, in the test file ...
- 1
1
vote
1
answer
27
views
Making a netcdf data using xarray
I am very very new to the world of data science as I only started using it in my new job so I would really appreciate help from the community experts (maybe also in simple words :)).
I am trying to ...
- 113