All Questions
35,613
questions
0
votes
0
answers
2
views
Adaptive Generation in EBMs
I have a question about one Wikipedia article about EBMs: Why does it adapt without training?
EBM generators are implicitly defined by the probability distribution, and automatically adapt as the ...
- 1
0
votes
0
answers
5
views
Timeseries (InfluxDB): How to deal with missing data?
Question Description
We are performing a lot of timeseries queries, these queries sometimes result in issues, they are usually performed through an API (Python) and sometimes result in complete ...
- 101
0
votes
0
answers
4
views
Online machine learning with synthetic data
I am working on a neural network for which the training data can be easily synthetically generated on the fly in arbitrary quantities (Bayesian networks). I've been doing training by generating a new ...
- 1
0
votes
0
answers
6
views
Python and ridge classifier score and accuracy are correlated inversely
We are using RidgeClassifier() over tweets to predict sentiments. The first step is the stemming which was done using ...
- 163
1
vote
2
answers
20
views
Validation accuracy almost zero while training accuracy much higher
I am training a large dataset with CNN. The training dataset has 46182 samples and 1414 different classes , while the validation dataset has 14053 samples and 790 different classes.
The first question ...
0
votes
0
answers
8
views
Classification-Problem based on limited Dataset (Need keywords to search/ reading recommendations)
I have a dataset of about 200 test subjects, each with age, testscore and a one of two possible traits. I want to define some kind of function, where I input age and testscore, that predicts which of ...
- 1
0
votes
0
answers
5
views
Which CNN model to use for the classification(20 classes) of gemstones (diamonds, sapphire, ruby etc) based on digital photo images and huge data set?
Im trying to build CNN Model for the classification of precious stones (like diamonds, sapphire, ruby) based on digital images. So I have data set of labeled 150,000 gemstone certifications and the ...
- 1
0
votes
0
answers
13
views
How to check that a model learns some relation of data you input?
For instance, I make a model, and it performs regrettion task. But, when looking at a True v.s. Predict plot, scattering of data points is similar. In this case, how can I judge whether my model ...
1
vote
1
answer
16
views
S-Shaped Learning Curve
For a model that I'm currently testing, I get an S-Shaped curve when plotting the MAE over consecutive epochs, as shown in the image below.
I was curious if this indicates a problem with the model or ...
- 111
0
votes
0
answers
11
views
In ChatGPT, can we remove the reward-model part?
We collected the [good]/[bad] feedback from the web page.
Then we remove the [bad] feedback data.
Then we only use the [good] feedback data to train the text-generation policy-model.
- 631
0
votes
0
answers
10
views
General approach for "regime detection" within timeseries data
Assume a stock market type of dataset with a handful of timeseries, representing volume of trades, volatility of trades, and similar meta metrics. If necessary, assume that is on an hourly basis.
What ...
- 101
0
votes
1
answer
14
views
How do I fit huge dataset into GPU memory?
I have data that I read from feather file, and it's divided into x and y, where y is the label for the data. x is DataFrame and y...
- 101
0
votes
0
answers
4
views
Large Numpy array causes error when trying to load
I'm making an ai using image recognition, so I recorded each frames of me playing into a numpy array. It worked just fine when the first time I exported all the images and got the 6 thousand of them. ...
0
votes
0
answers
9
views
Grad-CAM for CNNs with GAP layer
I'm new to deep learning, so maybe this is a silly question...
Do any adjustments need to be made for applying Grad-CAM on CNNs that use a Global Average Pooling (GAP) layer right before fully ...
0
votes
1
answer
16
views
How do I handle "Greater than X" in a field of integers?
I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there ...
- 1
0
votes
0
answers
7
views
can I use tf.keras.layers.MultiHeadAttention for image classification task?
Could you please let me know whether it is possible to use tf.keras.layers.MultiHeadAttention() for the image classification task without using Vision Transformer techniqe?
0
votes
0
answers
6
views
0
votes
0
answers
13
views
Scaling/Normalization again
I would like to ask for normalization .. again.
I'm working on LSTM model having a 100x4 dataset used for timeseries prediction.
My question is on which set of data the scalar/normalizer should be run ...
- 101
1
vote
0
answers
8
views
Finding associated words to a named entity
Is there a way to find a list of associated words to a Named Entity?
For instance : let the Named Entity be FIFA. Now FIFA is a Football Organization and hence related to the term football and all the ...
- 11
0
votes
1
answer
14
views
Does a neural network need to be retrained on both old and new data?
I am new to neural networks and wondered if it is possible to improve an existing model without retraining over a pervious data set.
For example, say I have a neural network that has been trained on a ...
- 101
0
votes
0
answers
7
views
Lightgbm model tuning produces unexpectedly different hyperparameters for similar datasets
I am trying to tune a lightgbm model for each half of a dataset that I have split by a particular feature (stock ticker in this case). Both halves have the same number of features, somewhat similar ...
1
vote
0
answers
9
views
How to use/vectorize SpaCy output?
I'm performing preprocessing on a dataset that I'm planning on using as a feature set for an RF model. I don't have too much experience with NLP so just want to make sure I'm processing my text fields ...
- 11
0
votes
0
answers
11
views
How to make a predictive model using a timeseries data consisted of binary information?
I have a set of data that is showing the state of an object as a function of time. I would like to know what and how I should be utilizing machine learning to predict the state of the object at some ...
- 101
0
votes
0
answers
7
views
Inductive embedding technique for nodes without features
I am working on a project that involves graph embeddings. I have an homogeneous directed graph where the nodes have no features. The only available information are the edges between nodes. I need to ...
- 1
0
votes
0
answers
13
views
Custom loss function in keras with class weights for each batch
I am new to deep learning and tensorflow. I am working on a speech binary classification problem, trying to replicate a research paper. Number of samples in class 1 are 2700 approx and in class 2 are ...
- 1
0
votes
0
answers
9
views
Encoding of categorical variables to reduce the effect of erroneous labels
I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, ...
- 11
1
vote
0
answers
8
views
what is a data lake catalog technically?
Is a datalake catalog, a file (XML) that the datalake consults before storing retrieving data from a Hadoop database?
Thanks.
- 11
0
votes
0
answers
7
views
multi label classifictaion - single class or group labels?
I currently working on a vehicle dataset, with the goal of detection and classification vehicle types (car, bus, truck, motorcycle).
In addition, for each vehicle, I want to detect and classify each ...
0
votes
0
answers
3
views
Should we reinsert trend after doing forecast using detrended data?
Basically, when we detrend a signal, we detect and remove a linear component of that signal. This produces a stationary version of that signal. And we can use various forecasting algorithms to ...
- 88
0
votes
0
answers
11
views
Transformer doesn't generalize on a time-series data
Data
120 patients.
Each patient has 5.7556e+06 samples on average, each sample consists of 5 features stored as a continuous high-frequency (1000Hz) time series.
Labels are 13 discrete classes ...
0
votes
0
answers
10
views
Looking for a standardized API for terminology services
I'm looking for a standard to access terminology services. A terminology service provides an generalized API to multiple terminologies like ICD10, MedDRA, Snomed, Loinc etc. But I'm looking for a ...
- 101
0
votes
0
answers
6
views
Is the same dataset used to train the individual models (model1 and model2) and the ensemble model(stacking)?
I've trained two individual models. I stacked them and built an ensemble model.
Code snippet:
...
- 152
0
votes
1
answer
35
views
Specifics about ChatGPT's Architecture
Does anyone know of reliable sources that have written about the architecture of OpenAI's ChatGPT - specifically regarding the following?:
Number of hidden layers
Number of attention heads
...
- 101
0
votes
0
answers
13
views
How to improve pytorch network to be more precise
I have a pytorch neural network which is being trained on data containing multiple financial values and some other non numeric values translated to numbers with self created logic (for example if ...
- 101
0
votes
1
answer
15
views
Best practices for serving user-specific large models in a web application?
First execuse any naive statement you may find below, i'm a newcomer to the field.
How do web applications that integrate fine-tuning of large machine learning/deep learning models handle the storage ...
- 3
0
votes
0
answers
8
views
Loss function for model with two-outputs
I have created a model for this Kaggle competition that outputs a classification of the level of the disease (from 0 to 4) from an image of the retina.
I now want to blend the predictions for both ...
- 101
0
votes
0
answers
11
views
How to deal with ambiguous classification outputs that exceed the specified threshold but are too close together?
I have a simple classification setup (intent classification). Once an input is received it's parsed using Multinomial Logistic Regression and then a score is predicted for each class. I pick the ...
0
votes
0
answers
11
views
What are the best practices for creating a texture library and subtracting textures from an image?
I have an image with patches of text blended into the picture.
E.g.,
I want to filter out the textures in the background image to reduce the influence of the textures in the text areas. I already ...
- 101
0
votes
1
answer
34
views
high degree polynomial model with sklearn does not fit
The idea was to gradually raise the degree of the polynomial. Here is the code that implements creating a random dataset, fitting the polynomial of the CHANGE_ME ...
- 1
0
votes
0
answers
11
views
EagerTensor object has no attribute 'astype'
I am using trained tf model output (TF 2.8) and trying to convert it to numpy array.
...
- 101
0
votes
2
answers
67
views
ChatGPT's Architecture - Decoder Only? Or Encoder-Decoder?
Does ChatGPT use an encoder-decoder architecture, or a decoder-only architecture? I have been coming across Medium and TowardsDataScience articles suggesting that it has an encoder-decoder ...
- 101
0
votes
0
answers
10
views
How to make the model fitting to the inverse-square law in Python?
I am studying basics of physics. I would like to learn to make a graphical solution to the following problem:
In a class room a lighting meter was used to measure the illuminance E at the distance r. ...
0
votes
0
answers
15
views
How to do forecasting with categorical timeseries?
I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
- 101
0
votes
0
answers
6
views
Data Analytics Platforms
What are some of your go to tools for data analytics? I've been using Tesser Insights platform for cleaning and analyzing data.
It is pretty convenient and does not require code.
1
vote
0
answers
15
views
Alternatives to twitter for large daily or weekly samples for sentiment analysis
Twitter, with their API, including the free tier, has been a go-to source for collecting large samples of texts expressing sentiment on various topics of interest. I just started a project in December ...
- 111
0
votes
0
answers
2
views
tensorflow dataset 'plant_leaves' cannot be imported to google Colab : DownloadRetryLimitReachedError
I'm trying to upload a tensorflow dataset called plant_leaves but it won't work and it shows a runtime error :
1
vote
1
answer
18
views
Random forest - estimate range instead of exact value
I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on ...
- 11
0
votes
0
answers
11
views
How to deal with DataCollator and DataLoaders in Huggingface?
I have issues combining a DataLoader and DataCollator. The following code with DataCollatorWithPadding results in a ...
- 101
0
votes
0
answers
13
views
Can I use Deep Learning Algorithm between every two columns of excel with text data?
If my data has this format
Now can I use LSTM in between each column? I have to classify keyword into column B first and then Column A. are there any other ways I can look into? But I had larger data ...
- 21
0
votes
0
answers
20
views
Data set dimension required for high number (thousands) of output class
I have a data set of about 50.000 images of malware (malware binary converted to image) and I identified about 1400 classes as malware families within this dataset. I want to make a deep learning ...