All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

Adaptive Generation in EBMs

I have a question about one Wikipedia article about EBMs: Why does it adapt without training? EBM generators are implicitly defined by the probability distribution, and automatically adapt as the ...
0 votes
0 answers
5 views

Timeseries (InfluxDB): How to deal with missing data?

Question Description We are performing a lot of timeseries queries, these queries sometimes result in issues, they are usually performed through an API (Python) and sometimes result in complete ...
  • 101
0 votes
0 answers
4 views

Online machine learning with synthetic data

I am working on a neural network for which the training data can be easily synthetically generated on the fly in arbitrary quantities (Bayesian networks). I've been doing training by generating a new ...
  • 1
0 votes
0 answers
6 views

Python and ridge classifier score and accuracy are correlated inversely

We are using RidgeClassifier() over tweets to predict sentiments. The first step is the stemming which was done using ...
  • 163
1 vote
2 answers
20 views

Validation accuracy almost zero while training accuracy much higher

I am training a large dataset with CNN. The training dataset has 46182 samples and 1414 different classes , while the validation dataset has 14053 samples and 790 different classes. The first question ...
0 votes
0 answers
8 views

Classification-Problem based on limited Dataset (Need keywords to search/ reading recommendations)

I have a dataset of about 200 test subjects, each with age, testscore and a one of two possible traits. I want to define some kind of function, where I input age and testscore, that predicts which of ...
  • 1
0 votes
0 answers
5 views

Which CNN model to use for the classification(20 classes) of gemstones (diamonds, sapphire, ruby etc) based on digital photo images and huge data set?

Im trying to build CNN Model for the classification of precious stones (like diamonds, sapphire, ruby) based on digital images. So I have data set of labeled 150,000 gemstone certifications and the ...
0 votes
0 answers
13 views

How to check that a model learns some relation of data you input?

For instance, I make a model, and it performs regrettion task. But, when looking at a True v.s. Predict plot, scattering of data points is similar. In this case, how can I judge whether my model ...
1 vote
1 answer
16 views

S-Shaped Learning Curve

For a model that I'm currently testing, I get an S-Shaped curve when plotting the MAE over consecutive epochs, as shown in the image below. I was curious if this indicates a problem with the model or ...
0 votes
0 answers
11 views

In ChatGPT, can we remove the reward-model part?

We collected the [good]/[bad] feedback from the web page. Then we remove the [bad] feedback data. Then we only use the [good] feedback data to train the text-generation policy-model.
  • 631
0 votes
0 answers
10 views

General approach for "regime detection" within timeseries data

Assume a stock market type of dataset with a handful of timeseries, representing volume of trades, volatility of trades, and similar meta metrics. If necessary, assume that is on an hourly basis. What ...
  • 101
0 votes
1 answer
14 views

How do I fit huge dataset into GPU memory?

I have data that I read from feather file, and it's divided into x and y, where y is the label for the data. x is DataFrame and y...
0 votes
0 answers
4 views

Large Numpy array causes error when trying to load

I'm making an ai using image recognition, so I recorded each frames of me playing into a numpy array. It worked just fine when the first time I exported all the images and got the 6 thousand of them. ...
0 votes
0 answers
9 views

Grad-CAM for CNNs with GAP layer

I'm new to deep learning, so maybe this is a silly question... Do any adjustments need to be made for applying Grad-CAM on CNNs that use a Global Average Pooling (GAP) layer right before fully ...
0 votes
1 answer
16 views

How do I handle "Greater than X" in a field of integers?

I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there ...
  • 1
0 votes
0 answers
7 views

can I use tf.keras.layers.MultiHeadAttention for image classification task?

Could you please let me know whether it is possible to use tf.keras.layers.MultiHeadAttention() for the image classification task without using Vision Transformer techniqe?
0 votes
0 answers
6 views

Ordinal Encoding for Differing Categories

As an example, I have a dataset of available games. ...
  • 1
0 votes
0 answers
13 views

Scaling/Normalization again

I would like to ask for normalization .. again. I'm working on LSTM model having a 100x4 dataset used for timeseries prediction. My question is on which set of data the scalar/normalizer should be run ...
  • 101
1 vote
0 answers
8 views

Finding associated words to a named entity

Is there a way to find a list of associated words to a Named Entity? For instance : let the Named Entity be FIFA. Now FIFA is a Football Organization and hence related to the term football and all the ...
0 votes
1 answer
14 views

Does a neural network need to be retrained on both old and new data?

I am new to neural networks and wondered if it is possible to improve an existing model without retraining over a pervious data set. For example, say I have a neural network that has been trained on a ...
  • 101
0 votes
0 answers
7 views

Lightgbm model tuning produces unexpectedly different hyperparameters for similar datasets

I am trying to tune a lightgbm model for each half of a dataset that I have split by a particular feature (stock ticker in this case). Both halves have the same number of features, somewhat similar ...
1 vote
0 answers
9 views

How to use/vectorize SpaCy output?

I'm performing preprocessing on a dataset that I'm planning on using as a feature set for an RF model. I don't have too much experience with NLP so just want to make sure I'm processing my text fields ...
0 votes
0 answers
11 views

How to make a predictive model using a timeseries data consisted of binary information?

I have a set of data that is showing the state of an object as a function of time. I would like to know what and how I should be utilizing machine learning to predict the state of the object at some ...
  • 101
0 votes
0 answers
7 views

Inductive embedding technique for nodes without features

I am working on a project that involves graph embeddings. I have an homogeneous directed graph where the nodes have no features. The only available information are the edges between nodes. I need to ...
  • 1
0 votes
0 answers
13 views

Custom loss function in keras with class weights for each batch

I am new to deep learning and tensorflow. I am working on a speech binary classification problem, trying to replicate a research paper. Number of samples in class 1 are 2700 approx and in class 2 are ...
  • 1
0 votes
0 answers
9 views

Encoding of categorical variables to reduce the effect of erroneous labels

I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, ...
  • 11
1 vote
0 answers
8 views

what is a data lake catalog technically?

Is a datalake catalog, a file (XML) that the datalake consults before storing retrieving data from a Hadoop database? Thanks.
  • 11
0 votes
0 answers
7 views

multi label classifictaion - single class or group labels?

I currently working on a vehicle dataset, with the goal of detection and classification vehicle types (car, bus, truck, motorcycle). In addition, for each vehicle, I want to detect and classify each ...
0 votes
0 answers
3 views

Should we reinsert trend after doing forecast using detrended data?

Basically, when we detrend a signal, we detect and remove a linear component of that signal. This produces a stationary version of that signal. And we can use various forecasting algorithms to ...
0 votes
0 answers
11 views

Transformer doesn't generalize on a time-series data

Data 120 patients. Each patient has 5.7556e+06 samples on average, each sample consists of 5 features stored as a continuous high-frequency (1000Hz) time series. Labels are 13 discrete classes ...
0 votes
0 answers
10 views

Looking for a standardized API for terminology services

I'm looking for a standard to access terminology services. A terminology service provides an generalized API to multiple terminologies like ICD10, MedDRA, Snomed, Loinc etc. But I'm looking for a ...
  • 101
0 votes
0 answers
6 views

Is the same dataset used to train the individual models (model1 and model2) and the ensemble model(stacking)?

I've trained two individual models. I stacked them and built an ensemble model. Code snippet: ...
0 votes
1 answer
35 views

Specifics about ChatGPT's Architecture

Does anyone know of reliable sources that have written about the architecture of OpenAI's ChatGPT - specifically regarding the following?: Number of hidden layers Number of attention heads ...
0 votes
0 answers
13 views

How to improve pytorch network to be more precise

I have a pytorch neural network which is being trained on data containing multiple financial values and some other non numeric values translated to numbers with self created logic (for example if ...
  • 101
0 votes
1 answer
15 views

Best practices for serving user-specific large models in a web application?

First execuse any naive statement you may find below, i'm a newcomer to the field. How do web applications that integrate fine-tuning of large machine learning/deep learning models handle the storage ...
0 votes
0 answers
8 views

Loss function for model with two-outputs

I have created a model for this Kaggle competition that outputs a classification of the level of the disease (from 0 to 4) from an image of the retina. I now want to blend the predictions for both ...
  • 101
0 votes
0 answers
11 views

How to deal with ambiguous classification outputs that exceed the specified threshold but are too close together?

I have a simple classification setup (intent classification). Once an input is received it's parsed using Multinomial Logistic Regression and then a score is predicted for each class. I pick the ...
0 votes
0 answers
11 views

What are the best practices for creating a texture library and subtracting textures from an image?

I have an image with patches of text blended into the picture. E.g., I want to filter out the textures in the background image to reduce the influence of the textures in the text areas. I already ...
  • 101
0 votes
1 answer
34 views

high degree polynomial model with sklearn does not fit

The idea was to gradually raise the degree of the polynomial. Here is the code that implements creating a random dataset, fitting the polynomial of the CHANGE_ME ...
0 votes
0 answers
11 views

EagerTensor object has no attribute 'astype'

I am using trained tf model output (TF 2.8) and trying to convert it to numpy array. ...
0 votes
2 answers
67 views

ChatGPT's Architecture - Decoder Only? Or Encoder-Decoder?

Does ChatGPT use an encoder-decoder architecture, or a decoder-only architecture? I have been coming across Medium and TowardsDataScience articles suggesting that it has an encoder-decoder ...
0 votes
0 answers
10 views

How to make the model fitting to the inverse-square law in Python?

I am studying basics of physics. I would like to learn to make a graphical solution to the following problem: In a class room a lighting meter was used to measure the illuminance E at the distance r. ...
0 votes
0 answers
15 views

How to do forecasting with categorical timeseries?

I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
  • 101
0 votes
0 answers
6 views

Data Analytics Platforms

What are some of your go to tools for data analytics? I've been using Tesser Insights platform for cleaning and analyzing data. It is pretty convenient and does not require code.
1 vote
0 answers
15 views

Alternatives to twitter for large daily or weekly samples for sentiment analysis

Twitter, with their API, including the free tier, has been a go-to source for collecting large samples of texts expressing sentiment on various topics of interest. I just started a project in December ...
0 votes
0 answers
2 views

tensorflow dataset 'plant_leaves' cannot be imported to google Colab : DownloadRetryLimitReachedError

I'm trying to upload a tensorflow dataset called plant_leaves but it won't work and it shows a runtime error :
1 vote
1 answer
18 views

Random forest - estimate range instead of exact value

I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on ...
  • 11
0 votes
0 answers
11 views

How to deal with DataCollator and DataLoaders in Huggingface?

I have issues combining a DataLoader and DataCollator. The following code with DataCollatorWithPadding results in a ...
  • 101
0 votes
0 answers
13 views

Can I use Deep Learning Algorithm between every two columns of excel with text data?

If my data has this format Now can I use LSTM in between each column? I have to classify keyword into column B first and then Column A. are there any other ways I can look into? But I had larger data ...
0 votes
0 answers
20 views

Data set dimension required for high number (thousands) of output class

I have a data set of about 50.000 images of malware (malware binary converted to image) and I identified about 1400 classes as malware families within this dataset. I want to make a deep learning ...

15 30 50 per page
1
2 3 4 5
713