All Questions

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

How to get pmi score and count for NLTK everygrams / pmi and count all possible sizes of phrases in my text . Same as below for everygrams

...
98fly's user avatar
  • 1
0 votes
0 answers
4 views

Feedback on this plan to make a data pipeline

I am picturing something like the following: List internet sources you want to acquire data from. Study the source code structure for each site. Try to build an “extractor” for each individual site ...
Julius Hamilton's user avatar
0 votes
0 answers
12 views

Model improvement

Whenever I try to solve some ML problem I get stuck on the first model I choose. I understand the bias-variance tradeoff, but I think it is not the only way to debug a model. Are there any tools to ...
DimitrijeCiric's user avatar
1 vote
0 answers
7 views

Train a LLM to learn the entropy of the use case

I want to train a LLM (prefered Llama-2-13b) to learn the entropy of german texts - to be specific sports news. I use perplexity as training metric and want to check the training success after the ...
Christian01's user avatar
0 votes
0 answers
5 views

Patterns in weights of trained model?

Apologies for a naive question. Let's say I am training a simple feed-forward neural network using stochastic gradient descent with a fixed architecture, learning rate, number of training epochs, and ...
user101010's user avatar
0 votes
0 answers
6 views

What should I do with a dataset where the number of targets in regression is greater than the number of samples in feature?

Suppose that I have a dataset for regression. This dataset has a feature vector with 2000 samples. However, 200 of them have the target value. I am wondering how I can use the other 1800 samples in ...
Aaron's user avatar
  • 201
0 votes
0 answers
4 views

Variational AutoEncoder using a matrix as dataset with a time component or ...?

The project is to repair data/smoothing. I would like to use a matrix of shape (5,13,3922): 5 differents types 13 features, one of them is the Times series, dates in format yyyy-mm-dd that I turned ...
Adurrow's user avatar
1 vote
1 answer
12 views

Chi-square test results interpretation

I am comparing with Chi Square the distributions of two categorical variables. Both have the same number of classes. After counting each class per variable, I obtain very similar counts but the p-...
crbl's user avatar
  • 111
1 vote
0 answers
10 views

Autoencoders are fitting anomalies too good

I have a set of ~ 5000 greyscale images with resolution of 64x128. I want to do an unsupervised anomaly detection. As a first try, I chose convolutional autoencoders (AE) and trained an AE model. I ...
vinodh_eee's user avatar
2 votes
1 answer
33 views

how can I fix my model to accurately assess zero values?

I'm building a random forest model that is designed to predict tree canopy cover. I'm using an R software environment and satellite remote sensing data as my predictors. I'm okay with under or over ...
Priya Patel's user avatar
0 votes
0 answers
8 views

Need malformed csv files

Is there a resource one can use to find malformed cvs files? I tried creating my own but I can only come up with so many errors. It is for a project I am working on to read and visualise malformed csv ...
StudentP's user avatar
-4 votes
0 answers
10 views

How python help in healthcare

Python, the god of all programming languages; has emerged as the savior for staggering growth in the Healthcare revolution. Global charts have been smashed with the best Python applications being ...
Sakshi Gaba's user avatar
0 votes
0 answers
17 views

What are tools and lesson do I need to make an AI drone?

Recently, I've been exploring AI drones online, and I've conceptualized an exciting idea: creating an advanced automated AI drone. This drone would be capable of flight controlled through text ...
user162294's user avatar
0 votes
0 answers
32 views

When is sampling bias acceptable?

Overview: Dataset is small and a bit messy and the task is to classify 5 classes wherein the targets are ordinal. Feature Engineering and Selection, Model Tuning, etc. did not produce acceptable ...
easymoneysniper's user avatar
1 vote
1 answer
18 views

XGB predict_proba estimates don't match sum of leaves

When using an XGB model in the context of binary classification, I observed that the test estimates given by predict_proba were close but not equal to the results I ...
Juan Felipe Salamanca Lozano's user avatar
0 votes
0 answers
7 views

What is the reference time point relative to which the capital-gain and capital-loss features of the UC Irvine Adults dataset are measured?

The Adults dataset available in the UC Irvine Machine Learning Repository is based on the 1994 census data (USA census, I presume). The dataset has two features named ...
Evan Aad's user avatar
  • 163
1 vote
0 answers
11 views

Are there any public medical image databases?

I'm not sure if I'm asking this in the right place, please direct me to the right place for me to ask this if I am not. I'm working on a project to look at images of the inner ear from an otoscope and ...
Skymero's user avatar
  • 11
3 votes
1 answer
668 views

How does a Neural Net handle an unseen class for a Categorical Feature?

Let's say I train a Neural Net, and I have a Categorical Feature X. During training, there are only 3 classes seen in feature X; A, B, C. Now, let's say I want to make predictions from this trained ...
the man's user avatar
  • 139
1 vote
1 answer
18 views

How can a citation dataset (like Cora) have strongly connected components?

This website https://snap.stanford.edu/data/cit-HepPh.html shows that the High-energy physics citation network has strongly connected components and it's driving me crazy. A SCC would mean that you ...
StackExchanger's user avatar
0 votes
0 answers
18 views

How can I know the expected inputs to a huggingface Trainer?

I'n a Colab notebook by huggingface here they print the raw dataset keys as such: ...
Ameen Izhac's user avatar
1 vote
0 answers
5 views

Problem with HDI dataset

I am new to Orange3. I am trying to follow the same steps in the following article https://orangedatamining.com/blog/data-mining-covid-19-epidemics-part-1/ But unfortunately the HDI dataset that is ...
Sara Raad's user avatar
1 vote
0 answers
13 views

Transfer learning for tabular data

I wonder if transfer learning can be used in tabular data similarly to how it's used in neural networks for image recognition. My idea would be to train a "general" model and then "...
Dudelstein's user avatar
0 votes
1 answer
18 views

tf.keras.layers.LSTM: where did all these parameters came from?

Why does this code... model = tf.keras.Sequential() model.add(tf.keras.layers.LSTM(9,input_shape=(6,1),activation='relu')) print(model.summary()) ... print 396 ...
BsAxUbx5KoQDEpCAqSffwGy554PSah's user avatar
0 votes
0 answers
6 views

Does Fine Tuning with Custom Label Build Upon the Capability of Zero Shot Classification or Does It Train from Scratch?

The task is to classify email text bodies into exclusive categories like feedback, complaint etc. I have a labelled dataset available having about 350 samples. I have tried the ...
Della's user avatar
  • 335
0 votes
0 answers
4 views

Using shape files in Orange

Is there any way to use shape files in orange for Geomaps? Otherwise how can I fetch shapes for states/districts not in the default ones provided. For context I want to use shapes of districts in an ...
Damitr's user avatar
  • 101
0 votes
0 answers
6 views

Access feature in custom compute_loss method for HuggingFace transformers

I'm running into a Hugging Face issue when fine-tuning a model and want to access other features during my custom compute_loss function. However, I am unable to ...
mochmoch's user avatar
0 votes
0 answers
13 views

Create model for manufacturing defect detection using python

Has anyone done some work on manufacturer defect detection? I need some input on this. I have done some research and found some examples like TensorFlow/Keras and CNN, but I need some real-time ...
Reetesh Nigam's user avatar
0 votes
0 answers
11 views

Building a dataset for classification

I'm thinking of building a powershell script classifier using different architectures of neural networks. I have approximately 6k powershell scripts (3k malicious, 3k benign). My questions are: How ...
freaks's user avatar
  • 1
0 votes
0 answers
9 views

Tensorflow SegNet architecture

I was unable to find a complete description of the SegNet architecture for image segmentation (specifically, the decoder layers). Therefore, I would like to clarify the correctness of my ...
D .Stark's user avatar
  • 101
1 vote
1 answer
20 views

How do you train a seq2seq model on sequences longer than its sequence length?

I was reading the GPT original paper here and in section 3.5 they mention evaluating on the CoQA dataset. I checked GPT has a sequence length of 512, yet most of the sequences in the CoQA are a few ...
Ameen Izhac's user avatar
0 votes
0 answers
10 views

a few records for training, and one record after training

I try to make some credit score task. I stuck in conceputal problem. There is: train_data (62 columns, 10339239 rows, 1250000 unique ID values [0 - 1249999]([min-max] ID values)) test_data (62 ...
Excentricitet's user avatar
0 votes
0 answers
22 views

Extremely Imbalanced and Gapped Dataset in Regression Problem

Currently I am working with a biological dataset with a range of 0-to-1 to do a multi-task regression with Deep Learning. However, this dataset has an empty gap in the range 0 to 0.2 (however there ...
Abdullah Faqih's user avatar
0 votes
0 answers
16 views

Is balancing imbalanced validation set for retraining model after hyperparameter tuning required?

The following are basic steps to modelling, but would like to ask in the case of imbalanced data, is balancing of train dataset required when retraining model on train + validation set after ...
curious-24-7's user avatar
0 votes
0 answers
9 views

What does a correlation between predictions of training data and predictions of validation data tell me?

In my Machine Learning intro course I made several scatter plots of predictions for the california housing data set. Here is the most complete of them (created by a pipe using a sklearn StandardScaler ...
S3k's user avatar
  • 1
1 vote
1 answer
9 views

Visualizing clusters of 3D images / volumes

I have a dataset of 3D (medical) images / volumes, think CT/MRI recordings. In other words, each of my samples consists of a 3D array of grayscale values, typically in the range of e.g. 1000x1000x500. ...
Eike P.'s user avatar
  • 111
0 votes
0 answers
4 views

Is GroupKFold needed if some samples have some of their feature values equal?

I am given a dataset $D$ of 10k enzyme-substrate complexes having a lock-key relationship, with each sample (complex) being characterized by enzyme features $x_e$ and substrate features $x_s$. That is,...
ado sar's user avatar
  • 181
1 vote
1 answer
25 views

Feature selection in binary classification

I have a dataset with two classes and am interested in learning which features are 'important' for predicting the class. There are a lot of features available and I want to find subset(s) that lead to ...
Shawn's user avatar
  • 33
0 votes
0 answers
9 views

Are there any Python (sklearn or other) ML libraries that can handle missing data?

I am running a model using a sparse matrix for the X. It can not be imputed. So are there any libraries who modify the main ML models to deal with this?
J_Bake's user avatar
  • 1
0 votes
0 answers
4 views

Trying to understand Ch.2.1 "2.1. Monads and their algebras" of "Categorical Deep Learning: An Algebraic Theory of Architectures"

I am trying to read article "Categorical Deep Learning: An Algebraic Theory of Architectures" https://arxiv.org/abs/2402.15332 and I am stuck with the Chapter 2.1 "2.1. Monads and their ...
TomR's user avatar
  • 141
1 vote
0 answers
6 views

Checking MixedModel convergence in Julia

I'm trying to teach myself how to use Julia as an alternative to R, and was running into some trouble with checking convergence of a MixedModel object. In R, if I use the following code ...
Vincent's user avatar
  • 11
0 votes
0 answers
13 views

One shot object detection

I want to detect objects from a video. I can only have one example image and based on this, I want to detect whether this image is present in the video or not. What could be the best-suited approach?
Rajeev Radhakrishnan's user avatar
1 vote
0 answers
13 views

Which model and classification algorithms should I use for deobfuscating/mapping symbols in source code?

I have a source code for an application that recently started undergoing obfuscation. Each new version alters the names of symbols, shuffles the order of classes, and employs other strategies. However,...
emi's user avatar
  • 111
0 votes
0 answers
5 views

Text progress bar for SQL query embedded in R?

I am using the RODBC package in R to query Teradata SQL. I would like to know if it is possible to create a text progress bar that tracks the progress of the SQL query.
feonyte's user avatar
0 votes
0 answers
8 views

keyword extraction and link to multiple sources

I have to perform a tech watch over a very rich subject which spans from software to deep physical issues and also, from business news to scientific articles. Usually, I would have first retrieved the ...
deb2014's user avatar
0 votes
0 answers
7 views

Efficient anomaly detection in unordered market data - is it possible?

I'm a little bit stuck on how to efficiently model anomaly detection for the following problem, probably because of my lack of experience with time series modelling: I retrieve market data sorted by ...
Skyence's user avatar
-1 votes
1 answer
13 views

Azure OpenAI API with LangChain issue

We(my company) have Azure OpenAI API for all our LLM projects. We are working on getting to know how to use this Azure OpenAI API using LangChain as we have been suggested this is the best framework ...
Arun Singh G's user avatar
-1 votes
0 answers
12 views

!pip install guardrails [closed]

error: subprocess-exited-with-error python setup.py egg_info did not run successfully. exit code: 1 [8 lines of output] Traceback (most recent call last): File "", line 2, in File "&...
PRATEEK GUPTA's user avatar
1 vote
0 answers
7 views

Annotation with nested and dynamic forms?

I am trying to label data where a piece of text can have any number of top level labels, and for each of those, it can have any number of child labels. For example, the text might be a description of ...
Dima's user avatar
  • 111
0 votes
0 answers
15 views

Clean CSV file using ML

I would like to build a Neural Network to clean a CSV file that contains wrong delimeters. My approach is the following: Identify rows that have more columns than the header. Define these rows as &...
DoktorMLNoob's user avatar
1 vote
0 answers
15 views

Best way to achieve High throughput with ChatGPT (and LLMs in general) APIs

I was wondering what the best way is to perform API calls to ChatGPT, and in general, with LLMs API providers, to achieve the highest throughput in a batch inference context. Suppose you have 1000 ...
Raffaele 's user avatar

15 30 50 per page
1
2 3 4 5
738