All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Is it possible to predict based on arbitrary attributes?

I have a job/task runner system that takes a task description as input from a queue and delegates it to a machine on which to run. Based on the task type and other properties, some which are task-...
1
vote
1answer
841 views

How to perform feature selection with Categorical Variables and Continuous Target, provided that data is not normally distributed?

Basically I am trying a use Multi Linear Regression Model to predict the salaries of employees. I have a total of 88 dependent feature from which 19 are categorical and the rest are continuous. I have ...
4
votes
2answers
64 views

What are the reasons for drawing initial neural network weights from the Gaussian distribution?

Are there theoretical or empirical reasons for drawing initial weights of a multilayer perceptron from a Gaussian rather than from, say, a Cauchy distribution?
2
votes
2answers
60 views

Backpropagation and gradient descent

I just want to clear one doubt - we use gradient descent to optimize the weights and biases of the neural network, and we use backpropagation for the step that requires calculating partial derivatives ...
0
votes
0answers
5 views

Do generative model produce varying outputs for same input

I am new to data sciences. I believe the generative model generate responses on-the-fly for a valid user input. Is it correct to assume that such models would generate different responses for the same ...
0
votes
0answers
4 views

Recommendation System with ALS Implicit

I created a model for Recommending top 10 items to users similar to the approach used here https://github.com/benfred/implicit/blob/master/examples/lastfm.py I wanted to evaluate the model using NDCG ...
1
vote
1answer
76 views

Why do histogram bars vanish when we keep the bins value high in matplotlib?

Also, the histogram bar widths are different on certain values of bin. How to keep the bar widths uniform? I have tried using the rwidth but that dos not solve my problem. Data: ...
1
vote
2answers
74 views

Clustering with sets as values

I have gathered a large amount of qualitative data and am now looking to cluster it so as to make sense of it. For this, I am using Biolab's Orange. In my data, specific values may co-occur in a ...
8
votes
4answers
2k views

How to deal with spelling errors NLP

I have some data where the main column is the description of one product. The main task is to extract the name of some product from this column, where it sometimes is spelled wrong and amended in ...
1
vote
1answer
85 views

Multi-class classification configuration

1) What are the appropriate activation and loss functions for multi-class classification problem? Is it so that: Up to 2 classes $\rightarrow$ Binary classification $\rightarrow$ Activation: Sigmoid ...
2
votes
0answers
9 views

Running out of memory when calculating loss using SigmoidFocalCrossEntropy

Code Versions: python == 3.8 tensorflow == 2.2.0 tensorflow-addons == 0.11.2 Recently I've been using tensorflow addon's focal loss function for one of my models. ...
1
vote
0answers
7 views

How to mantain the nested structure of a tf.dataset after applying map?

I'm creating a tf.dataset object containing 2 images as inputs and a mask as target. All of them are 3D in grayscale. After applying a custom map, the shape of the object changes from ...
2
votes
0answers
8 views

Clustering and getting threshold to classify data points

I have real world data set (120 data points) about enterprises, containing 4 features. I would like to put these enterprises in exactly 4 categories based on the values of these specific features (an ...
1
vote
0answers
17 views

Same model/same data different results in Keras/TF

So first to mention I used for all models the identical dataset (just in different shapes). I started with a binary classification based on text following the tutorial from https://keras.io/examples/...
1
vote
1answer
30 views

How does Wav2Vec 2.0 feed output from Convolutional Feature Encoder as input to the Transformer Context Network

I was reading the Wav2Vec 2.0 paper and trying to understand the model architecture, but I have trouble understanding how audio raw inputs of variable lengths can be fed through the model, especially ...
3
votes
1answer
55 views

Tool for annotation of images for semantic segmentation

I have been searching around for a software tool, that I can use for annotating images. More specifically I want to do annotation to be used for semantic segmentation, meaning I want to create masks. ...
1
vote
0answers
5 views

creating dataset/storing vectors from c++ to python

I am new to the data science field. I need to create dataset which consists of n inputs (vectors of size 5) and n outputs (vectors of size > 10,000). These vectors are generated using c++ code. ...
1
vote
1answer
9k views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
2
votes
1answer
271 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
2
votes
1answer
21 views

Suitable modelling approach to incorporate different signals

I am working with data in the form of content that is being reviewed to decide if it violates the policy standards or not. In the review process there are several methods to increase the chances of ...
4
votes
1answer
47 views

How does a random forest algorithm deal with a few irrelevant input variables

I have a list of variables from which I would like to train a Random Forest Algorithm. I suspect that a few of my input variables, which have noisy distributions, won't be able to predict much. Can I ...
2
votes
1answer
8 views

How do you choose an appropriate $k$ to achieve $k$-anonymity for data?

How do you choose an appropriate $k$ to achieve $k$-anonymity for a data? What methods exist that are agnostic to the business context for the problem?
0
votes
2answers
107 views

i'm using GridSearchCV to find parameter C for SVC() classifier present in sklearn.svm . I'm not getting the optimal result desired

this is a screenshot of my code. i used abc.best_estimator_ (my GridSearchCV model) to find out best results. As you can see grid has values of C=1 and C=100 along with other values. abc....
2
votes
1answer
78 views
+50

Feature importance with Text features

I would like to determine features importance in several models: support vector machine logistic regression Naive Bayes random forest I read that I will need an agnostic model, so I have thought to ...
2
votes
1answer
134 views

Feature Selection and Outlier Detection

How does feature selection impact outlier detection and also, removing outliers impact feature selection? It could be a basic question. However, just to know the boundaries, I asked. Thanks in advance....
1
vote
1answer
11 views

How Sklearn-crfsuit interpret text features

As we see here, to build an NER model we can pass text features (parts of the word, pos tag, structure of the word etc.) to Sklearn-CRF. I was wondering how does this package convert the text features ...
1
vote
2answers
19 views

How does checkpoint works to find the very best model during training?

When filepath="weights-improvement-{epoch:02d}-{val_acc:.2f}.hdf5" I get key error val_acc (I use tensorflow 1.14.0). When ...
1
vote
0answers
16 views

Edge-based Graph embedding for a street network

I've recently been reading a lot about Graph Embeddings, but most of the implementations focus on node-based embeddings. I'm trying to map a street network into a graph embedding where edges are ...
2
votes
1answer
100 views

Multiple XGBoost models or just 1 for a cetain type of category?

I am building a model to predict, say house prices. Within my data I have sales and rentals. The Y variable is the price of either the sales or rentals. I also have ...
1
vote
0answers
10 views

Label spreading for classification/clustering problems

I have a question regarding label propagation and label spreading semi-supervised algorithms. I am working on building a look-alike model to identify marketing personas. Using supervised learning ...
3
votes
1answer
332 views

Advice on dealing with very large datasets - HDF5, Python

I've recently started working on an application for visualization of really big datasets. While reading online it became apparent that most people use HDF5 for storing big, multi-dimensional datasets ...
0
votes
0answers
8 views

Cluster words into groups of similar meaning (synonyms)

How can words be clustered into groups of similar meaning (synonyms)? I started with pre-trained word embeddings (e.g., Google News), which is great, but not perfect - a limitation arises because the ...
0
votes
0answers
8 views

Time series forecasting for vibration prediction on Industrial machine production?

I'm working on a machine learning project related to an industrial machine. The goal of the project is to build a model that would be able to predict the vibration of the machine while it's in ...
1
vote
0answers
65 views

Tensor Backpropagation

I tried to make simple neural network layers as the following, including forward and backward propagation. Here is my reference. Firstly I assume an one layer FC: $Y = X \cdot W + B$ X is input, which ...
1
vote
2answers
26 views

Independence of Features assumption in Naive Bayes

How do we know if your features in my dataset are independent before applying Naive Bayes? Basically I want to know is it possible for us to get an idea before training our model if Naive Bayes will ...
0
votes
0answers
4 views

is it better to use embedding with attributes in collaborative filtering or content-based approach?

I have a dataset with different text documents, a set of users who each read a different document, some historical info such as their reading speed, and other attributes related to the texts and users....
0
votes
1answer
16 views

dataframe groupby time period [closed]

i need to count positions over a time period, grouped by date. One position can last more than one day: ...
2
votes
1answer
320 views

Sequence models word2vec

I am working a data-set with more than 100,000 records. This is how the data looks like: ...
0
votes
1answer
41 views

Can distortion be derived from inertia rather than recalculating it from scratch in case of kmeans?

I got this definitional difference between distortion and inertia from here: Two values are of importance here — distortion and inertia. Distortion is the average of the euclidean squared distance ...
0
votes
0answers
11 views

Linear regression on sparse matrix?

I have a matrix with sparse data. A small extract from it is seen below. The columns represent years and the rows represent different race tracks. The feature values are velocities on that specific ...
2
votes
1answer
88 views

How to generalize comments using NLP

I have list of log comments in CSV file. I want to cluster those log comments using K-Means and after that I want convert each cluster comments into general form. for eg. I have bunch of comments in ...
2
votes
1answer
169 views

How can I imporve accuracy for text classification and mapping using SVM?

I am working on a problem where I need to predict the text corresponding to another text in my training data file. For example: if I have value like the software in one of my columns and another ...
19
votes
2answers
11k views

What is the difference between semantic segmentation, object detection and instance segmentation?

I'm fairly new at computer vision and I've read an explanation at a medium post, however it still isn't clear for me how they truly differ.
0
votes
0answers
7 views

Big array and MemoryError: Unable to allocate memory (in Python)

I am trying to create a predictive model using linear regression with a dataset that has 157673 entries. The data (in a csv file) is in such format: ...
0
votes
0answers
8 views

Bag Level labels from instance labels

I have a 4 class classification problem using speech data. The speech signals are of variable length ranging from 1 sec to 8 sec. I am making chunks of 3 secs by chopping the longer utterances and ...
1
vote
0answers
15 views

Origin of the Boolean Model of Information Retrieval

Simple question, but I can't really find the answer to that: Who "invented" Boolean Retrieval? Of course, I assume that the concept grew over time, but is there a paper or publication that ...
0
votes
1answer
26 views

Model with 2 datasets: combine time series data and statistics

I am new to data science modelling so apologies if using wrong terminology in advance. I have a standard time series dataset of historical prices which is used to train/test a simple Random Forest ...
3
votes
3answers
65 views

Is it reliable to use TensorFlow (ML in general) to classify baggage bag tags based on the presence of a green stripe?

The images are identical except for the presence of the stripe on the side. I am trying to use a classify the images into 2 classes: greenStripe, noGreenStripe. I tried to use tensorflow retrain with ...
0
votes
1answer
16 views

How the Support Vector Machine will perform if the bias b = 0 in the equation of hyperplane?

We have a soft margin linear SVM and the equation is as follows : How the SVM will perform if b = 0, means the hyperplane is passing through the origin ?

15 30 50 per page
1
2 3 4 5
577