Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

2,424 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
9
votes
0answers
140 views

Is Minimax Linkage a Lance-Williams hierarchical clustering?

I found the following article on "Hierarchical Clustering With Prototypes via Minimax Linkage". It is stated in Property 6 that Minimax linkage cannot be written using Lance–Williams updates. A ...
8
votes
0answers
259 views

Finding linear transformation under which distance matrices are similar

I have n sets of vectors, where each set S_i contains k vectors in ...
6
votes
1answer
39 views

How to add data points to a trained KNN

I have a trained KNN, I created with https://github.com/kevinzakka/blog-code/blob/master/knn/knn.py I want to add more data points to the KNN but I am on a raspberry pi so limited by RAM and ...
6
votes
1answer
682 views

Input for LSTM for financial time series directional prediction

I'm working on using an LSTM to predict the direction of the market for the next day. My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
5
votes
1answer
54 views

Predict a sequence given many sequences

I'm trying to find an algorithm that would fit this use case: My data: a bunch of fixed-size int arrays, e.g. [0,2,3,4,5] [1,2,3,1,5] [4,1,2,4,5] ... Input: an ...
5
votes
1answer
353 views

Find matching text from a text column

This is my first time to use Data Analytics tool to figure out a solution to a problem. I have a table with following columns ...
5
votes
1answer
164 views

How to compute document similarities in case of source codes?

I try to detect the probability of common authorship (person, company) of different kind of source code texts (webpages, program codes). My first idea is to apply the usual NLP tools like any token ...
5
votes
1answer
125 views

Dealing with population instability

I am working on using machine learning to correctly predict a binary classification using an input dataset that I receive about once a month. The idea is that I train, test and validate the ...
5
votes
0answers
1k views

AdaBoost implementation and tuning for high dimensional feature space in R

I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
5
votes
1answer
568 views

Checkers playing Neural Network evolved with Genetic Algorithm becomes too sensitive to input data changes

I recently embarked on a very ambitious project and I have to say it has turned out a lot better than I expected, I succeeded in coding from scratch a neural network that plays checkers at a very ...
5
votes
1answer
1k views

Netflow anomaly detection python packages

Is anyone aware of any open source / python packages for Netflow Anomaly detection ? I found some on github but anyone who has more experience with it. please advise.
5
votes
1answer
128 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
0answers
451 views

differences between LSQR and FTRL when working with very sparse data

I have a 2M instances dataset with millions of very very sparse dummy variables created using the hashing trick = ...
4
votes
1answer
55 views

Data Science Pipelines vs Common CD/CL

What is the advantage of Data Science Specific CI/CD (kubeflow, Algo, TFX, mlflow, sagemaker pipelines) vs the already baked flavors that are more generic: Jenkins, Bamboo, Airflow, Google Cloud Build,...
4
votes
1answer
36 views

Training Machine Learning Model - Neural Network - Islands Problem

I was working on the following leetcode problem: Given a 2d grid map of '1's (land) and '0's (water), count the number of islands. An island is surrounded by water and is formed by connecting ...
4
votes
1answer
40 views

weightage attributes

If I have a dataset with binary classification and has many attributes with value of (0 or 1) means the occurrence of attribute is represented by 1 and absence is represented by 0, can I add weight ...
4
votes
1answer
91 views

How do I interpret loss in a neural network?

I am studying how to evaluate the performances of a convolutional neural network, and in particular I have seen that we have to look both at accuracy and loss. I don't understand why do we have to ...
4
votes
1answer
63 views

The output of Model from Decision Tree and Random Forests are different?

I have been made a model using both Decision Tree and Random Forest. But, when I tried to test the model on the same DataFrame the output is different. How is this possible? The data file from my ...
4
votes
1answer
88 views

Evaluating machine learning explainers?

I'm working on a project where multiple machine learning explainers (LIME and SHAP, potentially more coming) are applied to pre-trained models (neural networks) to help explain the predictions of ...
4
votes
2answers
38 views

How to model a binary classification problem in a evolving environment

Suppose you have a problem with two classes YES or NO. While the YES class is fixed, in the sense that the observations do not evolve, the observations of class NO may evolve during time and may ...
4
votes
2answers
90 views

Is there a rule of thumb when designing neural network in deep reinforcement learning?

In deep learning, we can assess model's performance with loss function value and improve model's performance with K-fold cross-validation and so on. But how can we design and tune neural network used ...
4
votes
0answers
50 views

Training value neural network AlphaGo style

I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
4
votes
2answers
361 views

Image recognition of selfie images

I developed an Android app that lets anyone upload pictures of encyclopedic things (bridges, museums, dishes, landscapes, paintings, etc) to Wikimedia Commons. Unfortunately, 5% of the users find it ...
4
votes
1answer
149 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
4
votes
1answer
68 views

Predicting change of shapes/coordinates

I'm trying to find a way to predict/calculate how a shape (e.g. outline of a glacier) will change in the future—based on its history (previous shape) and additional factors (e.g. Δtemperature). In my ...
4
votes
2answers
60 views

Classifier that optimizes performance on only a subset of the data?

I'm working on machine learning problem where I'm only interested in getting high accuracy within a narrow band of my predicted likelihoods. Specifically, I want an algorithm that will score very ...
4
votes
0answers
368 views

Hyperparameter tuning in multiclass classification problem: which scoring metric?

I'm working with an imbalanced multi-class dataset. I try to tune the parameters of a DecisionTreeClassifier, ...
4
votes
2answers
494 views

How to deal with position bias in search?

In search, position of the search result affects the click-through rate a great deal. How do people usually deal with this ? In practice how to remove such bias to create unbiased training data for ...
4
votes
2answers
11k views

Multivariate Time-Series Clustering

I have a streaming data along with timestamp dataset that looks like this: 1.png Timestamp can be inclusive of "seconds" too, but the data may or may not change every second. it depends on the ...
4
votes
0answers
645 views

AUC computation on multilabel classification

I'm using Tensorflow for an auto-tagging task on audio clips. The problem is actually a multilabel classification problem meaning that each clip can have multiple tags at the same time. Regarding ...
4
votes
2answers
908 views

Combining Machine Learning classifier with NLTK Vader for Sentiment Analysis

As a part of my university project, I am researching/developing a sentiment analysis model wherein I am trying to combine NLTK Vader (SentimentIntensityAnalyzer) results with a Machine Learning ...
4
votes
2answers
206 views

Support Vector Regression trained with data sets

I am now searching for a long time on the internet and on papers for an answers of simple questions. Am I able to train a Support Vector Regression algorithm with different data sets? If yes, how is ...
4
votes
1answer
1k views

Convolutional Network for Text Classification

I am trying to train a convolutional neural network with Keras at recognizing tags for Stack Exchange questions about cooking. The i-th question element of my data-set is like this: ...
4
votes
1answer
613 views

Time Series pattern recognition and classification problem

I have some labeled sensor data. Now, I would like to know how to extract features from time series using DFT, DWT, and HAAR transforms. I know that the transformations above transform a signal to ...
4
votes
1answer
428 views

How to implement patternet in python as it is in matlab?

NOTE: This question was first posted in cross-validated website but I was instructed to move it off that website as it was not a good fit. I am new in implementation of machine learning, neural ...
4
votes
0answers
254 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
0answers
394 views

Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? Your usual classification/regression setup Lets say the data is a classic regression/classification problem: several ...
4
votes
1answer
157 views

How to automatically verify official documents?

I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
3
votes
1answer
39 views

Practical limitations of machine learning

Working on some applied machine learning problems, I've started to encouter some practical difficulties. Those difficulties relate to - but are not limited to - : convergence of the learning process, ...
3
votes
0answers
20 views

Possible flaw in the MDS method for dimensionality reduction

The MDS (multidimensional scaling) method is used to solve the problem of dimensionality reduction. Basically, it does the following: given $n$ points $x_1,\cdots,x_n\in\mathbb R^d$, try to find a ...
3
votes
0answers
94 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
3
votes
1answer
61 views

Segmentation Network produces noisy output

I've implemented a SegNet and SegNet ReLU variant in PyTorch. I'm using it as a proof-of-concept for now, but what really bothers me is the noise produced by the network. With ADAM I seem to get ...
3
votes
1answer
76 views

tensorflow pseudo inverse doesn't work for complex matrices!

The Tensorflow documentation here says that: tf.linalg.pinv is ''analogous to numpy.linalg.pinv. It differs only in default value of rcond''. However, tf.linalg.pinv requires the matrix to ...
3
votes
1answer
33 views

Data prediction using scikit-learn and a list

I have a group of lists detailing temperatures over differing amounts of time. My goal is to use machine learning to identify periods in which a machine is turned on and off, where turning on the ...
3
votes
0answers
185 views

How to do feature selection for clustering and implement it in python?

I am trying to implement k-means clustering on 60-70 features and I came across a post for feature selection technique on quora by Julian Ramos, but I fail to understand few steps mentioned. I am ...
3
votes
1answer
60 views

What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm? I understand that EDA helps us generate good and helpful insights ...
3
votes
1answer
17 views

Identify members who are likely to switch where they receive drug administration

I have access to medical claim data from a large health insurance company. As some of you may know there is a large delta between the price of drug X depending on where it is administered. My ...
3
votes
1answer
30 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
3
votes
0answers
16 views

High dimensional data stream summarization and processing

Can anyone recommend a method for summarizing and processing high dimensional data streams efficiently and effectively for anomaly detection? In fact, I investigated the different methods for data ...
3
votes
2answers
57 views

How best to show the best model over multiple labels?

I have 4 models I trained and I want to display their prediction success over 45 different labels I tested them on. I get a very messy plot when I naively try to place them one on top of the other. ...

1
2 3 4 5
49