All Questions

6,141 questions with no answers
Filter by
Sorted by
Tagged with
86 views

Meta-analysis of public 16S data

I am trying to start a meta-analysis for which I want to extract some 16S-based information from public databases. Moreover, I want to relate this information with any metadata found in the associated ...
399 views

Illustrating the dimensionality reduction done by a classification or regression model

Tl;DR: You can predict something, but how do you explain the prediction? Your usual classification/regression setup Lets say the data is a classic regression/classification problem: several ...
263 views

3D map using leaflet

I'm trying to create 3D bars on this map. Can anyone please advise if this is possible, and how? http://leafletjs.com/examples/choropleth.html My data: UFO sightings in the USA (location wise). ...
30 views

What is the difference between trax vs tensorflow?

What is the main difference between trax vs tensorflow? both of them deep learning library and implemented by google team. https://github.com/google/trax https://github.com/tensorflow/tensorflow
37 views

How to properly do feature selection when comparing different models?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
48 views

Word2Vec: Why do some dimensions of an embedding have an interpretation, and why does addition/subtraction of embedding vectors work?

I'm reading about Word2Vec from this source: http://jalammar.github.io/illustrated-word2vec/. Below is the heatmap of the embeddings for various words. In the source, it's claimed that we can get an ...
36 views

What are the 'protos' in TF Object Detection?

I am struggling to understand what are the 'protos' in TF Object Detection? Why do we need them here? Also, while setting up the TF API we need to download and compile protocol buffers. There is also ...
20 views

How to train YOLOV4?

I am going to write yolov4 real-time object detection, and I have to do it for car then vehicle plate number, but it does not have to find plate number if there is no car, first car then number on car,...
14 views

Non semi positive definite kernel matrix

What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite? This is my flow of thought: In kernel methods $w.x$ is ...
74 views

Why the sigmoid activation function results in sub-optimal gradient descent?

I need some help understanding the second shortcoming of the sigmoid activation function as described in this video from Stanford. She says that because the output of sigmoid is always positive, that ...
67 views

Find missing object(s) in image with a priori knowledge about the missing object(s) (w.r.t base image)

Problem Statement: I am working on developing a method, or borrow/modify/combine existing ones, where given an golden image (reference or base with all expected objects to be present), it is able to ...
42 views

Detecting pushups based on pose data

I've been playing with Google's MLKit, and decided to detect push ups. As a quick test, I took the position of the left shoulder, and plotted the Y Axis. Here's how a variety of trials look: Five ...
18 views

Force Matching in Coarse Grained Molecular Dynamics with Jax - Forces do not match when neglecting energy loss

I am currently exploring force matching approaches for molecular dynamic simulations. As I am still in an exploration state, I'd tried investigated Force Matching Neural Network Colab Notebook ...
46 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
32 views

Train NN for balanced accuracy

I'm trying to get a NN (a CNN, to be precise) to predict labels on hugely imbalanced data, i.e. some classes are much bigger than others. I want to optimize the balanced accuracy. So I give the class ...
29 views

A/B test results contradictory with offline machine learning model performance

This seems to be a common problem when bringing machine learning models to production. Let's say we have an optimized machine learning model which gives decent performance metric in the unseen testing ...
75 views

Best way to remove useless features when there are more than 100,000 features?

I am in a situation where i have more than 100,000 features, and i need to select the top features to give them to my final neural network model. So far i have been using RandomForestClassifier in ...
29 views

What is the concept behind the categorical-encoding used in the CatBoost benchmark problems?

I'm working through CatBoost quality benchmark problems (here). I'm particularly intrigued by the methodology adopted to convert categorical features to numerical values as described in the ...
193 views

How to choose threshold for gensim Phrases when generating bigrams?

I'm generating bigrams with from gensim.models.phrases, which I'll use downstream with TF-IDF and/or gensim.LDA ...
54 views

Help interpreting GAN output, and how to fix it?

After a few tries, I had trained a GAN to produce semi-sensible output. In this model, it almost instantly found a solution and got stuck there. The loss for both the discriminator and generator were ...
75 views

What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
59 views

Difference Bagging and Bootstrap aggregating

Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron. Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from ...
63 views

How to find lagged cross correlation between time series?

I have 2 time series, $X$ and $Y$, and I'm trying to find the best lag range that correlates $X$ to $Y$ (find the amount(s) of lag of $X$ that best correlate to the target variable $Y$). For instance, ...
16 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
46 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
46 views

Bag of words: Prediction on new (out-of-sample) data

I'm working with a bag of words in R: library(tm) corpus = VCorpus(textsource) dtm = DocumentTermMatrix(corpus) dtm = as.matrix(dtm) I use the matrix ...
28 views

Dataset availability for automatic text summarization

I'm working on an automatic text summarization NLP problem and looking for a dataset with USA legal case reports similar to the Australian legal case reports dataset in UCI repository. Can you please ...
39 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
191 views

Converting pandas dataframe to scipy sparse arrays

Converting pandas data frame with mixed column types -- numerical, ordinal as well as categorical -- to Scipy sparse arrays is a central problem in machine learning. Now, if my pandas' data frame ...
29 views

What's the definition of retraining？

In transfer learning, we always use new data to retrain the pre-trained model. But, what is the specific and official definition of retraining? Or what papers mentioned this definition, in transfer ...
73 views

Graph Embedding Vs Graph Convolution Network

I'm new in Graph-Embedding and GCN(Graph/Geometric Convolution Network). I'm confused and not very much sure about "How training works in GCN"? As per my understanding, GCN training data will be ...
20 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
209 views

Balancing the dataset using imblearn undersampling, oversampling and combine?

I have the imbalanced dataset data['Class'].value_counts() Out[22]: 0 137757 1 4905 Name: Class, dtype: int64 ...
73 views

What happens with activations?

I am playing with convolution network, assembling something between AlexNet and ResNet. Not very deep, about 10 conv. layers including 2 through residual connection, and 3 fully-connected layes at the ...
21 views

Possible flaw in the MDS method for dimensionality reduction

The MDS (multidimensional scaling) method is used to solve the problem of dimensionality reduction. Basically, it does the following: given $n$ points $x_1,\cdots,x_n\in\mathbb R^d$, try to find a ...
154 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
25 views

How to visualize grouped clusters in a graph

I was trying to use Gephi, which seems to have a 'group' feature in some of the WIKI explanations, but actually does not appear in my GUI. There is a tool that creates groups from partitions, but it ...
23 views

Gaussian process regressor returns almost identical std for all datapoints

I am using a Gaussian process regressor as the regressor for active learning and I use its standard deviation to choose the next training inctance (the one with the highest std is chosen). However, ...
26 views

preprocessing time sequence

I have a long list of event (400 unique events, sequence ~10M long). I want to train an RNN to predict next event. The preprocessing steps i took are: (1) turning to OneHotEncoding using pandas: <...
72 views

What are the data preprocessing steps required before running K-Modes?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
300 views

Plotting Gradient Descent in 3d - Contour Plots

I have generated 3 parameters along with the cost function. I have the $\theta$ lists and the cost list of 100 values from the 100 iterations. I would like to plot the last 2 parameters against cost ...
23 views

Visualizing the difference of a set of strings

I have a distance metric on a collection of strings on the order of tens of thousands. What would be an intuitive way to summarize how 'different' these strings are or when they overlap? My goal is, ...
55 views

SuperLearner Cross validation with iid time series

I created a number of ML models in R and I aim at combining them to form an ensemble. I learned about SuperLearner library which cross validates many models and returns the weight to each model in ...
1k views

How to apply a groupby rolling function to create multiple columns in the dataframe

I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo and the link to the data is here and the example here. Some Sample of data ...
31 views

If I use Gibbs sampling with a Bayesian model, what do I have to check is memoryless?

Right now I am trying to better understand how Bayesian modeling works with just the basics. I found through reading tutorials that some very basic Bayesian models like Bayesian Hierarchical Modeling ...
50 views

How can I improve the accuracy of my model? (Cab Cancellation Prediction)

I am trying to predict based on several parameters like trip type, car type, source of booking, start time, lead time (start- book) and a few other params whether or not a customer will cancel. From ...
55 views

How to encode multiple inputs and multiple outputs

I have a piece of math more complicated than I can understand at the moment, a dynamic graph visualization. It uses a physics metaphor of springs and magnets, where the vertices act as magnets ...
43 views

Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best hyperparameters ...
50 views

Terrible Tensorflow performance on powerful production server

Background I am building a simple face detection API with Python, Flask and the MTCNN face detector. My problem is that the model and API are running really quickly (batch of 100 images takes 0.5-0....
96 views

How does one derive the modified tanh activation proposed by LeCun?

In "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), LeCun and others propose a modified tanh activation function of the form: $$f(x) = 1.7159 * tanh(\frac{2}{3}*x)$$ ...

15 30 50 per page