# Questions tagged [theory]

Theory relates to theoretical questions regarding data science and machine learning.

61 questions
Filter by
Sorted by
Tagged with
14 views

### Reinforcement Learning: Formally, does an exponentially decaying epsilon satisfy GLIE?

I am aware that exponentially decaying exploration constants (epsilon) are used practically. Formally, though, do they satisfy the GLIE condition? Specifically, relating this to the Borel-Cantelli ...
26 views

### Check the validity of the distribution in the proof of No-Free-Lunch Theorem

I'm reading the proof of No-Free-Lunch Theorem (quoted at the end of this question) in Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, p.37, the author wrote: ...
13 views

### Why do we use Gradient Descent for Norm Batch since the function has only 2 parameters?

So here is the formula for Z sedile ( gamma * Z + Beta). Since it's a function of 2 parameters why there is still stated in courses that we need to compute gradient descent to find the slope ? Since ...
20 views

### What is an appropriate definition for the action space in my RL problem?

I am thinking about how to define the action space in my RL problem. The goal is to try many different RL algorithms (value-, policy-based, and hybrids), in order to compare their performance, across ...
1 vote
59 views

### Where does AI/ML theories come into play when nowadays the AI libraries already so powerful?

I've read up posts so-call for beginning ML, claiming you need linear algebra, statistics, complicated optimization to just getting start in ML/AI. And on top of these, there comes the ML/AI ...
1 vote
26 views

### Are there any established approaches for dealing with a Degenerate Feedback Loop?

Scenario: I develop a model which forecasts the likely sales success of a particular enquiry based on outcomes of past similar enquiries. I then assign this likelihood score to new enquiries when they ...
173 views

### How to scale a subset of data with respect to the entire dataset

I am developing a financial time-series prediction model using sklearn using StandardScaler for scaling purposes. I train a model, and then use the model regularly ...
55 views

341 views

### Why does Jim Gray call "data-driven science" a new paradigm?

Wikipedia it says about data science: Turing award winner Jim Gray imagined data science as a "fourth paradigm" of science (empirical, theoretical, computational and now data-driven) and ...
1 vote
167 views

### Can a neural network be of variable depth?

It is very common for neural networks to be asymmetric about the x axis, that is, to have many more nuerons in the first few layers than in the last few layers. Common example: But can neural ...
1 vote
4k views

### What is an object detection problem with only one class called?

Object detection is defined as the problem in which a model needs to figure out the bounding boxes and the class for each object. A lot of ML solutions for object detection base around having "two ...
56 views

I was reading this nice tutorial about Pytorch's basics: https://pytorch.org/tutorials/beginner/pytorch_with_examples.html In the first example (pure Numpy), the author starts the backward phase by ...
1 vote
101 views

### What is the best approach for send time optimization? [closed]

I could no find a lot information about how the companies doabout send time optimization, either for push notifications or email campaigs. having historical data about clicks and sends what would be ...
1 vote
568 views

### Is the search space of Hyperparameters Continuous or Discrete?

I am looking into hyper-parameter tunning and was curious about whether the search space is considered continuous or discrete? My understanding of both those cases: 1. Continuous would make it 'easier'...
34 views

### What is the name of this statistical interaction?

What is the name of the following statistical / informational interaction: given A, I know exactly what B is. given B, I know to some extent what A is. I'm not looking for a probability but rather ...
35 views

### Structuring experiment/training data with months in mind

We're using a whole year's data to predict a certain target variable.The model works like data - OneHot encoding the categorical variables - MinMaxScaler - PCA (to choose a subset of 2000 components ...
104 views

### Encoding correlation

I have rather theory-based question as I'm not that experienced in encoders, embeddings etc. Scientifically I'm mostly oriented around novel evolutionary model-based methods. Let's assume we have ...
2k views

### Explanation of inductive bias of Candidate Elimination Algorithm

The definition of inductive bias says that The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs given inputs ...
120 views

### Understanding LSTM Keras implementation

So I understand what LSTM units are. But I have trouble understanding the implementation / function in Keras framework. Let's say, I add a layer ...
62 views

### Popular classification algorithms over time

In the book "Deep Learning with Python" by Francois Chollet (2018), in section 1.2.4 one can find: Decisions trees learned from data began to receive significant research interest in the ...
40 views

### Is it possible for a neural net to score as high as a different form of supervised learning?

I've been working with the Adult Census Income dataset from UCI http://archive.ics.uci.edu/ml/datasets/adult I've created two different models, one using a gradient boosted classifier with sklearn, ... 145 views

84 views

### A question on realizable sample complexity

I came across the following exercise, and I just can't seem to crack it: Let $l$ be some loss function such that $l \leq 1$. Let $H$ be some hypothesis class, and let $A$ be a learning algorithm. ...
1 vote
58 views

### What could cause validation set to consistently perform better than training?

I'm training a neural network with a very small dataset just to get things set up, before training on a much larger set. (I only have about 500 data points available to me at this time, with more ...
115 views

### Decentralized machine learning

Currently, to train a model, you need to collect a huge blob of data. Are there feasible concepts of decentralized machine learning? Like, feed the model somehow from isolated data sources, or merge ...
66 views

### What does it mean when someone says "Most of the data science algorithms are optimization problems"

I was trying to understand the Gradient Descent algorithm from this article and the author says Most of the data science algorithms are optimization problems I come from software engineering ...
110 views

### What should be the requirement for training data in order to obtain a good regression model using neural network?

I have made a neural network regression model using the theory for the first time and would like to clarify some basic doubts, whose concrete answers I couldn't find yet. Data:- I have 3000 samples ...
156 views

### Intuition behind Occam's Learner Algorithm using VC-Dimension

So I'm learning about Occam's Learning algorithm and PAC-Learning where for a given hypothesis space $H$, if we want to have a model/hypothesis $h$ that has an True error of $error_D \leq \epsilon$, ...
1 vote