Questions tagged [theory]

Theory relates to theoretical questions regarding data science and machine learning.

62 questions
Filter by
Sorted by
Tagged with
30 views

Does Increasing Dimensionality Before Compression Make Sense for Anomaly Detection with Autoencoders?

Given a dataset $X$ of shape $(n, p)$ such that $n \gg 1$ and $p \approx 10$, I would like to train an autoencoder to solve an anomaly detection problem. I did some experiments considering a classical ...
1 vote
34 views

Are there any general theoretical results about the behavior of data in the neighborhood of a single data point?

I know from calculus that any relatively well-behaved function $y=f(x)$ can be approximated by a linear function $y=ax+b$ within a sufficiently small neighborhood around each point of an independent ...
• 1,146
1 vote
40 views

Сan we say that the regression problem is essentially a classification problem with an infinite number of classes?

I'm a newcomer to machine learning and currently diving into supervised learning methods. I've already grasped the theoretical basics of classification tasks and have just started exploring regression....
16 views

Reinforcement Learning: Formally, does an exponentially decaying epsilon satisfy GLIE?

I am aware that exponentially decaying exploration constants (epsilon) are used practically. Formally, though, do they satisfy the GLIE condition? Specifically, relating this to the Borel-Cantelli ...
30 views

Check the validity of the distribution in the proof of No-Free-Lunch Theorem

I'm reading the proof of No-Free-Lunch Theorem (quoted at the end of this question) in Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, p.37, the author wrote: ...
• 155
1 vote
63 views

Where does AI/ML theories come into play when nowadays the AI libraries already so powerful?

I've read up posts so-call for beginning ML, claiming you need linear algebra, statistics, complicated optimization to just getting start in ML/AI. And on top of these, there comes the ML/AI ...
• 411
1 vote
29 views

Are there any established approaches for dealing with a Degenerate Feedback Loop?

Scenario: I develop a model which forecasts the likely sales success of a particular enquiry based on outcomes of past similar enquiries. I then assign this likelihood score to new enquiries when they ...
• 111
230 views

How to scale a subset of data with respect to the entire dataset

I am developing a financial time-series prediction model using sklearn using StandardScaler for scaling purposes. I train a model, and then use the model regularly ...
• 111
57 views

406 views

Why does Jim Gray call "data-driven science" a new paradigm?

Wikipedia it says about data science: Turing award winner Jim Gray imagined data science as a "fourth paradigm" of science (empirical, theoretical, computational and now data-driven) and ...
• 752
1 vote
195 views

Can a neural network be of variable depth?

It is very common for neural networks to be asymmetric about the x axis, that is, to have many more nuerons in the first few layers than in the last few layers. Common example: But can neural ...
• 211
1 vote
4k views

What is an object detection problem with only one class called?

Object detection is defined as the problem in which a model needs to figure out the bounding boxes and the class for each object. A lot of ML solutions for object detection base around having "two ...
• 113
58 views

I was reading this nice tutorial about Pytorch's basics: https://pytorch.org/tutorials/beginner/pytorch_with_examples.html In the first example (pure Numpy), the author starts the backward phase by ...
• 103
1 vote
113 views

What is the best approach for send time optimization? [closed]

I could no find a lot information about how the companies doabout send time optimization, either for push notifications or email campaigs. having historical data about clicks and sends what would be ...
1 vote
732 views

Is the search space of Hyperparameters Continuous or Discrete?

I am looking into hyper-parameter tunning and was curious about whether the search space is considered continuous or discrete? My understanding of both those cases: 1. Continuous would make it 'easier'...
• 11
36 views

What is the name of this statistical interaction?

What is the name of the following statistical / informational interaction: given A, I know exactly what B is. given B, I know to some extent what A is. I'm not looking for a probability but rather ...
• 105
37 views

Structuring experiment/training data with months in mind

We're using a whole year's data to predict a certain target variable.The model works like data - OneHot encoding the categorical variables - MinMaxScaler - PCA (to choose a subset of 2000 components ...
• 1,359
120 views

Encoding correlation

I have rather theory-based question as I'm not that experienced in encoders, embeddings etc. Scientifically I'm mostly oriented around novel evolutionary model-based methods. Let's assume we have ...
• 834
2k views

Explanation of inductive bias of Candidate Elimination Algorithm

The definition of inductive bias says that The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs given inputs ...
124 views

Understanding LSTM Keras implementation

So I understand what LSTM units are. But I have trouble understanding the implementation / function in Keras framework. Let's say, I add a layer ...
• 63
66 views

Popular classification algorithms over time

In the book "Deep Learning with Python" by Francois Chollet (2018), in section 1.2.4 one can find: Decisions trees learned from data began to receive significant research interest in the ...
43 views

Is it possible for a neural net to score as high as a different form of supervised learning?

I've been working with the Adult Census Income dataset from UCI http://archive.ics.uci.edu/ml/datasets/adult I've created two different models, one using a gradient boosted classifier with sklearn, ...
149 views

• 1,267
88 views

A question on realizable sample complexity

I came across the following exercise, and I just can't seem to crack it: Let $l$ be some loss function such that $l \leq 1$. Let $H$ be some hypothesis class, and let $A$ be a learning algorithm. ...
1 vote
58 views

What could cause validation set to consistently perform better than training?

I'm training a neural network with a very small dataset just to get things set up, before training on a much larger set. (I only have about 500 data points available to me at this time, with more ...
137 views

Decentralized machine learning

Currently, to train a model, you need to collect a huge blob of data. Are there feasible concepts of decentralized machine learning? Like, feed the model somehow from isolated data sources, or merge ...
68 views

What does it mean when someone says "Most of the data science algorithms are optimization problems"

I was trying to understand the Gradient Descent algorithm from this article and the author says Most of the data science algorithms are optimization problems I come from software engineering ...
119 views

What should be the requirement for training data in order to obtain a good regression model using neural network?

I have made a neural network regression model using the theory for the first time and would like to clarify some basic doubts, whose concrete answers I couldn't find yet. Data:- I have 3000 samples ...
• 103
157 views

Intuition behind Occam's Learner Algorithm using VC-Dimension

So I'm learning about Occam's Learning algorithm and PAC-Learning where for a given hypothesis space $H$, if we want to have a model/hypothesis $h$ that has an True error of $error_D \leq \epsilon$, ...
• 101
1 vote