Questions tagged [experiments]

An experiment is a procedure carried out to support, refute, or validate a hypothesis. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs when a particular factor is manipulated.

Filter by
Sorted by
Tagged with
1
vote
1answer
27 views

Checking if ML model is possible

How can I check if a machine learning model is feasible on a given dataset? What techniques like EDA, correlation etc. can be used to judge if a model is possible i.e. data and predictor variables ...
0
votes
0answers
15 views

Experimental design with small dataset

I am working with small medical images dataset, my goal is to classify images into categories. The dataset classes have rather big intra class variation, as on some of the images broken bones, screws, ...
1
vote
1answer
101 views

Annotation tool for classification experiments

What tools are available that provide an interface to present text classification results? I need to keep and compare the results of different classification algorithms for unsupervised data. The tool ...
4
votes
2answers
418 views

Can we use difference-in-differences with a biased A/B test?

We noticed we had a biased sample in our A/B test and was wondering if difference-in-differences would help us make valid conclusions about the data, or if there was another way to proceed. We ran an ...
2
votes
1answer
31 views

Are there any reports on Hardware errors affecting experiments?

I recently wrote this as a list of what I have seen / can think of as problem sources that make it hard to reproduce (replicate?) an experiment. I think I have seen most of them, except for hardware ...
4
votes
1answer
250 views

A/B testing: How to calculate p-value on post test segments?

My question on A/B testing is about doing post test segmentation analysis. For example: I run an A/B test on my website to track bounce rate. On the treatment group, i put a video to explain ...
0
votes
1answer
693 views

Approaches to A/B testing when you can't randomize on the user level

I am trying to find suggestions for different approaches to running tests when it isn't possible to randomly assign users to test and control buckets. For example, if I own 10 shops and have a ...
1
vote
1answer
177 views

Do Data Miners realise Machine Learning cannot establish Causality? [closed]

Do people in Machine Learning (or, more generally, in Data Mining) realise that no causal link can be inferred from a correlation? My impression of the ML/AI community is that most people involved ...
5
votes
2answers
484 views

AB testing : When AA testing doesn't work

After 6 months of AB testing on our CRM tool (Oracle Responsys, but this could be true with anyone), the test exhibited some weird results so we decided to pause everything, and to make some good old ...
-1
votes
1answer
85 views

Avoiding leakage on my random forest?

I am training a random forest model. I am wondering if it is safe (leakage?) to use on my training set the parameter average price of a car calculated using all my data points. The issue is that some ...
1
vote
2answers
198 views

How to determine if a company decision was successful or not?

I'm trying to figure out if a decision taken in a company (offering discounts for specific products) is successful or not. I have done some research and saw that A/B testing might be a way to do this ...
4
votes
3answers
2k views

What are the methods to ensure that the population split for A/B test is random?

Before launching an A/B test, what are the methods to ensure that the population split in control and target group is random for a particular label say, purchase rate.
3
votes
2answers
77 views

How do I learn experimental methodology? When is it relevant?

I just graduated in Computer Science, with a very theoretical background but without any kind of Data Science or Artificial Intelligence experience, and I working on my own to discover those two ...
1
vote
0answers
22 views

Experiments on time spent by the visual cortex

I'm working on a object detection problem, using convolutional neural networks which have a lot in common with how the human visual cortex works. I've previously seen remarks on tasks in computer ...
4
votes
2answers
774 views

Recommendations for storing time series data

As part of my thesis I've done some experiments that have resulted in a reasonable amount of time-series data (motion-capture + eye movements). I have a way of storing and organizing all of this data, ...
3
votes
1answer
56 views

Attributing causality to single quasi-independent variable

Apologies if this isn't the correct place to ask - I'm not sure if this fits best with Stats or Data Science. I'm using analytics to help marketers identify attributes of their users correspond to ...
3
votes
0answers
33 views

Timing sequence in MapReduce

I'm running a test on MapReduce algorithm in different environments, like Hadoop and MongoDB, and using different types of data. What are the different methods or techniques to find out the execution ...
10
votes
2answers
332 views

Book keeping of experiment runs and results

I am a hands on researcher and I like testing out viable solutions, so I tend to run a lot of experiments. For example, if I am calculating a similarity score between documents, I might want to try ...
3
votes
1answer
71 views

Estimating Variance Reduction Resultant from Additional Data

I couldn't quite think of how best to title this, so recommendations are welcome. Same goes for the tags (I don't have the reputation to use the tags that I thought were appropriate). The question is ...
9
votes
4answers
246 views

How to debug data analysis?

I've came across the following problem, that I recon is rather typical. I have some large data, say, a few million rows. I run some non-trivial analysis on it, e.g. an SQL query consisting of several ...