# Tag Info

## New answers tagged data-mining

1

The automated generation of pivot tables is a well-studied topic. In fact, Google has a patent on the technology. The general approach is: Identify low-cardinality, categorical data features that have many repetitions. Those features become the row and column candidates for the pivot table. Identify numerical data that are the candidates for the data (or ...

0

To answer your question we need to understand what the aim of the clustering analysis that you are doing. Some of goal's of clustering analysis are: Outlier Detection, Pattern Detection, Grouping Data together, etc Now depending on the type of data, we can choose the algorithm that best fits the data at hand. If you have only numerical features, then you ...

3

When it comes to choosing a threshold, I can see 3 approaches: Make an educated guess This is what you are currently doing. You pick a value and would need to argue why this is a reasonable threshold. Obviously, the argument is as strong as the assumptions you make. Unsupervised way If you compute the matching score for all pairs between A and B, you can ...

1

It is not the case that all data resembles some manifold, for most reasonable meanings of the phrase "resembles some manifold". Mathematically, zero dimensional manifolds are collections of points, and technically speaking all finite data sets can be thought of as zero dimensional manifolds. However, I'm quite sure that's not what you had in mind when you ...

0

I believe your Text version is outdated. Please update Text add-on. For issues with Orange, please always head over to the issue tracker and see if your issue already exists. If not, open a new issue.

0

This seems to me now to be a typical case of a bipartite graph. In the real world, this can be viewed with customers and products; where each customer is linked to a product that they bought, but node customers or product nodes cannot be linked to each other. I do not know the SOTA for that, but a useful tool to check with Python is NetworkX. Perhaps, you ...

0

From the top of my head I think you could build something similar to an autoencoder. $X$ as input, $Y$ as label: $Y' = f_\theta(X) \approx Y$ as label, $f_\theta$ is a Neural Network, $\theta$ the weights and your loss $\mathcal{L} = d(Y,Y')$, where $d$ is some convex distance measure like Mean-Squared-Error or Binary-Crossentropy (if you scale your ...

0

Make X as feature Y as Label Build a model on it with 80% example. Predict Y for 20% and check if it is very close to Y_actual. If you are getting a good score. Then you are good. Predict Y for new X If Y_pred is close to Y_actual. You may assume both are linked.

0

On the surface, There are feature selection methods type like filter, wrapper etc. which uses techniques like chi-sqaure, ANOVA,information gain etc. There also exists feature importance method in sklearn ExtraTreesClassifier which gives you the importance of the feature. here are some to give you a glimpse of- https://machinelearningmastery.com/feature-...

0

I too would fall back on parsing either the HTML or the entities using regular expressions. My experience is though that this always gets unelegant quickly. Do you have a somewhat clear idea of the relevant sources? If the better part of the relevant data comes from a limited number of pages, you could maintain a list of sources with matching wrappers. ...

0

You can split it into two dataframe containing only one of P1 and P2 first. df1 = df[df.index[df['Product'] == 'P1'].tolist()] df2 = df[df.index[df['Product'] == 'P2'].tolist()] And then marge df1 and df2 on Customer ID and Product df1.merge(df2, 'inner', left_on=['Customer ID', 'Product'], right_on=['Customer ID', 'Product'], copy=False)

0

Since the daily dataset does not contain labels, you could aggregate the daily data into annual and then do the join. It sounds like a (binary) classification problem, which can be done using methods such as logistic regression. You will however have to handle missing values caused by the left join, one method would be imputing them. Or just doing an inner ...

Top 50 recent answers are included