Questions tagged [hierarchical-data-format]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
24 views

What DNN topology can I use to tackle a hierarchical multi-class classification problem?

Suppose that the sample set consists of labelled data, where each label corresponds to a class (say a sub-topic), and every class belongs to a group (a topic). The model should be able to predict the ...
1
vote
0answers
56 views

Advice on dealing with very large datasets - HDF5, Python

I've recently started working on an application for visualization of really big datasets. While reading online it became apparent that most people use HDF5 for storing big, multi-dimensional datasets ...
1
vote
0answers
68 views

Best classification technique for following kind of data set

I have a large table where each record or row represents a single salesperson, and there are 50 columns or dimensions where each column represents one of 50 products potentially sold by any given ...
-1
votes
1answer
17 views

Finding the best “depth” of ICD9 codes with pseudo-hierarchical clustering

Here is a common problem in health care modeling. Did I just invent a new algorithm or has someone already thought of this? The goal is to find the most homogeneous partition of patients by medical ...
1
vote
1answer
31 views

What is the best to identify the proper hierarchy of this data?

So I worked on a hierarchical clustering algorithm to be able to determine which items are most similar, and what attributes are most important. I have two tables: Table 1: contains a bunch of item ...
1
vote
0answers
37 views

Using an ontology to recognize named entities in free text

I'm trying to solve a fairly basic problem in NPL efficiently. What tool or software package would you use to identify the words, or group of words that are part of an given ontology within a free ...
1
vote
1answer
234 views

Using PCA to cluster multidimensional data (RFM variables)

So i am performing k-means clustering on RFM variables (Recency, Frequency, Monetary). The RFM variables are in the form of quantiles (1-4). I used PCA and found the PCA components. I then used the ...
2
votes
0answers
116 views

Using neural networks for classification in Hierarchical data

I am stuck in a problem wherein I have hierarchical data, say $A->B->C$(smaller to biggest), and the smallest unit of data is a block(A consists of multiple blocks, B consists of multiple A's, ...
0
votes
1answer
38 views

How to treat input that inherently has a tree structure?

When having a single vector you use an MLP neural network When having a 2D structure you use a CNN neural network When having a sequence you use a RNN neural network Now you have preprocessed an ...
1
vote
1answer
1k views

Machine learning for predicting HTML Elements on a web page?

My goal is to implement an assistant for crawling web data for users that don't understand anything about HTML or DOM. I will show a web page to the user and the user has to select, what data he is ...
3
votes
1answer
570 views

Derivation of Ward's formula for agglomerative clustering

Figure describe formula of ward method for hierarchical or agglomarative clustering i.e. increment of total error after merging two cluster. How did they get $$\frac{n_A\cdot n_B}{n_A+n_B} ||\vec ...
1
vote
0answers
50 views

Forecasting daily usage of a prouct

Lets say that you have panel data, of daily consumption of product p1 of 10000 individuals. The panel data is on a daily basis for only one month, this means that $t=1,...,30$. The question is how do ...
2
votes
0answers
66 views

How to mitigate the hierarchical error propagation in tree-structured classification

Suppose we have a multi-class classification problem, where the number of classes $K \geq 3$ We use a tree structure of multiple SVMs to divide and conquer the problem, with one example in the figure ...
1
vote
0answers
449 views

What are the strategies for feature engineering in a hierarchical/relational structured data?

I have a feature set with each feature having its own feature-set, and they are laid out in a SQL database with foreign key relationships. The target is defined on the top level feature-set and is a ...
2
votes
0answers
60 views

Spatial clustering of data points on a grid to obtain variable resolution map with constant statistical confidence

I have a grey valued image which is calculated as the mean of a series of images. The value of each pixel is therefore associated to a standard error. The pixel values and the relative standard error ...
1
vote
2answers
225 views

Complete link clustering

I'm conjecturing that with Complete-linkage clustering two elements from the same cluster will always be closer to each other some other element from another cluster. In more formal terms: Let $C$ ...
6
votes
2answers
1k views

Efficient dynamic clustering

I have a set of datapoints from the unit interval (i.e. 1-dimensional dataset with numerical values). I receive some additional datapoints online, and moreover the value of some datapoints might ...
29
votes
3answers
8k views

Hierarchical Data Format. What are the advantages compared to alternative formats?

What are the main benefits from storing data in HDF? And what are the main data science tasks where HDF is really suitable and useful?