New answers tagged

0

I've attempted to create a procedure for this which splits the data into two partitions, but I would appreciate feedback as to whether my implementation is correct from numpy.core.defchararray import count import pandas as pd import numpy as np import numpy as np from math import floor, log2 from sklearn.decomposition import PCA import matplotlib.pyplot as ...


0

Given two vectors of attributes, A and B, the cosine similarity, cos(θ), is represented using a dot product and magnitude as ${\displaystyle {\text{cosine similarity}}=S_{C}(A,B):=\cos(\theta )={\mathbf {A} \cdot \mathbf {B} \over \|\mathbf {A} \|\|\mathbf {B} \|}={\frac {\sum \limits _{i=1}^{n}{A_{i}B_{i}}}{{\sqrt {\sum \limits _{i=1}^{n}{A_{i}^{2}}}}{\sqrt ...


1

In linear regression models (aka OLS), you can interpret the estimated coefficient(s) as the percentage change in case $y$ and $x$ are log-transformed using the natural log (see this post "Both dependent/response variable and independent/predictor variable(s) are log-transformed" or see also this post). For a model like: $$ log_e(y) = \beta_0 + \...


0

I found a useful calculator for solving this problem. https://www.socscistatistics.com/tests/chisquare2/default2.aspx


0

The accuracy is likely to go down if you change the cutoff point to 0.9, since any model tries to separate the classes so that the probability of the correct class is higher than 0.5. But the only way to know would be to actually do the experiment (I assume that the results that you show are obtained with the default cutoff). AUC is a complex measure for a ...


0

I've worked out an example using pen and paper. Essentially, select a Beta value/ Then we create a singleton bucket. Then we check if the value at the position is less then beta if it is then add it to the bucket. The range and frequency can be determined from the histogram i.e. look at x=1, y=3. We end up with a range of 3 and frequency of 1.


0

I have worked out an example. Essentially, we use the matrix on the right hand side to determine $a, b, c,$ and $d$. For objects $x_1$ to $x_2$. A1 A2 A3 A4 0 1 0 0 1 1 0 1 0->1 1->1 0->0 0->1 a=1 c=1 b=1 d=2 (a+c)/(a+b+c+d) = (0+2)/(1+1+1+2)


0

According to this resource. $h$ is a real number such that $h \ge 1$. It represents the Manhattan Distance when $h = 1$ (i.e., L1 norm) and Euclidean Distance when $h = 2$ (i.e., L2 norm). We find the attribute $f$ that gives the maximum difference in values between the two objects.


2

Roughly all three concepts are related. Drift means the relationship between input and output is dynamic and changes (stochastically) over (sufficiently long periods of) time. That is, it is not stationary. For example, consumers' criteria about what to buy, change over time, for example as people become more eco-conscious. More importantly drift, when it ...


0

Further research indicates the most relevant technique to address this problem is data valuation, also known as data shapley. Here are 2 relevant papers on the topic: Data Shapley: Equitable Valuation of Data for Machine Learning What is your data worth? Equitable Valuation of Data I spent a week implementing this technique on my project. It is highly ...


3

What you describe is ordinal encoding. If there is an inherent order to your data (such as age), you can definitely try it. And yes, one-hot encoding does increase dimensionality and sparsity of the data. But these two are not the only ways to handle categorical data. Here is a list of different encoders, and a paper where some of these techniques are ...


4

Which encoding technique to use depends on your data/features. Ordinal encoding is used when there ia a sense of order in your feature. For example you have a feature performance which has values worst, bad good. Here you should use ordinal encoder which will result in worst = 0, bad = 1 and good = 2. We used ordinal encoding because good is better than bad ...


2

You are correct - one hot encoding, by definition, increases your dimensions and (most likely) also the sparsity. Your numerical mapping can be rather misleading since e.g a random forest would interpret adult>child which, in the case of age, makes sense. But say the mapping was {1:"dog",2:"cat",3:"horse"}and not the age of ...


1

The definitions are given on slide 33. For example, for recall of each cluster you need to find niki which you can infer from the definition above: Assign to cluster i the class ki such that ki = argmaxjnij which simply says for each cluster, the index of the highest n is defined as ki. For the case of cluster 1 on the left, it is cluster 3, so the ...


1

At some time, I'll also require the assistance of programmers. He initially approached a private individual, but he worked on the project for a long time with no results. I needed to go a step further and work with a single global company https://www.avenga.com/industries/pharma-life-sciences/. The guys worked quickly and efficiently to complete the job! The ...


0

your use case isn't entirely clear but if i may make some assumptions the company ID in both tables refer to the same company (so ID 0 is the same company in both table) you already have a good idea of feature engineering and know which algo to use for your final classification BUT i am going to treat it like i am modeling it if i were you, i would use 2 ...


Top 50 recent answers are included