New answers tagged

0 votes

Why getting error for this very simple class?

Try turning __int__ to __init__.
Bobby's user avatar
  • 1
0 votes

Why getting error for this very simple class?

You put three underscores around init. Should be two.
lel's user avatar
  • 1
0 votes

Understanding the stochastic average gradient (SAG) algorithm used in sklearn

Yes, this is accurate. There are two fixes to this issues Instead of initializing y_i =0, instead spend one pass over the data and initialize ...
Robert Mansel Gower's user avatar
1 vote

Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?

There's a library called joblib-spark that you can use to leverage a Spark cluster. It lets you take the Scikit-learn code you've written and train it in a distributed, parallel way across a Spark ...
George Sultani's user avatar
1 vote
Accepted

How to vectorize and speed-up double for-loop for pandas dataframe when doing text similarity scoring

The basic problem here is that Pandas is very slow compared to NumPy in many operations like indexing. I rewrote this part of code: ...
Tomasz Witkowski's user avatar
0 votes

How to solve a non-linear system with the GAUSS-NEWTON algorithm in Python? (Jacobian matrix J, etc.)

Python mathematical libraries numpy and scipy have routines for solving systems of linear equations: ...
Roger V.'s user avatar
  • 101
3 votes
Accepted

Is it bad to average several MAEs calculated from chunks of a big test dataset?

Can I save the MAE from each chunk of data and then average them ? Yes. This is perfectly fine. Why? Think about the metric's definition. Caveat: We assume $k$ chunks of equal chunk size $cs$. If ...
J_H's user avatar
  • 418
0 votes

In elbow curve how to find the point from where the curve starts to rise?

When applying this method to data without previous visual inspection, how could this method be modified or supplemented to detect a failure case to indicate there is no elbow or kneebow? I.e. no curve ...
bedr's user avatar
  • 1
0 votes

Why do people prefer Pandas to SQL?

In many organizations, the SQL server has too many corporate gatekeepers and too many hoops (logins, permissions, uat/prod env, etc.) that the citizen developer doesn't want to have to jump thru.
1 vote
Accepted

Parsing MIT format binary data produced with DATAQ instrument

I found this Python package from the MIT Laboratory for Computational Physiology that seems to be specifically meant to process that kind of data. This notebook demonstrates the functionalities of the ...
noe's user avatar
  • 23.8k
0 votes

How do I get my Neural network to ignore certain values?

My experience - I struggled with a similar issue some time ago in a different problem, where I wanted my network to ignore part of the input data. I tried a few approaches: setting a constant small ...
Tomasz Witkowski's user avatar
0 votes

Scikit-learn and TensorFlow with very different MLP models

I know that your question was asked almost a year ago, but still maybe someone will find it useful. There are two problems: The first is you are using softmax activation, yet only have one output ...
Martinez's user avatar
0 votes

Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?

I had the same problem as you. However, when you apply the technique in the accepted answer, you cancel out the predictive power gained by class weights. I also stumbled on CalibratedClassifierCV ...
Koray Can Canut's user avatar
1 vote

Understand and compute confidence interval and coefficient of variation for regression model

Don't think xgbregressor is that similar to linear regression, so you might be better with understanding the latter first. You took quotes from wiki, but do you understand how it works in practice? ...
Cryo's user avatar
  • 443
1 vote

How to deal with high data volumes? (Tools, techniques, concepts, etc.)

As it was mentioned, you can use your memory several times more efficiently using other tools (numpy, Redis, C++, R / data.table, etc), but after that you will still hit your laptop RAM limit and then ...
Valentas's user avatar
  • 1,084
2 votes

How to deal with high data volumes? (Tools, techniques, concepts, etc.)

The fastest data structures in Python are sets. They are low-level structures, and you can manage your data quickly. To avoid using too much RAM, you can build them progressively with a CSV with a ...
Nicolas Martin's user avatar
0 votes

Validation loss much higher than training loss

Another possible issue could stem from imputing missing values. Say you have cases where there are observations with completely missing values and different labels. If all of the data with missing ...
Chris Coffee's user avatar
0 votes

How to encode & scale IP addresses as input for ML models

The domain of all possible random IP addresses is astronomically large, and they have no relation to each other. So any given IP address only provides information about that specific IP address, ...
ekohrt's user avatar
  • 1
3 votes

keep track of all versions of variable values corresponding to a version (commit) of code in jupyter notebook

There are probably some jupyter notebook plugins like this that can do that but it seems very expensive to keep track of the history of every object update made in a big notebook (which seems to be ...
Giovanni Amorim's user avatar
0 votes

Can lag features be applied into test data without label?

If lagged values are not available for test data you could use model predictions iteratively to create estimated lagged values and use them as input and get results. Just be sure that you understand ...
Giovanni Amorim's user avatar
1 vote

Scale dataset while preserving relative distributions between columns

Using pandas, you could apply your own standard scaling to each category: Groups columns per category using pandas.DataFrame.groupby i.e., ...
Adam Jaamour's user avatar
0 votes

causal inference vs sensitivity analysis to create acurate ML predictions?

Your post has 2 questions, so I will try to answer both: Difference between Causal Inference (CI) and Sensitivity Analysis (SA). Sensitivity Analysis (SA) can be described as the following (Naik & ...
Adam Jaamour's user avatar
1 vote

How to run our python scripts utilizing our device's GPU?

While I am not familiar with the full extent, it seems that Modular's Mojo uses the GPU to greatly boost performance. You can use native python code, or use their syntax to enhance the code you're ...
Skouzini's user avatar
0 votes

Time Series Forecasting for Multiple Store Sales with Simultaneous Timestamps

In my opinion, you should be predicting the sales of each store separately and then each store will have a date only once in the dataset.
Kriti's user avatar
  • 188
0 votes

changing to gray scale

Converting an image from RGB to grayscale means combining the data from the three channels into a single channel. You are currently trying to do the conversion by simply reshaping the array, for this ...
Oxbowerce's user avatar
  • 7,157
0 votes

changing to gray scale

The error you're seeing is because the sizes of the thing you're trying to reshape don't match the sizes you're trying to reshape into. I'm guessing that your input has 3 channels here? Instead of <...
brewmaster321's user avatar
0 votes

How to interpret my logistic regression result with statsmodels

To summarize from the comments: statsmodels doesn't automatically add an intercept. Use the predicted probabilities, not just the hard classification (that you've obtained by rounding the ...
1 vote
Accepted

Which python libraries do you recommend for label ranking?

I found this github repository https://github.com/JasonLC506/LabelRanking that implements some algorithms that I found pretty useful. It only supports python2, but I was able to convert it locally ...
medislamm123's user avatar

Top 50 recent answers are included