New answers tagged python
0
votes
0
votes
Why getting error for this very simple class?
You put three underscores around init. Should be two.
- 1
0
votes
Understanding the stochastic average gradient (SAG) algorithm used in sklearn
Yes, this is accurate. There are two fixes to this issues
Instead of initializing y_i =0, instead spend one pass over the data and initialize ...
1
vote
Any Interface/Library that can take the Python ML code and run on spark cluster without learning PySpark?
There's a library called joblib-spark that you can use to leverage a Spark cluster. It lets you take the Scikit-learn code you've written and train it in a distributed, parallel way across a Spark ...
1
vote
Accepted
How to vectorize and speed-up double for-loop for pandas dataframe when doing text similarity scoring
The basic problem here is that Pandas is very slow compared to NumPy in many operations like indexing.
I rewrote this part of code:
...
- 171
0
votes
How to solve a non-linear system with the GAUSS-NEWTON algorithm in Python? (Jacobian matrix J, etc.)
Python mathematical libraries numpy and scipy have routines for solving systems of linear equations: ...
- 101
3
votes
Accepted
Is it bad to average several MAEs calculated from chunks of a big test dataset?
Can I save the MAE from each chunk of data and then average them ?
Yes.
This is perfectly fine.
Why?
Think about the metric's definition.
Caveat: We assume $k$ chunks of equal chunk size $cs$.
If ...
- 418
0
votes
In elbow curve how to find the point from where the curve starts to rise?
When applying this method to data without previous visual inspection, how could this method be modified or supplemented to detect a failure case to indicate there is no elbow or kneebow? I.e. no curve ...
- 1
0
votes
Why do people prefer Pandas to SQL?
In many organizations, the SQL server has too many corporate gatekeepers and too many hoops (logins, permissions, uat/prod env, etc.) that the citizen developer doesn't want to have to jump thru.
1
vote
Accepted
Parsing MIT format binary data produced with DATAQ instrument
I found this Python package from the MIT Laboratory for Computational Physiology that seems to be specifically meant to process that kind of data.
This notebook demonstrates the functionalities of the ...
- 23.8k
0
votes
How do I get my Neural network to ignore certain values?
My experience - I struggled with a similar issue some time ago in a different problem, where I wanted my network to ignore part of the input data. I tried a few approaches: setting a constant small ...
- 171
0
votes
Scikit-learn and TensorFlow with very different MLP models
I know that your question was asked almost a year ago, but still maybe someone will find it useful.
There are two problems:
The first is you are using softmax activation, yet only have one output ...
- 3
0
votes
Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?
I had the same problem as you. However, when you apply the technique in the accepted answer, you cancel out the predictive power gained by class weights.
I also stumbled on CalibratedClassifierCV ...
1
vote
Understand and compute confidence interval and coefficient of variation for regression model
Don't think xgbregressor is that similar to linear regression, so you might be better with understanding the latter first. You took quotes from wiki, but do you understand how it works in practice? ...
- 443
1
vote
How to deal with high data volumes? (Tools, techniques, concepts, etc.)
As it was mentioned, you can use your memory several times more efficiently using other tools (numpy, Redis, C++, R / data.table, etc), but after that you will still hit your laptop RAM limit and then ...
- 1,084
2
votes
How to deal with high data volumes? (Tools, techniques, concepts, etc.)
The fastest data structures in Python are sets.
They are low-level structures, and you can manage your data quickly.
To avoid using too much RAM, you can build them progressively with a CSV with a ...
- 4,589
0
votes
Validation loss much higher than training loss
Another possible issue could stem from imputing missing values. Say you have cases where there are observations with completely missing values and different labels. If all of the data with missing ...
- 21
0
votes
How to encode & scale IP addresses as input for ML models
The domain of all possible random IP addresses is astronomically large, and they have no relation to each other. So any given IP address only provides information about that specific IP address, ...
- 1
3
votes
keep track of all versions of variable values corresponding to a version (commit) of code in jupyter notebook
There are probably some jupyter notebook plugins like this that can do that but it seems very expensive to keep track of the history of every object update made in a big notebook (which seems to be ...
- 326
0
votes
Can lag features be applied into test data without label?
If lagged values are not available for test data you could use model predictions iteratively to create estimated lagged values and use them as input and get results. Just be sure that you understand ...
- 326
1
vote
Scale dataset while preserving relative distributions between columns
Using pandas, you could apply your own standard scaling to each category:
Groups columns per category using pandas.DataFrame.groupby i.e., ...
- 273
0
votes
causal inference vs sensitivity analysis to create acurate ML predictions?
Your post has 2 questions, so I will try to answer both:
Difference between Causal Inference (CI) and Sensitivity Analysis (SA).
Sensitivity Analysis (SA) can be described as the following (Naik & ...
- 273
1
vote
How to run our python scripts utilizing our device's GPU?
While I am not familiar with the full extent, it seems that Modular's Mojo uses the GPU to greatly boost performance. You can use native python code, or use their syntax to enhance the code you're ...
- 11
0
votes
Time Series Forecasting for Multiple Store Sales with Simultaneous Timestamps
In my opinion, you should be predicting the sales of each store separately and then each store will have a date only once in the dataset.
- 188
0
votes
changing to gray scale
Converting an image from RGB to grayscale means combining the data from the three channels into a single channel. You are currently trying to do the conversion by simply reshaping the array, for this ...
- 7,157
0
votes
changing to gray scale
The error you're seeing is because the sizes of the thing you're trying to reshape don't match the sizes you're trying to reshape into. I'm guessing that your input has 3 channels here? Instead of
<...
- 1,015
0
votes
How to interpret my logistic regression result with statsmodels
To summarize from the comments:
statsmodels doesn't automatically add an intercept.
Use the predicted probabilities, not just the hard classification (that you've obtained by rounding the ...
1
vote
Accepted
Which python libraries do you recommend for label ranking?
I found this github repository https://github.com/JasonLC506/LabelRanking that implements some algorithms that I found pretty useful. It only supports python2, but I was able to convert it locally ...
- 11
Top 50 recent answers are included