Questions tagged [finance]
Field concerned with the allocation (investment) of assets and liabilities over space and time, often under conditions of risk or uncertainty.
63 questions
1
vote
1
answer
29
views
Splitting training and test set on a time series problem
I have an OHLCV dataset that starts on 01-01-2000 and ends on 31-12-2003 and I want to evaluate a model, say an SVM regressor.
What is the correct routine to evaluate the performance of the model from ...
- 11
0
votes
0
answers
20
views
How to download all listed companies on Yahoo Finance with detailed data (e.g., sector, market cap) for research purposes?
I'm currently working on a data science research project and I need to download a comprehensive dataset of all listed companies available on Yahoo Finance. The dataset should include detailed ...
0
votes
1
answer
37
views
Efficient anomaly detection in unordered market data - is it possible?
I'm a little bit stuck on how to efficiently model anomaly detection for the following problem, probably because of my lack of experience with time series modelling:
I retrieve market data sorted by ...
- 1
0
votes
0
answers
17
views
How to get probability of an outcome from skewed t distribution in R
I am trying to calculate the probability of stock return to be greater than X in next 28 days, using the skewed t-distribution as it fits the best to the ...
1
vote
1
answer
30
views
How to match the two datasets on Electoral bonds given by SBI?
Just stumbled across the Electoral bond dataset https://www.kaggle.com/datasets/shaundanielll/electoral-bond-data-state-bank-of-india
There are two csvs- One contains Encashment data(Date of ...
- 11
1
vote
1
answer
42
views
Optimization problem of find a pair of values (x, y) such that they produce an output z
Problem
I am trying to create a process that determines the rent and purchase price of a home that gives me a capitalization rate of 0.08 for instance.
Background
Normally I calculate the ...
- 21
0
votes
0
answers
42
views
Measuring effectiveness of debt reminders
I am working for a bank that reminds its customers that they owe money back periodically starting from 3 days before their credit deadline, on the day of the deadline, 3 days after the deadline, a ...
0
votes
1
answer
670
views
Model/variable gini?
I'm working with a colleague concurrently between R and MS Excel looking at credit risk scorecard modelling. In Excel he has calculated what he says is the gini coefficient for certain variables, ...
1
vote
0
answers
47
views
Discrete wavelet transform - DWT (beginner)
I recently stumbled upon this article : https://www.bportugal.pt/sites/default/files/anexos/papers/wp201612_0.pdf
In the paper they use DWT and I am having trouble understanding how to construct them. ...
- 234
0
votes
0
answers
74
views
Which error metric is best for financial returns?
I am trying to predict price change i.e most of the time values around zero (+ and -). In my backtest I predict only one period during test. I would like to know in each iteration which model was the ...
- 234
0
votes
0
answers
16
views
Data source for financial data mining
I plan to do data modeling in the financial area for my master's dissertation. I am thinking of finding the connection between a certain company or country characteristics ( x values) and their ...
- 1
1
vote
0
answers
72
views
Workflow for stock prediction in machine learning
I'm trying to find the best workflow for a stock prediction problem.
My idea goes as follows : I will use a classfication and a regression at the same time
Classification (-1 ; 0 ; 1)
Regression (...
- 234
1
vote
1
answer
321
views
Persistence and stationarity together
I am trying to analyse a time series. I want to get only quantitative results (so, I'm excluding things like "looking at this plot we can note..." or "as you can see in the chart ...&...
- 179
2
votes
0
answers
49
views
change target variable value to reflect better affordability
Context
I am working on a regression problem trying to predict affordability.
My dataset contains daily installments repaying a purchase in a form of contract. Essentially, a minimum daily rate the ...
- 21
1
vote
0
answers
64
views
Legal issues with Machine Learning [closed]
I'm confused with why people claim that current legal system cannot handle any wrongdoings of algorithms that involve Machine Learning and Artificial Intelligence. The claim is that it is impossible ...
- 11
0
votes
1
answer
138
views
"Up or down but not sideways" bimodal time series prediction - what is the best way to model it?
Say I have a time series (e.g. bitcoin price). I want to predict tomorrow's price, specifically tomorrow's % change in price from today. Let's say this is gaussian distributed, with the mean at 0%.
If ...
- 111
1
vote
0
answers
34
views
Linear regression of times series data with heteroskedasticity
I am trying to find out if stock market movements, on average and in extreme conditions, affect gold prices. I am following the regression model proposed by Baur and McDermott (2010) which is given as:...
- 11
0
votes
3
answers
1k
views
Regression and Classification, which is better in financial market price prediction?
I want to use a model to trade in finanical market.
which i have several features, like macd, rsi, or other common features.
and my target is to make a tradeable predict in every time point.
so my ...
- 117
0
votes
1
answer
45
views
Steps to fit a Machine learning model for prediction of up and down market movement
I have around 5 years of data of an index containing many features on a daily basis. I want to classify whether the index will move up or down the next trading day (up or down movement is determined ...
1
vote
0
answers
55
views
Optimize Yahoo Finance Code for Analysis [closed]
I am trying to analyze a number of companies using financial data I gathered from Yahoo Finance. I am also using the yfinance API to get some more details about the company using functions. Since I am ...
- 321
1
vote
0
answers
32
views
How does the GAN based prediction in K. Zhang et al. (2018) improve performance?
In Stock Market Prediction Based on Generative Adversarial Network by K. Zhang et. al, the authors feed financial data (X0...Xt) into an LSTM to predict Xt+1.
Then, they evaluate whether the series (...
- 604
1
vote
1
answer
1k
views
How do I get Multiple CSV files (csv file names will be column names) from a folder to a pandas dataframe?
I think the title is enough for what is my problem here. I have 100 tickers in a folder. I got all csv files to one list. but I wanna see the tickers name which these are csv file names.
what should ...
- 11
0
votes
2
answers
222
views
Should I concat multiple stock timeseries datasets into one?
I have several timeseries datasets of stock data, with fundamental indicators. I would like to build a model that selects stocks for buy and hold.
I understand that to perform this task I have two ...
- 1
1
vote
1
answer
730
views
How to find the best fitting parametric distribution for an empirical dataset (stock returns)?
Given some real-valued empirical data (time series), I could convert it to a histogram to have an (non-parametric) empirical distribution of the data, but histograms are blocky and jagged.
Instead, I ...
- 214
1
vote
0
answers
340
views
Matching financial reconciliation data / matching multiple rows based on column values
I'm working with financial reconciliation data and the ask is to train the algorithm to match transactions (that are otherwise manually matched if the existing application didn't because not all the ...
- 11
2
votes
2
answers
66
views
Predicting financial data (choosing a model)
it is my first time doing something with financial data. I have a dataset with account numbers and some other information about each client (some clients span more than one row since we have info for ...
1
vote
0
answers
20
views
Correcting high AR(1) coefficients in dynamic Gordon model
I have just finished my thesis on a heterogeneous dividend expectations model applied to the COVID-19 crisis! However after receiving some feedback there is one last issue I want to resolve. I'm using ...
0
votes
3
answers
62
views
What's the best way to do classification basing on two given datasets (annual data and daily data)?
I want to do binary-classification basing on two given dataset, one is annual statistical data of a company and has the label I should be able to predict like this:
...
- 26
0
votes
1
answer
39
views
How to measure Covid impact by analysing credit card transaction of customer
I Want to know how can I identify that is the customer is in financial distress due to the COVID situation using its credit card transactions.
I have a daily transaction of customers till current ...
- 11
1
vote
0
answers
22
views
Lasso stricter with more data
I am currently analyzing investment strategies, and have implemented a backtest accordingly. This essentially means that I predict returns each month by using all prior historical data. Consequently, ...
- 111
2
votes
0
answers
35
views
Financial Time Series and Machine Learning question
I am working on a Machine Learning project applied to a financial time series.
Initially, I grabbed features (open, high, low, close) and implemented a Random Forest.
One of the subsequent tasks ...
- 21
2
votes
0
answers
500
views
Will flattening multivariate time series data before clustering make the results meaningless?
I have a large number of financial time series that I wish to do cluster analysis on. Each time series has the same length and spans multiple years of daily data (returns, volatility, etc.). As part ...
- 121
1
vote
1
answer
172
views
Image classification on stock data
I am trying to classify stock images into 4 categories:
Category 1: Image patterns which gives more than 10% profit next 3
day.
Category 2: Image patterns which gives less than 10% profit but
...
- 111
1
vote
0
answers
13
views
Corpus suggestion for financial domain [closed]
I am looking for a financial corpus or any form of publicly available financial texts which is replete with technical terms and acronyms.
Any suggestion is appreciated.
- 111
-1
votes
2
answers
5k
views
What is the approx minimum size of dataset required to build 90% correct model?
I am working with a financial dataset size which is around 3000. I have attempted the supervised-learning regression techniques and not able to go beyond 70% accuracy.
Features: 10
Data size:3700
...
- 153
1
vote
2
answers
78
views
Understandable and explainable machine learning model
I want to find formula for best financial portfolio.
Inputs: Historical fundamental data for last 15 years. For 3000 companies for every quatal we have things like ...
- 121
1
vote
0
answers
20
views
Bootstrapped curves and machine learning
I am exploring machine learning models to employ for my use case as described below and would appreciate some expertise
Effectively I have bootstrapped bond prices to derive a zero coupon curve for ...
- 121
1
vote
2
answers
2k
views
Splitting training and test set with financial data
I am using trees algorithms (decision tree, random forest and XGBoost) to forecast the sign of the returns in the stock market (classification).
I am using this article as a reference:
http://...
- 13
0
votes
1
answer
176
views
How do I find all the drastic changes in the stock and fetch news for those days of the stock?
I am analyzing MindTree company's stocks as an internship project using python pandas and have been given the task to find all the drastic changes by looking at the produced plot for the stock and ...
user77166
0
votes
1
answer
35
views
May I use the same data in several time series intervals?
I am playing with RNNs / LTSMs for a classification task in predicting financial data.
I have a time-series going many years back, and are planning to divide it into a number of shorter time-...
- 115
1
vote
2
answers
286
views
LSTM for financial data
I'm using LSTM to predict financial data. As input data I use log returns and I want to predict the next day market movement. Do I need to retrain the ANN every day in order to keep time consistency ...
- 85
2
votes
2
answers
4k
views
How is data science applied in investment banking?
What do data scientists do at investment banks? What tools are they using? What kind of analysis are they doing? Why are they doing it? Etc.
- 467
2
votes
2
answers
2k
views
Binary Classification Probabilities
I'm using keras in R to predict financial time series. I guess that ANN converges.
But when I do predictions I can see that probabilities don't sum up to 1
I'm using ...
- 85
1
vote
1
answer
759
views
Predict the loss amount for insurance policies using the historical trends and features?
Anyone who has worked with insurance policy datasets, please guide me to an appropriate dataset for this problem .
Also, what is this 'loss amount' and what model will be appropriate for it?
2
votes
1
answer
1k
views
Can LSTM Predict The Next Few Days Of Stock Price?
I have searched many websites and forums describing stock price forecast using LSTM.
They shared two things in common: one is that all the sources make predictions with same set of data and none of ...
- 21
3
votes
1
answer
123
views
Structure the dataset for financial machine learning
I am trying to construct a dataset to apply MLP in forecasting financial returns. The main idea is that I want to predict future equity returns (1 month ahead, but the horizon can vary, just to give ...
- 77
1
vote
0
answers
258
views
Any tool that can help on manually label a time series data please?
i am working with a 10 years weekly financial time series data set, which has the standard format date open high low close volume.
i would like to manually label (classify) the data set with label/...
- 141
1
vote
0
answers
83
views
Are there any APIs for financial pattern recognition? [closed]
I have seen this question on the site, however it is asking for the techniques that can be used for implementation. The thing I want to know is that, on the other hand, if are there any APIs or ...
- 111
2
votes
1
answer
109
views
Preprocess list data
I got question about preparation data for my ML algorithm.
Raw data has format similar to:
...
- 123
2
votes
2
answers
1k
views
how many rows have values from the same columns pandas
I have a df with many columns that represent the market cap of companies that compose an index. The index of the dataframe is dates. Before the company enters the index or after it leaves it, the ...
- 21