Data Science

Questions tagged [web-scraping]

Ask Question

The tag has no usage guidance.

17 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
1
vote
0answers
11 views

Web scraping using Beatiful Soup

I have this code and I wanna extract holidays, petrol and temperature but I don't know where is the problem. I need your help as soon as possible, please. I want to add this extraction to my dataset ...
2
votes
1answer
37 views

looking for public dataset for stock market

I want to do some modelling and data visualization on historical stock data, including price, volume, financials, etc. Is there an public dataset available for stock price history? I looked at a few, ...
1
vote
1answer
19 views

Web Scraping: Multiple small files or one large file?

I plan to scrape some forums (Reddit, 4chan) for a research project. We will scrape the newest posts, every 10 minutes for around 3 months. I am wondering how best to store the JSON data from each ...
1
vote
0answers
14 views

Good database/plug-in to scrape for academic paper info?

I am trying to scrape the web to find information about a set of academic papers. Unfortunately, for many of the papers, I only have the author's name, a year and part of a title. (For example, [...
0
votes
0answers
13 views

How can I look up secondary keywords in the European Court of Human Rights' document database?

I am experimenting with automating look-up of information from cases from the European Court of Human Rights. This is an example case. Fiddling with the HTTP request syntax and inspecting what the ...
0
votes
0answers
8 views

How do i scrape a data from one specific column?

I want to get the data from this specific column of 'Worldwide' but every time I enter the code: ...
2
votes
1answer
27 views

getting error while scrapping Amazon using Selenium and bs4

I'm working on a class project using BeautifulSoup and webdriver to scrap Disposable Diapers on amazon for the name of the item, price, reviews, rating. My goal is to have something like this where ...
0
votes
1answer
29 views

Use Python to Scape Data on website ends with aspx

https://blue.kingcounty.com/Assessor/eRealProperty/default.aspx This is the website I want to scrape. For example, I input 500 4TH AVE I just want to get Parcel Number and Lot size. I was trying ...
1
vote
0answers
16 views

Cyber crime - data set

I'm doing science project on my university to make an app, which uses AI to detect cyber crimes. I'm looking for sites to make my data set. Do you know any with ads like tobacco, alcohol, ...
1
vote
1answer
24 views

Scraping mixed elements and passing to SQL

I'm running webscrapes via python which are retrieving data from csv's hosted on the web. I'd like to pass the data into a MSSQL database. An issue I have is the mixed elements/data types in the csv....
0
votes
1answer
46 views

Can I download Twitter data via web scraping for research?

I want to do a sentiment analysis using twitter data. Was thinking about hardcoding a cURL script to download data, from a Google Cloud service (I'll run the data on a neural network on the server, to ...
1
vote
1answer
20 views

Turning Histogram values into Numerical format ( Excel-xslx, Pandas-DataFrame, etc.)

I am trying to do a correlation study about personality traits as described in Hofstede's :https://www.hofstede-insights.com/product/compare-countries/ . I would like to have the values described in ...
0
votes
0answers
11 views

building a machine learning model that can detect tables from any webpage without building custom selectors

For example the following image is from Steve Jobs page on Wikipedia. I would like to build a model that can fetch the following info on the image below (Born, Died, Cause of death...etc). basically ...
-1
votes
1answer
16 views

How would one get the HTML structure of a web page as a numeric vector? [closed]

Suppose you want to cluster, or classify, web pages inside a domain. In the same domain similar web pages always have the same structure more or less. (think of every product page an e-commerce would ...
1
vote
0answers
89 views

Bootstrap Template for a Dash App

Where Can I find a Bootstrap Template for a Dash app? Are there implemented examples? I cannot find nothing about this. I am building a Dashboard with Dash/Plotly but I want to improve the style of ...
5
votes
1answer
180 views

Scraping financial web data

I recently started working as a data scientist and I am starting a web scraping and NLP project using Python. The idea is to create a program that searches for public information on the company's ...
1
vote
0answers
22 views

Web scrapping with beautifulSoup is done slowly

I have developed a web scrapping code in Python which takes data from Hattrick.org's matches and returns them in a table so it can be mined, determined likelihood of goals, etc. I have the difficult ...