89 lines
3.4 KiB
Markdown
89 lines
3.4 KiB
Markdown
|
# FundaScraper
|
|||
|
|
|
|||
|
|
[](https://www.repostatus.org/#active)
|
|||
|
|
[](https://app.travis-ci.com/whchien/funda-scraper)
|
|||
|
|
[](https://codecov.io/gh/whchien/funda-scraper)
|
|||
|
|
[](https://pepy.tech/project/funda-scraper)
|
|||
|
|
[](https://pypi.org/project/funda-scraper/)
|
|||
|
|
[](https://www.python.org/dev/peps/pep-0008/)
|
|||
|
|
|
|||
|
|
`FundaScaper` provides you the easiest way to perform web scraping from Funda, the Dutch housing website.
|
|||
|
|
You can find houses either for sale or for rent, and the historical data from the past few year are also attainable.
|
|||
|
|
|
|||
|
|
Please note:
|
|||
|
|
1. Scraping this website is only allowed for personal use (as per Funda's Terms and Conditions).
|
|||
|
|
2. Any commercial use of this Python package is prohibited. The author holds no liability for any misuse of the package.
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Install
|
|||
|
|
1. The easiest way is to install with pip:
|
|||
|
|
```
|
|||
|
|
pip install funda-scraper
|
|||
|
|
```
|
|||
|
|
2. You can also clone the repository to your local machine with:
|
|||
|
|
```
|
|||
|
|
git clone https://github.com/whchien/funda-scraper.git
|
|||
|
|
cd funda-scraper
|
|||
|
|
export PYTHONPATH=${PWD}
|
|||
|
|
python funda_scraper/scrape.py --area amsterdam --want_to rent --find_past False --page_start 1 --n_pages 3
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Quickstart
|
|||
|
|
```
|
|||
|
|
from funda_scraper import FundaScraper
|
|||
|
|
|
|||
|
|
scraper = FundaScraper(area="amsterdam", want_to="rent", find_past=False, page_start=1, n_pages=3)
|
|||
|
|
df = scraper.run(raw_data=False, save=True, filepath="test.csv", min_price=500, max_price=2000)
|
|||
|
|
df.head()
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
You can pass several arguments to `FundaScraper()` for customized scraping:
|
|||
|
|
- `area`: Specify the city or specific area you want to look for, e.g. Amsterdam, Utrecht, Rotterdam, etc
|
|||
|
|
- `want_to`: You can choose either `buy` or `rent`, which finds houses either for sale or for rent.
|
|||
|
|
- `find_past`: Specify whether you want to find the data in the past or the ones in the market. If `True`, only historical data will be scraped. The default is `False`.
|
|||
|
|
- `page_start`: Indicate which page you want to start scraping. The default is `1`.
|
|||
|
|
- `n_pages`: Indicate how many page you want to scrape. The default is `1`.
|
|||
|
|
- `min_price`: Indicate the lowest amount for the budget
|
|||
|
|
- `max_price`: Indicate the highest amount for the budget
|
|||
|
|
|
|||
|
|
The scraped raw result contains following information:
|
|||
|
|
- url
|
|||
|
|
- price
|
|||
|
|
- address
|
|||
|
|
- description
|
|||
|
|
- listed_since
|
|||
|
|
- zip_code
|
|||
|
|
- size
|
|||
|
|
- year_built
|
|||
|
|
- living_area
|
|||
|
|
- kind_of_house
|
|||
|
|
- building_type
|
|||
|
|
- num_of_rooms
|
|||
|
|
- num_of_bathrooms
|
|||
|
|
- layout
|
|||
|
|
- energy_label
|
|||
|
|
- insulation
|
|||
|
|
- heating
|
|||
|
|
- ownership
|
|||
|
|
- exteriors
|
|||
|
|
- parking
|
|||
|
|
- neighborhood_name
|
|||
|
|
- date_list
|
|||
|
|
- date_sold
|
|||
|
|
- term
|
|||
|
|
- price_sold
|
|||
|
|
- last_ask_price
|
|||
|
|
- last_ask_price_m2
|
|||
|
|
- city
|
|||
|
|
|
|||
|
|
You can use `scraper.run(raw_data=True)` to fetch the data without preprocessing.
|
|||
|
|
|
|||
|
|
## More information
|
|||
|
|
|
|||
|
|
You can check the [example notebook](https://colab.research.google.com/drive/1hNzJJRWxD59lrbeDpfY1OUpBz0NktmfW?usp=sharing) for further details.
|
|||
|
|
Please give me a [star](https://github.com/whchien/funda-scraper) if you find this project helpful.
|
|||
|
|
|
|||
|
|
|