Open Pension Crawler

The repo contains the crawler to get the xls/x data from the Investment firms, in use for the Open Pension project. Open Pension is a "Hasadna" project, that aimed to revealing the secrets behind the Israeli pension market.

Pre Requirements

• Make sure you have Python 3.x and pip installed.

Installation

pip install -r requirements.txt.

Run

• To run yl-invest crawler scrapy crawl yl-invest -o yl-invest.json
• To run as-invest crawler scrapy crawl as-invest -o as-invest.json
• To run clalbit crawler scrapy crawl clalbit -o clalbit.json

Tests

Not yet.

Contribute

Just fork and do a pull request (;