Downloader Core¶
Table of Contents
Download DODFs from the Buriti Website and save on proper directory.
Download monthly pdfs of DODFs.
Usage example:
downloader = Downloader()
downloader.pull(start_date, end_date)
Downloader Class¶
- class dodfminer.downloader.core.Downloader(save_path='./')[source]¶
Responsible for the download of the DODFs Pdfs.
- Parameters
save_path (str) – Path to save the downloads.
- _download_path¶
Folder in which the downloads will be stored.
- _prog_bar¶
Indicate if download should contain a progress bar.
- pull(start_date, end_date)[source]¶
Make the download of the DODFs pdfs.
All dodfs are downloaded from start_date to end_date inclusively. The Pdfs are saved in a folder called “data” inside the project folder.
- Parameters
start_date (str) – The start date in format mm/yyyy.
end_date (str) – The start date in format mm/yyyy.
Note
The name or the path of the save folder are hard coded and can’t be changed due to some nonsense software engineer decision.
Downloader Private Methods¶
One does not access directly none of those methods, but they are listed here in case the programmer using the downloader library needs more informations.
Path Handling¶
Methods that handle the creation of the paths to the dowloaded DODFS.
- Downloader._create_single_folder(path)[source]¶
Create a single folder given the directory path.
This function might create a folder, observe that the folder already exists, or raise an error if the folder cannot be created.
- Parameters
path (str) – The path to be created
- Raises
OSError – Error creating the directory.
- Downloader._make_month_path(year, actual_date)[source]¶
Create and return the folder for the year and month being download.
- Parameters
year (int) – The year respective to the folder.
actual_date (
datetime
) – The date in which the downloadedcorresponds. (DODF) –
- Returns
The path to the actual month in which the download is being made.
URL Making¶
Methods that construct an URL to further make the download request. .. .. automethod:: dodfminer.downloader.core.Downloader._make_url .. .. automethod:: dodfminer.downloader.core.Downloader._make_href_url .. .. automethod:: dodfminer.downloader.core.Downloader._make_download_url
Web Requests¶
Methods that handle the download request and its execution.
Others¶
Other methods for the downloader library.
- classmethod Downloader._string_to_date(date)[source]¶
Convert the date to datetime.
- Parameters
date (
datetime
) – The date to be converted in string format.- Returns
Return the date formated in string now as datetime datatype.
- Raises
Exception – date passed through cli is in wrong format.