Historic Data Retrieval Process
From Maisqual Private Wiki
This page describes the process of data retrieval, extraction and treatment for Historic Batch Process.
It should be used as a template for defining Jenkins projects for analysis.
From an architectural point of view, we have a single Perl script for all steps.
 Process overview
This is to be executed on-demand.
The whole process has been divided into the following parts:
- Get raw data from data sources: SVN, Bugzilla, etc.
- Build survey.
- Project pre-processing: run any data provider: Checkstyle, PMD, etc.
- Project processing: build project, run tests, run SQuORE analysis.
- Project post-processing: extract and consolidate data issued from project processing.
- Data pre-processing: gather, import, and format data.
- Data processing: Run R scripts, plot data.
- Data post-processing & Publishing.