Services > Input Data
Input Data
Successful data science projects depend on high-quality, accessible, and reliable data. In our experience, the most common challenges facing data scientists are:
- Data Accessibility
- Data Reliability
- Data Quality
Data Accessibility
Data scientists spend a significant amount of time locating and preparing data before analysis can begin. To improve accessibility, we recommend transferring the required data from enterprise data warehouses into the Data Science Management (DSM) project data store. This provides faster access and allows project teams to work independently while maintaining governance.
A large amount of public data is also available. Through our Products and Open Source Loaders, we provide tools that simplify loading public data into PPDM databases and DSM projects.
Data Reliability
Reliable data is essential for producing trusted analytical and predictive results. We recommend establishing:
- A single source of truth
- Data source traceability
- Stakeholder collaboration and validation
- Consistent frames of reference and unit conversions
These practices help ensure that the data being used is trusted, understood, and suitable for decision-making.
Data Quality
Poor-quality data leads to poor outcomes. Data quality should be measured and monitored across several dimensions:
- Entirety
- Completeness
- Consistency
- Validity
- Uniqueness
Quality issues should be identified and addressed through cleansing, correction, validation rules, and predictive methods where appropriate.
The higher the quality of the data, the more reliable and accurate the resulting analytics and predictions will be.