Data Engineering is the process of designing, constructing, and maintaining data processing systems. I work with clients to build complex data pipelines for data integration, transformation, enrichment and distribution, with built-in data security and provenance features.
Scalable and fault-tolerant data pipelines to integrate and distribute R&D data based on Apache NiFi, a free and open-source solution.
Develop and deploy container-based, pluggable data services for R&D data transformation and enrichment.
Select among different data model paradigms (E-R, ontology, JSON schema) based on data structures and system/user requirements.
Implement role-based data security policies and single sign-on integration for R&D databases and applications.