To allow for efficient work flow I took the initiative to automate serval processes encapsulating them into template based scripts.
Automation Scripts:
- Developed over 30 Pandas scripts to automate end-to-end ETL processes for all 21 data sources.
- Ensured data integrity and security by storing processed data in a secure location, with plans for migration to Azure ADLS.
