Skill set:
Python, SQL, Airflow, DBT, Azure, Java
Job profile:
We are seeking experienced Data Engineers to participate in the implementation, maintenance, and optimization of data lakes, data warehouses, and master data management systems. These systems support organizational dashboards, reporting solutions, and data sets for machine learning. The ideal candidate will have expertise in designing, managing, and integrating data from various platforms and formats, with a strong understanding of healthcare data and technologies.
Roles & Responsibilities:
- Design, develop, and maintain scalable and robust data pipelines using Python, SQL, Airflow, and dbt.
- Build and optimize data workflows and data integration processes.
- Ensure data quality, consistency, and reliability across various data sources.
- Implement and manage ETL processes to support data analytics and business intelligence.
- Develops and maintain processes for collecting, aggregating, matching, consolidating, and quality-assuring data.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
- Work with cloud platforms (AWS or Azure) to manage and scale data infrastructure.
- Monitor and troubleshoot data pipelines and workflows to ensure smooth operations.
- Implement best practices for data security, privacy, and compliance.
- Stay updated with the latest trends and technologies in data engineering and recommend improvements to existing processes and tools.
Other specifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- At least 10 years of working with SQL and NoSQL databases, with the ability to write and debug complex queries o Experience with data processing pipelines including ETL, map/reduce, and streaming platforms
- Experience with the development of reports and dashboards
- Experience with multiple programming languages such as Java, C#, Go, Python, JavaScript, and SQL
- Strongly preferred - Experience working with a wide variety of database systems such as PostgresSQL, Microsoft SQL Server, MySQL, Oracle DB, etc.
- Strongly preferred - Experience with, and understanding of, healthcare data and integration technologies including APIs, EDI, HL7, FHIR, XML, and other formats, protocols, and data streams Familiarity with data warehousing concepts and technologies.
- Strong problem-solving skills and attention to detail.
- Familiarity with CI/CD pipelines and version control systems like Git.
- Excellent communication and collaboration skills