Technical Lead Data Engineer
Certain Advantage is seeking a Technical Lead Data Engineer based in London. This role is on a contract basis and is hybrid.
The Company
We're working with a business that's on a strong growth trajectory, backed by ambitious plans. With a commitment to innovation and excellence, they are poised to make a significant impact in the industry.
The role:
- Lead the design, development, and implementation of data engineering solutions for our data Lakehouse platform.
- Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
- Provide technical leadership, mentoring, and coaching to the data engineering team.
- Ensure best practices in data engineering, including data modeling, ETL processes, and data quality.
- Optimize big data workloads in Spark and other big data technologies.
- Manage CI/CD pipelines using Azure DevOps or Git.
- Develop and maintain event-driven pipelines using .NET and other relevant technologies.
- Implement and manage Databricks Unity Catalog and other data governance tools.
- Deliver projects in the streaming data world using Kafka, KSQL DB, and similar technologies.
- Utilize reporting tools such as Power BI and Qlik for data visualization and reporting.
The individual:
- Extensive experience with Databricks, Apache Spark, and data lakes.
- Proficiency in Python and SQL.
- Experience with CI/CD processes using Azure DevOps or Git.
- Knowledge of streaming data technologies such as Kafka and KSQL DB.
- Familiarity with reporting tools like Power BI and Qlik.
- Software engineering experience with .NET for event-driven pipelines and automation testing.
- Experience with Databricks Unity Catalog.
- Experience working in Energy trading related projects.
- Strong skills in data modeling, including Snowflake modeling, star schema modeling, and other techniques.
Apply today!
#J-18808-Ljbffr