Job Title: Databricks Administrator
Location: Bangalore / Hybrid
Job Type: Full-time
Job Description:
We are seeking a skilled and experienced Databricks Administrator to join our team. As a Databricks Administrator, you will be responsible for the installation, configuration, maintenance, and optimization of our Databricks environment. You will work closely with our data engineering and data science teams to ensure the smooth operation of Databricks clusters and the efficient execution of data processing and analytics workflows.
Responsibilities:
1. Install, configure, and maintain Databricks clusters to support data processing and analytics workloads.
2. Monitor Databricks cluster performance and troubleshoot issues to ensure optimal performance and availability.
3. Work closely with data engineering and data science teams to understand their requirements and optimize Databricks configurations accordingly.
4. Implement security best practices to ensure the confidentiality, integrity, and availability of data processed by Databricks clusters.
5. Automate routine tasks related to Databricks administration using scripting and orchestration tools.
6. Collaborate with infrastructure and cloud teams to provision and manage resources for Databricks clusters.
7. Stay updated on the latest developments in Databricks and related technologies to propose and implement improvements to our environment.
8. Document Databricks configurations, processes, and procedures to facilitate knowledge sharing and troubleshooting.
Requirements:
1. Bachelor's degree in computer science, information technology, or a related field.
2. Proven experience as a Databricks Administrator or in a similar role managing big data platforms.
3. In-depth knowledge of Databricks architecture, components, and deployment best practices.
4. Hands-on experience with configuring and optimizing Databricks clusters for performance and scalability.
5. Strong understanding of cloud computing concepts and experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
6. Proficiency in scripting languages such as Python, PowerShell, or Bash for automation tasks.
7. Excellent problem-solving skills and the ability to troubleshoot complex issues in distributed systems.
8. Good communication skills and the ability to collaborate effectively with cross-functional teams.
9. Relevant certifications such as Databricks Certified Associate Developer/Administrator would be a plus.