Azure Databricks_1023

About the Role

We are seeking a highly skilled Platform Support Engineer to manage the administration, optimization, and scalability of our Databricks Environment. This role combines hands-on platform administration with DevOps automation, incident resolution, and cross-functional collaboration to ensure the Databricks platform remains secure, efficient, and high-performing.

Key Responsibilities

1. Platform Administration & Support
• Administer and support Databricks workspaces, clusters, jobs, and integrations with Azure services to ensure seamless day-to-day operations.
• Troubleshoot and resolve platform issues such as job failures, cluster misconfigurations, and access control challenges.
• Implement and maintain monitoring and alerting tools (e.g., Azure Monitor, Dynatrace) to ensure real-time observability of platform health, performance, and cost.
2. Infrastructure Automation & DevOps
• Manage Databricks and Azure infrastructure (e.g., VNets, NSGs, Private Endpoints) using Terraform (Infrastructure as Code).
• Develop and maintain CI/CD pipelines in GitHub Actions to automate deployments and synchronize environments.
• Identify opportunities for automation to reduce manual overhead and improve platform reliability and efficiency.
3. Security, Compliance & Governance
• Enforce platform security best practices including RBAC, network security configurations, data encryption, and audit logging.
• Manage integrations with Azure Active Directory and ensure secure access via private endpoints and identity controls.
• Ensure compliance with internal security policies and applicable regulatory requirements.
4. Incident Management
• Respond to Incidents/service requests (via tools like ServiceNow/Jira), troubleshoot end-to-end technical issues, and document root causes.
• Manage change requests and maintain detailed documentation of solutions, processes, and configurations.
• Participate in rotational shifts to ensure global coverage and timely resolution.
• Maintain SLAs and ensure high customer satisfaction through effective communication and resolution.
5. Collaboration & Continuous Improvement
• Provide DevOps and Databricks expertise to internal teams and customers, fostering best practices in Databricks usage and cost management.
• Stay up-to-date with emerging cloud and data technologies (e.g., Kafka, Event Hubs) to drive innovation and continuous improvement of the platform.

Skills & Qualifications

• Education: Bachelor’s degree in Computer Science, IT, or related field.
• Experience:
• 5+ years in DevOps, Cloud Operations, or Platform Engineering, with hands-on expertise in Databricks administration.
• Advanced proficiency in Microsoft Azure (VMs, VNets, NSGs, Storage, Active Directory, IAM).
• Strong scripting skills in Python, Bash, SQL or PowerShell 
• Expertise in IaC tools (Terraform, ARM templates) and CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins etc).
• Experience with incident management tools (ServiceNow, Jira) and ITIL processes.
• Technical Skills:
• Deep understanding of Databricks architecture, Delta Lake, and Spark optimization.
• Ability to troubleshoot platform performance bottlenecks, security gaps, and cost inefficiencies.
• Soft Skills:
• Excellent communication, collaboration, and stakeholder management.
• Proactive problem-solving and ability to thrive in fast-paced environments.

Preferred Qualifications

• Certifications: Databricks Certified Engineer, Azure Administrator/Engineer, or equivalent.
• Experience with MLOps tools (MLflow, Kubeflow) and data visualization platforms (Power BI, Tableau).
• Knowledge of real-time data processing (Kafka, Event Hubs) and advanced analytics workflows.

Why Join Us?

• Lead transformative projects on a cutting-edge data platform with Fortune 500 clients.
• Flexible remote/hybrid work options and continuous learning opportunities.
• Competitive salary, health benefits, and performance-based incentives.

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us. 
We at Allianz believe in a diverse and inclusive workforce and are proud to be an equal opportunity employer. We encourage you to bring your whole self to work, no matter where you are from, what you look like, who you love or what you believe in.
We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, disability or sexual orientation.
Join us. Let's care for tomorrow.