Consultant-Data Engineer
Trivandrum, KL, IN, 695581
Data Engineer + AI
Experience Range: 5-7 years
We are seeking a talented and experienced Azure Databricks / Synapse Engineer to join our dynamic team at Allianz where you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will collaborate with cross-functional teams to ensure the efficient collection, storage and retrieval of data for analysis and reporting purposes enabling better insight and enabling decision making.
As a trusted expert, you will design and implement reliable ETL/ELT workflows, integrate both real-time and batch data sources, and ensure secure, well-governed data pipelines and products. You will excel in a collaborative, client-focused environment, with a passion for solving complex data challenges, driving innovation, and delivering high-quality data solutions.
In addition, you will support and enable AI-driven data solutions by designing and maintaining data architectures that power Generative AI use cases, including Retrieval-Augmented Generation (RAG), vector databases, embedding models, semantic search, and knowledge retrieval platforms
A continuous improvement mindset is essential, with the ability to identify opportunities to automate repetitive processes, streamline data engineering workflows, and adopt emerging AI technologies to enhance efficiency, reliability, and business value.
The key responsibilities of the role include:
- Working with peers and senior colleagues on solutions design activity for items allocated to you in the backlog.
- Developing, implementing, testing and documenting scalable ETL (Extract, Transform, Load) data pipelines and workflows to efficiently ingest and transform data from various sources.
- Working collaboratively with internal customers to deliver high-quality fit for purpose solutions.
- Implementing data security measures to ensure compliance with data privacy regulations.
- Peer reviewing work of colleagues to ensure delivery of robust solutions and improve your understanding of the code base.
- Collaborating with stakeholders and analysts to understand their data requirements.
- Staying up-to-date with industry trends and best practices in data engineering to drive continuous improvement and innovation.
- Design and support data platforms that enable Generative AI, RAG, vector search, and semantic retrieval solutions.
- Develop and optimize data pipelines for embedding generation, document indexing, and AI-ready data products.
- Leverage Microsoft Copilot and other AI-assisted engineering tools to improve code quality, automate development tasks, accelerate troubleshooting, and increase delivery velocity.
- Continuously evaluate and adopt modern automation, orchestration, and AI technologies to optimize platform operations and engineering efficiency.
About You
Experience & Skills:
- Bachelor’s or Master’s degree, ideally in Computer Science, Information Systems or a related field
- Proven 5-7 years of work experience as a Data Engineer with either Azure Synapse or Databricks (Spark/PySpark, Delta Lake, Unity Catalog etc)
- Strong SQL and Python (or equivalent language) skills for data manipulation and automation.
- Familiarity with Databricks / Synapse Workflows and other orchestration tools.
- Knowledge of data modelling methodologies (star schemas, Data Vault).
- Familiarity with medallion architecture, data lakehouse principles and distributed data processing.
- Experience with version control tools (GitHub) and CI/CD pipelines.
- Understanding of data governance, security, and compliance frameworks.
- Working Exposure to AI/ML workloads like embedding model, RAG.
- Utilize advanced AI, automation, and productivity tools (e.g., Microsoft Copilot, Claude) to enhance efficiency, automate routine activities, and accelerate the delivery of business solutions.
- Ideally some knowledge of the Motor Insurance Market
105635 | Data & AI | Professional | Non-Executive | Allianz Services | Full-Time | Permanent
Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.
At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.
We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.
Join us. Let's care for tomorrow.
