The Principal Data Engineer is a key contributor to building and optimizing the Bank’s data pipelines and data infrastructure within a highly scalable, unified platform. This role involves working on batch and streaming data solutions, integrating with tools like Apache Iceberg, Spark on Kubernetes, and Kafka, and ensuring seamless integration for analytics and governance. Additionally, the role includes working with orchestration tools such as Airflow and Argo, and lineage solutions like OpenLineage and DataHub.
Strategy
• Pipeline Development: Design and implement efficient batch and streaming data pipelines to support the platform’s hybrid processing capabilities. • Platform Integration: Ensure seamless integration of data pipelines with Apache Iceberg, Kafka, Airflow, and Argo to optimize data workflows. • Innovation: Contribute to the adoption of new technologies and methodologies for data processing and management to meet evolving business needs. • Data Quality: Implement and automate data validation processes using tools like Great Expectations. • Automation: Develop reusable pipeline templates and framework components for efficient deployment across projects.
Business
• Deliver data pipelines and workflows that directly support analytical and operational use cases, ensuring timely and accurate data availability.
People & Talent
• Team Collaboration: Work closely with platform engineers, framework engineers, and business stakeholders to deliver cohesive data solutions. • Skill Development: Continuously upskill on modern data engineering tools and practices, contributing to a high-performing team.
Governance • Compliance & Regulatory Adherence: Ensure compliance with internal and external regulatory requirements, including data sovereignty, privacy, and resilience standards. • Data Governance & Quality: Implement data governance policies implementation to maintain data quality, integrity, accuracy, and consistency across the platform. Implement monitoring tools to proactively address data quality issues.
Regulatory & Business Conduct • Display exemplary conduct and live by the Group’s Values and Code of Conduct. • Take personal responsibility for embedding the highest standards of ethics, including regulatory and business conduct, across Standard Chartered Bank. This includes understanding and ensuring compliance with, in letter and spirit, all applicable laws, regulations, guidelines and the Group Code of Conduct. • Effectively and collaboratively identify, escalate, mitigate and resolve risk, conduct and compliance matters.
Key stakeholders • Rest of Data & Analytics team • WRB/CIB/GF Architecture teams • Business and Functions Data & Analytics Team
• Bachelor’s degree in Computer Science, Data Engineering, or related field. • Proven experience in building data pipelines with Apache Spark, Kafka, and orchestration tools like Airflow or Argo. • Hands-on expertise with data formats like Iceberg and governance tools such as Apache Ranger, OpenLineage, or DataHub. • Strong SQL skills and familiarity with tools like Trino.
We're an international bank, nimble enough to act, big enough for impact. For more than 170 years, we've worked to make a positive difference for our clients, communities, and each other. We question the status quo, love a challenge and enjoy finding new opportunities to grow and do better than before. If you're looking for a career with purpose and you want to work for a bank making a difference, we want to hear from you. You can count on us to celebrate your unique talents and we can't wait to see the talents you can bring us.
Our purpose, to drive commerce and prosperity through our unique diversity, together with our brand promise, to be here for good are achieved by how we each live our valued behaviours. When you work with us, you'll see how we value difference and advocate inclusion.
Together we:
About our Transformation, Technology & Operations Team
Our Transformation, Technology & Operations (TTO) team is the powerhouse for the Bank. We aim to go further, faster, to ensure we're agile and ready for tomorrow, today. Our diverse network enables us to innovate and build banking solutions that support communities to prosper. We're a place where talented people are encouraged to grow, learn, and thrive, to drive their own career journeys, to reach their full potential.
When you work with us, you’re protecting the reputation and legacy of a 170 year organisation and building on it. We’re driven by progress and continuously evolving to ensure we’re agile and ready for tomorrow, today.
In line with our Fair Pay Charter, we offer a competitive salary and benefits to support your mental, physical, financial and social wellbeing.