|Date Posted||November 10, 2021|
We are looking for a Senior Data Engineer to support the Data Engineering Manager in migrating our on-premise data warehouse to an Azure data platform, ensuring the highest development standards are established across the end to end environment.
You will manage a range of data sources and make sure the data platform for analytic is robust and data is being ingested and structured correctly, making sure it is scalable and optimised for cost and efficiency.
Role & Responsibilities
- Understand the structure and meaning of data in multiple internal and external data sources.
- Maintaining, standardising and revising engineering processes to enable continuous improvement
- Enforce best practice across the team - source control, clean code, development standards, techniques & analysis.
- Peer reviews team members work as part of the release process.
- Ensure that the data and analytics platform is GDPR compliant.
- Keep up to date with external trends in data and analytics platforms, recommending new tools and techniques which meet business requirements now and for the future.
- Ensures provision of a stable environment and continuity of service
- Design, develop, unit test and deliver processes that extract data from sources, transform data to implement business logic and populate the BI data stores (ETL).
- Implement data cleansing activities in ETL and help define business rules to support data quality.
- Adapt ETL processes to accommodate changes in business requirements or source systems.
- Maintain live, test and development environments in partnership with IT.
Skills & Qualifications
- Experienced in managing and automating data pipelines, using the latest technologies like Databricks, coding languages such as Python, R, SQL across a range of platforms.
- Strong exposure to the Microsoft Azure platform (data lake, synapse, data factory)
- Strong knowledge in creating and optimising SSIS packages.
- An expert in data management and engineering, applying this specialist knowledge in collaboration with your team and stakeholders across a range of disciplines including business analysis, data architecture, data optimisation and testing.
- Passion for data and open to new technologies.
- Well versed in dimensional modelling (i.e. Kimball methodology)
- Broad understanding of BI information exploitation methods including ad-hoc queries, data mining techniques, data visualisation, real-time intelligence and data exploration.
- Good stakeholder management capability, engagement and communication skills.
- Solid experience of business analysis, including requirements gathering.
- Creative problem solver