Principal Data Engineer

Location: Babraham Research Campus, Cambridge

Type: Full time, permanent 

Start: Immediate

Salary: Competitive / Hours: 40 p/w

Office based position (hybrid working available)

Your role in our team:

We are looking for a highly experienced and passionate Principal Data Engineer to partner with and support our Scientists and Bioinformaticians by addressing their data management challenges. In this highly collaborative role you will help acquire, curate, store and retrieve any type of data and metadata required by the teams. You will design the databases required by different teams to manage and make their data searchable.

As Principal Data Engineer you will support the development of a data pipeline for continuous ingestion of new datasets from external/internal sources into the relevant databases. This will involve the Principal Data Engineer working closely with the Bioinformaticians to set up a scalable analysis platform, integrated with API calls to the dataset.

Your key responsibilities will include:

• Develop pipeline to acquire and ingest data required by the Scientists and the Bioinformaticians
• Develop and oversee the design of data architectures, including data models, data integration, and data storage solutions. Collaborate with the IT team to implement robust data infrastructure and ensure seamless data flow across systems.
• Drive the use of data analytics and business intelligence tools to derive valuable insights from data. Work closely with Scientists and Bioinformaticians to develop advanced analytics solutions. Promote data-driven decision-making across the organization.
• Implement data quality management processes to identify and rectify data errors or inconsistencies. Monitor data quality metrics and take corrective actions as needed. Ensure data accuracy and reliability.
• Ensure compliance with relevant standards such as ISO and GxP and data security best practice
• Advise on optimized data analysis infrastructure and resources
• Develop libraries and APIs for access to the dataset
• Provide secure access to data and resources
• Manage storage of raw data set and setup archival process
• Determine best practices for data management and storage
• Monitoring data, lab equipment and IT infrastructure usage, with dashboard

You…

• Have a Bachelors’ degree in a field with a strong quantitative and informatics aspect (such as Computer Science/ Mathematics/ Statistics) or equivalent experience, followed by significant experience developing and working with a variety of databases and data sets, preferably within the Life Science sector
• Have demonstrable experience independently supporting the data needs of small groups of data scientists and informaticians 
• Are a strong collaborator, used to working cross-functionally across all levels within a growing organisation
• Have a deep interest in data as a resource, with instinct and passion for supporting missions that are data-driven and depend on getting data provisioning right
• Have a proactive problem-solving approach and a high level of initiative, along with good ability to organise yourself well at work
• Have excellent written and verbal communication skills

With essential experience in…

• Significant experience developing and working with a variety of databases and data sets
• Additional extensive, deep experience in data engineering with demonstrable experience of;
  • Developing/ optimising high-volume data pipelines, large datasets and big-data architectures
  • Successfully and independently serving data needs of a small group of data scientists/ informaticians and other users of data
  • Performing analysis on at least a couple of types of data sets to understand their properties and advising end-user teams on their value
  • Successfully building processes for transforming data, creating unique data structures to suit end uses, ensuring sufficiency of metadata, and developing methods for automated delivery of data sets (software tools, APIs)
  • Working on building and using data stores in AWS
  • Working with a variety of stakeholders and cross-functional teams, performing analysis of their data requirements and documenting it
  • Working and programming in the Linux environment
  • Managing multiple requests and priorities in a fast paced environment
  • Big data tools and stream-processing systems such as: Hadoop, Spark, Kafka, Storm, Spark-Streaming
  • Relational SQL and NoSQL databases, including Postgres and Cassandra.
  • Data pipeline and workflow management tools: Luigi, Airflow, etc.
  • AWS cloud services: EC2, S3, Glue, Athena, API Gateway, Redshift
  • Experience with object-oriented and scripting languages: Python, JavaScript, Scala, etc.
  • Designing and building APIs (RESTful, etc.)
  • Ontologies such as Gene Ontology, and ontological modelling tools and editors such as W3C Wiki, Basic Formal Ontology, etc.
  • Understanding of FAIR principles
  • Experience having built GxP systems

and possibly...

• Experience working within a rapidly expanding, scale up biotech environment
• Experience in the direct management of small teams
• Practical experience working to ISO and GxP standards
• Experience with BI tools such as Power BI, Kibana and AWS Quicksight
• Exposure to cost optimisation in relation to data management