Research Software Engineer - User Configurable AI - Ready Datasets

Your role

We are looking for a Research Software Engineer (A2) to work on the development of data infrastructure services to support Machine Learning (ML) efforts within the European Union’s Destination Earth (DestinE) initiative.

ECMWF produces world-leading scientific datasets. Connecting them to the myriad of users around the world is a challenging but truly valuable mission. Patterns of data use have been changing more rapidly than ever before, driven significantly by the explosive growth in AI-model use and training, while the resolution and diversity of the data is ever increasing.

In this world, novel approaches are critical to make datasets available to a wider audience, and for a wider range of uses. ECMWF has developed a range of new technologies for accessing meteorological data. Recently we have developed Zarr-FDB a prototype system to create on-demand, user-defined views of data stored within our indexed data stores (the FDB), built making use of other developments (including Polytope and Gribjump). This allows custom views to be created according to scientifically-meaningful descriptions, which can be directly used for data exploration, visualisation, data-science and ML training.

In this role, you will explore and innovate solutions to improve data and related services to support the scientific and technical development of ML models and processing for the DestinE digital twins. You will develop tools and workflows to assist with assembling and efficiently serving datasets for ML training, and resulting meteorological output. You will have opportunities and be expected to bring creative solutions for handling large datasets across multiple data centres.

This will involve taking the Zarr-FDB prototype functionality and bringing it to life as a user-facing service working with our production team, user-facing service owners, infrastructure providers and researchers to build a robust, observable, performant and above all useful capability.

You will work in the Data Management Services Team in the Development Section of the ECMWF Forecast and Services Department. We build robust and performant software and services to handle and manipulate large volumes of meteorological data. This software is used heavily in ECMWF’s operations, research, projects and initiatives, through all stages of our workflows from acquisition of observations, through model output on the HPC to archival and dissemination, and across a wide range of contexts from HPC, through dedicated storage infrastructure to cloud systems.

We have a special focus on a semantic approach to data handling, such that the technical realities of data handling are abstracted from the scientific software. We expend significant effort to navigate scalability challenges posed by future model upgrades and to exploring new technology and approaches for the future. This supports ECMWF’s strategy of producing cuttingedge science and world-leading weather forecasts and monitoring of the Earth system, and DestinE’s goal to operationalise digital twin technology.

In summary, you will be developing future-proofed data-access, supporting our ML efforts and our external users with tailored, high-performance, scalable, semantically driven data access. This role is funded by the Destination Earth Initiative and collaborates closely with other activities of our Member States and with our partners ESA and EUMETSAT.

About ECMWF

The European Centre for Medium-Range Weather Forecasts (ECMWF) is a world leader in Numerical Weather Predictions providing high-quality data for weather forecasts and environmental monitoring. As an intergovernmental organisation, we collaborate internationally to serve our members and the wider community with global weather predictions, data and training activities that are critical to contribute to safe and thriving societies.

The success of our activities depends on the funding and partnerships of the 35 Member and Co-operating States who provide the support and direction of our work. Our talented staff together with the international scientific community, and our powerful supercomputing capabilities, are the core of a 24/7 research and operational centre with a focus on medium and long-range predictions. We also hold one of the largest meteorological data archives in the world.

ECMWF has also developed a strong partnership with the European Union and has been entrusted with the implementation and operation of the Destination Earth Initiative and the Climate Change and Atmosphere Monitoring Services of the Copernicus Programme and the Strengthening Early Earning in Africa (SEWA) Programme. Other areas of work include High Performance Computing and the development of digital tools that enable ECMWF to extend provision of data and products covering weather, climate, air quality, fire and flood prediction and monitoring.

Our vision: The strength of a common goal

Our mission: Deliver global numerical weather predictions focusing on the medium-range and monitoring of the Earth system to and with our Member States

ECMWF is a multi-site organisation, with its headquarters in Reading, UK, a data centre in Bologna, Italy, and a large presence in Bonn, Germany, as a central location for our EU-related activities. ECMWF is internationally recognised as the voice of expertise in numerical weather predictions for forecasts and climate science.

www.ecmwf.int

About DestinE | Destination Earth

ECMWF is one of the entities entrusted with implementing the DestinE Initiative of the European Commission, alongside ESA and EUMETSAT, and works closely with over 100 partner institutions across Europe. DestinE delivers high-resolution, configurable digital twins of the Earth system that simulate past, present and plausible future environmental conditions and enable “what-if” scenario exploration.

ECMWF is responsible for the delivery of these digital twins and of the Digital Twin engine, the software infrastructure needed to power the digital twins on the European HPC Joint Undertaking (EuroHPC) and ECMWF’s supercomputers and to handle and enable access to their data via the DestinE infrastructure, as well as for a range of Artificial Intelligence/ Machine Learning (AI/ML) activities. These include developments towards an ML Earth system model and a range of AI solutions that increase system interactivity and usability of digital twin data.

The third phase of DestinE (June 2026 June 2028) marks the transition of the Climate Change Adaptation and Weather-induced Extremes Digital Twins, together with the Digital Twin Engine, into sustained operations. During this phase, the focus will be on consolidating, operating, and further evolving these key elements, as well as on maturing the AI Earth-system model components and AI-based solutions developed in phase 2.

For more about DestinE and the Climate DT, see https://destine.ecmwf.intandhttps://destination-earth.eu.

Your responsibilities

• Develop and support tools to handle meteorological data for ML training.
• Explore novel technologies and methodologies to improve ML data generation and access, cloud-optimised data access, and integration with existing services.
• Contribute to operationalisation of Zarr-FDB, and then ongoing production integration and development support for our wider data ecosystem.
• Contribute to tooling and infrastructure for handling DestinE digital twin data.
• Contribute to the our open-source software stack.

What we are looking for

• Excellent analytical and problem-solving skills with a proactive, continuous improvement approach.
• Initiative and ability to work collaboratively with other ECMWF teams but also able to work independently.
• Good interpersonal and communication skills.
• Dedication, passion, and enthusiasm to succeed both individually and across teams of developers
• Highly organised with the capacity to work on a diverse range of tasks to tight deadlines.
• Ability to collaborate effectively in an international work environment and with people from diverse backgrounds.

Your profile

• Advanced university degree (EQ7 level or above) or equivalent professional experience in computer science or engineering, computational science, physics or natural sciences, mathematics, or a related discipline.
• Demonstrated experience developing object-oriented software in C++ and Python is required.
• Experience developing scientific software to process large datasets is an advantage.
• Experience bringing prototype systems into operational contexts is an advantage.
• Experience developing interfaces between languages including Python, C/C++ and Rust is an advantage.
• Experience in system design or network development is an advantage.
• Competence in Object-Oriented Analysis and Design, preferably in C++ or Python is required.
• Demonstrated ability of programming in UNIX/Linux systems is required.
• Familiarity with software lifecycle maintenance is highly desirable.
• Ability to write software in a distributed computing or scientific computing environment is an advantage.

Candidates must be able to work effectively in English; knowledge of one of the Centre’s other working languages (French or German) is an advantage.

If you feel that you have the relevant profile and motivation to join us but don't meet precisely all of the skills above, we still encourage you to apply! ECMWF invests in people and provides training and development opportunities.

Other information

Grade remuneration: The successful candidates will be recruited according to the scales of the Co-ordinated Organisations. Details of salary scales and allowances are available on the ECMWF website at www.ecmwf.int/en/about/jobs.

Starting date: as soon as possible.

Candidates are expected to relocate to the duty station, either Bonn, Germany, or Reading, UK. As a multi-site organisation, ECMWF has adopted a hybrid organisation model which allows flexibility to staff to mix office working and teleworking, including away from the duty station (within the area of our member states and co-operating states). The successful applicant and members of their family forming part of their household will be exempt from immigration restrictions.

Interviews by videoconference (MS Team) are expected to take place shortly after the vacancy closing date. Who can apply

Applicants are invited to complete the online application form by clicking on the apply button below.

At ECMWF, we consider an inclusive environment as key for our success. We are dedicated to ensuring a workplace that embraces diversity and provides equal opportunities for all, without distinction as to race, gender, age, marital status, social status, disability, sexual orientation, religion, personality, ethnicity and culture. We value the benefits derived from a diverse workforce and are committed to having staff that reflect the diversity of the countries that are part of our community, in an environment that nurtures equality and inclusion.

Applications are invited from nationals from ECMWF Member States and Co-operating States as well as nationals of European Union member states. In these exceptional times, we also welcome applications from Ukrainian nationals for this vacancy. Applications from nationals from other countries may be considered in exceptional cases.

ECMWF Member States and Co-operating States are: Austria, Belgium, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Israel, Italy, Latvia, Lithuania, Luxembourg, Montenegro, Morocco, the Netherlands, Norway, North Macedonia, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Türkiye and the United Kingdom.