Poster Abstract

P.1 Alberto Maurizio Chavan (European Southern Observatory)

Re-engineering data processing for resilience: ALMA's new message-passing backbone

Processing of the data taken by the ALMA Telescope occur over four continents and fourteen timezones. The initial system to support data processing was created out of existing and ad-hoc components connected via a variety of protocols. The design was focused on processing as much data as possible without manual intervention. Once all the data and control flows were fully operational, the system was re-designed to be more resilient, where individual component failures will not necessarily affect the overall flow of data. Standardising on asynchronous message passing allowed the system to gracefully deal with critical components having limited (or no) availability without loss of data.

The re-engineering of the data processing system was completed in four months of very intense development work, including rewriting components from scratch and converting others to asynchronous communication. The resulting system, called ADAPT, was rigorously tested by both the computing team and Science Operations, leading to a formal acceptance in mid-July 2019. After further end-to-end integration testing, ADAPT will go live for ALMA’s Cycle 7 observations in October 2019.