Oral Abstract

Oral Contribution (O10.3) Alexandar Mechev (Leiden Sterrewacht)

Theme: Delivering accessible and science-ready radio data

LOFAR data: From archive to arXiv

The LOFAR telescope has produced tens of petabytes stored on tape at three Long Term Archive locations. Typically, this data is staged and downloaded by individual researchers and processed on their local compute clusters. While this is a reasonable course of action for a few observations, processing tens, hundreds or even thousands of LOFAR observations is currently not possible at scale. We take advantage of the data locality at the Archive locations and provided compute infrastructure to create a comprehensive framework for batch and interactive processing of LOFAR data. Using this framework, it is easy to produce science ready images from thousands of observation. Moreover, we make it easy for researchers to build up and test their pipelines before deploying them in bulk. Our work makes large scale studies with LOFAR data easier to accomplish, as well as makes it easy to perform Integration tests of pipelines on real LOFAR data. Ultimately, we make it easier for LOFAR data to escape the archive and make its way to arXiv.