OpenNeuro and DataLad

A history of OpenNeuro’s adoption of DataLad and the evolution of DataLad and git-annex support on the platform. In 2017 OpenNeuro was preparing to launch with the original data backend implemented as block storage without git-annex. The decision was made to move OpenNeuro to DataLad and a quick prototype for this backend service was created and eventually brought to production for the public release of OpenNeuro. Since 2017 the platform has evolved to support many of the unique advantages of DataLad datasets. This talk discusses the architecture of OpenNeuro, some of the challenges encountered using git-annex as the center of our application’s data model in cloud environments, solutions developed, and future work to improve upon OpenNeuro’s archival and distribution of DataLad datasets.