HDF5 and Spark do not play well with each other…
… so what if we just use HDF5 for sharing (import/export) and Spark for the rest? That’s what Jeremy and Cyrille and me wondered while sitting in Janelia Farm labs… well, actually in its bar with our laptops, but now you are in context ;)
Please check the nbviewer-formatted notebook and our ongoing tests.