The
first Reproducible Quality-Efficient Systems Tournament (ReQuEST) will
debut at ASPLOS’18 ( ACM conference on Architectural Support for
Programming Languages and Operating Systems, which is the premier forum
for multidisciplinary systems research spanning computer architecture
and hardware, programming languages and compilers, operating systems and
networking).
Organized
by a consortium of leading universities (Washington, Cornell, Toronto,
Cambridge, EPFL) and the cTuning foundation, ReQuEST aims to provide a
open-source tournament framework, a common experimental methodology and
an open repository for continuous evaluation and multi-objective
optimization of the quality vs. efficiency Pareto optimality of a wide
range of real-world applications, models and libraries across the whole
software/hardware stack.
ReQuEST will use the established artifact evaluation methodology together with the Collective Knowledge framework
validated at leading ACM/IEEE conferences to reproduce results, display
them on a live dashboard and share artifacts with the community.
Distinguished entries will be presented at the associated workshop and
published in the ACM Digital Library.
To win, the results of an entry do not necessarily have to lie on the
Pareto frontier, as an entry can be also praised for its originality,
reproducibility, adaptability, scalability, portability, ease of use,
etc.
The
first ReQuEST competition will focus on deep learning for image
recognition with an ambitious long-term goal to build a public
repository of portable and customizable “plug&play” AI/ML algorithms optimized across diverse data sets, models and platforms from IoT to
supercomputers (see live demo). Future competitions will consider other emerging
workloads, as suggested by our Industrial Advisory Board.
For more information, please visit http://cKnowledge.org/request
