http://parsec.cs.princeton.edu/

https://github.com/graph500/graph500/tree/v2-spec (the current reference implementation requires MPI, but the v2 branch still has serial and OpenMP versions (etc.))

https://github.com/benchmark-subsetting/NPB3.0-omp-C

http://www.highproductivity.org/SSCABmks.htm (this web site does not exist any more, but there seems to be a copy of some of the benchmarks https://github.com/gtcasl/hpc-benchmarks/tree/master/SSCA2v2.2)

https://github.com/kokkos/kokkos-kernels/tree/master/perf_test
https://github.com/kokkos/kokkos/tree/master/benchmarks

A proposal list has been added (docs/Proposals/TestSuite.rst) in r345074

Roman has been using his image processing suite as part of the AMD Piledriver (BDVER2) scheduler model development, he might be able to advise if this is something that would work well as a test-suite addition:

https://github.com/darktable-org/rawspeed

Roman has been using his image processing
Not really processing, decoding.

suite as part of the AMD
Piledriver (BDVER2) scheduler model development, he might be able to advise
if this is something that would work well as a test-suite addition:

https://github.com/darktable-org/rawspeed

It totally could, both as benchmark, and a test, i would love that.
I even brought it up in https://reviews.llvm.org/D46714#1095030

One caveat: it requires a test set, which is kinda heavy - 756M right now,
not something you'd want to put into test-suite repo.
https://raw.pixls.us/data-unique/ (+ there is rsync), we are looking into
putting it into a git repo to simplify syncing.

I added it to the proposal list (http://llvm.org/docs/Proposals/TestSuite.html) in r345166.

A data set of that size might also be unsuitable for compiler performance testing because a large portion of the runtime will be I/O (unless the host has a sufficient amount of RAM and we can somehow guarantee it being in the cache).

Instead, could we craft smaller RAW images, or use well-compressible images (e.g. all-gray) that are decompressed on-the-fly?

I added it to the proposal list
(http://llvm.org/docs/Proposals/TestSuite.html) in r345166.
Thanks!

A data set of that size might also be unsuitable for compiler performance
testing because a large portion of the runtime will be I/O (unless the host
has a sufficient amount of RAM and we can somehow guarantee it being in the
cache).
I'm not sure i follow. What specifically is the concern here?
The disk IO itself, or that it will be counted as the time consumed by benchmark?
Because the latter is absolutely avoidable by only counting the actual decoding time, not reading time.
(The whole file is already read to memory first, and decoded from memory.)

Instead, could we craft smaller RAW images, or use well-compressible images
(e.g. all-gray) that are decompressed on-the-fly?

Sadly, it's not your typical libpng / libjpeg.
There aren't actual specs for these compression schemes available.
And even then, one would need to implement the compressors,
and then ensure that they correctly roundtrip, etc.
So while it's not unfeasible, just not too feasible..

I'm not sure i follow. What specifically is the concern here?
The disk IO itself, or that it will be counted as the time consumed by
benchmark?

The latter

Because the latter is absolutely avoidable by only counting the actual
decoding time, not reading time.
(The whole file is already read to memory first, and decoded from memory.)

Most programs in test-suite, which the easiest to do, have their total execution time measured (timeit.c or linux perf). It's the user time (not wall-clock or kernel), I don't how much I/O has still an influence on it.

I now see that rawspeed's benchmarks (https://github.com/darktable-org/rawspeed/tree/develop/bench) are already using Google Benchmark, which is understood by test-suite's microbenchmark.py.

I'm not sure i follow. What specifically is the concern here?
The disk IO itself, or that it will be counted as the time consumed by
benchmark?

The latter
OK, great :)

Because the latter is absolutely avoidable by only counting the actual
decoding time, not reading time.
(The whole file is already read to memory first, and decoded from memory.)

Most programs in test-suite, which the easiest to do, have their total
execution time measured (timeit.c or linux perf). It's the user time (not
wall-clock or kernel), I don't how much I/O has still an influence on it.

I now see that rawspeed's benchmarks
(https://github.com/darktable-org/rawspeed/tree/develop/bench)
Eh, those "benchmarks" are bad; so far, i never really looked into writing small proper benchmarks.

are already using Google Benchmark
Absolutely :)
The actual interesting thing is https://github.com/darktable-org/rawspeed/blob/develop/src/utilities/rsbench/main.cpp

which is understood by test-suite's microbenchmark.py.

https://gitlab.com/chriscox/CppPerformanceBenchmarks

https://gitlab.com/chriscox/CppPerformanceBenchmarks

Thanks for the suggestion. I added it to https://llvm.org/docs/Proposals/TestSuite.html in r347369.