Skip to content

GH-3451. Add a JMH benchmark for variants #3452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
steveloughran wants to merge 12 commits into
base: master
Choose a base branch
from
Open

GH-3451. Add a JMH benchmark for variants #3452

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
3e35efd
GH-3451. Add a JMH benchmark for variants
steveloughran Mar 17, 2026
520c80c
WiP: add a deeper version.
steveloughran Mar 25, 2026
63e6096
revert plans for a deeper version
steveloughran Mar 26, 2026
85a2570
fixes to this benchmark (copilot review)
steveloughran Mar 27, 2026
4edec9a
Measure parquet write/read costs.
steveloughran Mar 27, 2026
e6ccbfc
WiP, on a file read/write benchmark
steveloughran Mar 30, 2026
0889e2d
building a projection schema
steveloughran Mar 30, 2026
2e29ea6
projection works
steveloughran Mar 30, 2026
77bfce4
Test to validate impact of lean schema against unshredded files.
steveloughran Mar 30, 2026
c1c333d
GH-561 ongoing work
steveloughran Apr 1, 2026
b9a5d32
GH-561 profile driven review of VariantConverters.
steveloughran Apr 2, 2026
d75278a
GH-561 Variant benchmarks
steveloughran Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions parquet-benchmarks/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@
<artifactId>parquet-common</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-variant</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
Expand Down
4 changes: 4 additions & 0 deletions parquet-benchmarks/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ read | Reading files with different compression, page and block sizes.
write | Writing files.
checksum | Reading and writing with and without CRC checksums.
filter | Filtering column indexes
variant | Variant performance

Examples:

Expand Down Expand Up @@ -100,6 +101,9 @@ else
"filter")
BENCHMARK_REGEX="org.apache.parquet.benchmarks.FilteringBenchmarks"
;;
"variant")
BENCHMARK_REGEX="org.apache.parquet.variant.Variant*"
;;
esac

echo JMH command: java -jar ${SCRIPT_PATH}/target/parquet-benchmarks.jar $BENCHMARK_REGEX $JMH_OPTIONS
Expand Down
Loading