-
Notifications
You must be signed in to change notification settings - Fork 1
/
Benchmark.txt
258 lines (217 loc) · 10.7 KB
/
Benchmark.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.JavaSerializer.simpleObjectDeserialization
# Run progress: 0.00% complete, ETA 00:02:36
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: JavaSerializedSimpleObjectSize: 325
JavaSerializedComplexObjectSize: 508
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
28758.272 ns/op
# Warmup Iteration 2: 19783.999 ns/op
Iteration 1: 19171.540 ns/op
Iteration 2: 18962.081 ns/op
Iteration 3: 18987.660 ns/op
Iteration 4: 18991.632 ns/op
Result "benchmark.JavaSerializer.simpleObjectDeserialization":
19028.228 ±(99.9%) 623.159 ns/op [Average]
(min, avg, max) = (18962.081, 19028.228, 19171.540), stdev = 96.435
CI (99.9%): [18405.069, 19651.387] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.JavaSerializer.simpleObjectSerialization
# Run progress: 16.67% complete, ETA 00:02:12
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: JavaSerializedSimpleObjectSize: 325
JavaSerializedComplexObjectSize: 508
5320.806 ns/op
# Warmup Iteration 2: 5819.006 ns/op
Iteration 1: 4563.551 ns/op
Iteration 2: 4603.952 ns/op
Iteration 3: 4598.300 ns/op
Iteration 4: 4590.296 ns/op
Result "benchmark.JavaSerializer.simpleObjectSerialization":
4589.025 ±(99.9%) 115.558 ns/op [Average]
(min, avg, max) = (4563.551, 4589.025, 4603.952), stdev = 17.883
CI (99.9%): [4473.466, 4704.583] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.ProtoBuf.protobufSimpleDeserialization
# Run progress: 33.33% complete, ETA 00:01:46
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: 379.002 ns/op
# Warmup Iteration 2: 364.028 ns/op
Iteration 1: 356.628 ns/op
Iteration 2: 354.969 ns/op
Iteration 3: 356.416 ns/op
Iteration 4: 355.893 ns/op
Result "benchmark.ProtoBuf.protobufSimpleDeserialization":
355.976 ±(99.9%) 4.776 ns/op [Average]
(min, avg, max) = (354.969, 355.976, 356.628), stdev = 0.739
CI (99.9%): [351.201, 360.752] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.ProtoBuf.protobufSimpleObjectSerialization
# Run progress: 50.00% complete, ETA 00:01:19
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: 1042.872 ns/op
# Warmup Iteration 2: 894.315 ns/op
Iteration 1: 879.601 ns/op
Iteration 2: 879.863 ns/op
Iteration 3: 876.438 ns/op
Iteration 4: 880.670 ns/op
Result "benchmark.ProtoBuf.protobufSimpleObjectSerialization":
879.143 ±(99.9%) 12.017 ns/op [Average]
(min, avg, max) = (876.438, 879.143, 880.670), stdev = 1.860
CI (99.9%): [867.126, 891.160] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.Quanta.simpleObjectDeserialization
# Run progress: 66.67% complete, ETA 00:00:53
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: QuantaSerializedSimpleObjectSize: 82
QuantaSerializedComplexObjectSize: 266
612.254 ns/op
# Warmup Iteration 2: 525.948 ns/op
Iteration 1: 520.347 ns/op
Iteration 2: 519.710 ns/op
Iteration 3: 520.379 ns/op
Iteration 4: 519.452 ns/op
Result "benchmark.Quanta.simpleObjectDeserialization":
519.972 ±(99.9%) 2.998 ns/op [Average]
(min, avg, max) = (519.452, 519.972, 520.379), stdev = 0.464
CI (99.9%): [516.974, 522.970] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.Quanta.simpleObjectSerialization
# Run progress: 83.33% complete, ETA 00:00:26
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: QuantaSerializedSimpleObjectSize: 82
QuantaSerializedComplexObjectSize: 266
862.414 ns/op
# Warmup Iteration 2: 720.802 ns/op
Iteration 1: 721.947 ns/op
Iteration 2: 721.986 ns/op
Iteration 3: 720.166 ns/op
Iteration 4: 723.108 ns/op
Result "benchmark.Quanta.simpleObjectSerialization":
721.802 ±(99.9%) 7.858 ns/op [Average]
(min, avg, max) = (720.166, 721.802, 723.108), stdev = 1.216
CI (99.9%): [713.944, 729.660] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.MessagePack.messagePackDeserialization
# Run progress: 0.00% complete, ETA 00:01:44
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
1817.502 ns/op
# Warmup Iteration 2: 1546.107 ns/op
Iteration 1: 1554.409 ns/op
Iteration 2: 1557.193 ns/op
Iteration 3: 1550.131 ns/op
Iteration 4: 1582.202 ns/op
Result "benchmark.MessagePack.messagePackDeserialization":
1560.984 ±(99.9%) 93.314 ns/op [Average]
(min, avg, max) = (1550.131, 1560.984, 1582.202), stdev = 14.440
CI (99.9%): [1467.670, 1654.298] (assumes normal distribution)
# JMH version: 1.28
# VM version: JDK 11.0.11, OpenJDK 64-Bit Server VM, 11.0.11+9-post-Debian-1
# VM invoker: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
# VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Blackhole mode: full + dont-inline hint
# Warmup: 2 iterations, 3 s each
# Measurement: 4 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: benchmark.MessagePack.messagePackSerialization
# Run progress: 25.00% complete, ETA 00:01:20
# Fork: 1 of 1
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
# Warmup Iteration 1: 1887.809 ns/op
# Warmup Iteration 2: 1388.383 ns/op
Iteration 1: 1383.630 ns/op
Iteration 2: 1416.927 ns/op
Iteration 3: 1522.778 ns/op
Iteration 4: 1447.127 ns/op
Result "benchmark.MessagePack.messagePackSerialization":
1442.615 ±(99.9%) 383.850 ns/op [Average]
(min, avg, max) = (1383.630, 1442.615, 1522.778), stdev = 59.401
CI (99.9%): [1058.765, 1826.466] (assumes normal distribution)
# Run complete. Total time: 00:02:39
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score Error Units
JavaSerializer.simpleObjectDeserialization avgt 4 19028.228 ± 623.159 ns/op
JavaSerializer.simpleObjectSerialization avgt 4 4589.025 ± 115.558 ns/op
MessagePack.messagePackDeserialization avgt 4 1560.984 ± 93.314 ns/op
MessagePack.messagePackSerialization avgt 4 1442.615 ± 383.850 ns/op
ProtoBuf.protobufSimpleDeserialization avgt 4 339.191 ± 5.755 ns/op
ProtoBuf.protobufSimpleObjectSerialization avgt 4 885.939 ± 15.484 ns/op
Quanta.simpleObjectDeserialization avgt 4 485.450 ± 7.972 ns/op
Quanta.simpleObjectSerialization avgt 4 734.367 ± 7.255 ns/op