Add a simple benchmark for ActiveRecord by tenderlove · Pull Request #5 · ruby/ruby-bench

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

tenderlove · 2021-04-20T21:00:24Z

This benchmark just creates a bunch of records in an in-memory database
and then reads the records back out. The "read" is the part that is
being measured

My machine seems kind of noisy (the stddev is high), but here are the results:

ruby 3.1.0dev (2021-04-20T20:54:33Z yjit 1986541a57) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 1986541a57
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 21.1 7.5 19.6 8.8 1.08 0.91 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:06:44Z

Cool. You're right that the measurements are very noisy. Too noisy to really conclude anything ATM. Good move trying it on AWS.

I would like to see the running time and complexity increased a bit. Maybe adding more iterations, creating more posts, and adding a few extra fields to the schema (what kind of fields would a realistic web forum have on a post, for instance, maybe upvotes, author_id, etc?).

This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove · 2021-04-20T21:27:51Z

I added more columns and increased the iterations. Benchmark results on my machine look like this:

Total time spent benchmarking: 68s
ruby 3.1.0dev (2021-04-20T21:22:37Z yjit 24128733a7) [x86_64-darwin20]
YJIT is enabled
last_commit=cYjitCodeComment is only defined if we're not in debugging mode
git branch yjit
git commit hash 24128733a7
------------- ----------- ---------- --------- ---------- ----------- -------
bench interp (ms) stddev (%) yjit (ms) stddev (%) interp/yjit 1st itr
active_record 218.7 3.4 190.3 4.0 1.15 1.16 ------------- ----------- ---------- --------- ---------- ----------- -------
Legend:
- interp/yjit: ratio of interp/yjit time. Higher is better. Above 1 represents a speedup.
- 1st itr: ratio of interp/yjit time for the first benchmarking iteration.

maximecb · 2021-04-20T21:28:46Z

Perfect!

eregon · 2023-03-23T13:48:18Z

+
+run_benchmark(10) do
+ 1000.times do |i|
+ Post.where(id: i + 1).first.title


See #207 (comment)

THe initial snapshot was from a few years back, the application changed a lot since then. I had to run a bunch of migrations and had to fiddle with various sqlite3/msyql incompatibilities. Also some route changes, but with a bit of effort I managed to make it work again. What is interesting is it seems lobste.rs became 2-3-x as slow in these couple years, I haven't yet profiled to know why. This branch: ``` itr: time ruby#1: 1532ms ruby#2: 1013ms ruby#3: 910ms ruby#4: 852ms ruby#5: 851ms ruby#6: 901ms ruby#7: 857ms ruby#8: 886ms ruby#9: 875ms ``` main: ``` itr: time ruby#1: 884ms ruby#2: 482ms ruby#3: 344ms ruby#4: 358ms ruby#5: 300ms ruby#6: 316ms ruby#7: 315ms ruby#8: 328ms ruby#9: 309ms ```

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

maximecb reviewed Apr 20, 2021
View reviewed changes

Comment threadbenchmarks/active_record/benchmark.rb Outdated

Add a simple benchmark for ActiveRecord
cdfbc21
This benchmark just creates a bunch of records in an in-memory database and then reads the records back out. The "read" is the part that is being measured

tenderlove force-pushed the active_record branch from 6c85435 to cdfbc21Compare April 20, 2021 21:26

maximecb merged commit 314895e into mainApr 20, 2021

maximecb deleted the active_record branch April 20, 2021 21:28

maximecb restored the active_record branch April 20, 2021 21:28

tenderlove deleted the active_record branch April 20, 2021 21:29

eregon mentioned this pull request Jan 8, 2023
Add benchmarks for Sequel with common plugins #159
Merged

eregon mentioned this pull request Mar 23, 2023
Improve the Sequel benchmark to avoid many singleton classes #207
Merged

eregon reviewed Mar 23, 2023
View reviewed changes

Uh oh!

Conversation

tenderlove commented Apr 20, 2021

Uh oh!

Uh oh!

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!

Uh oh!

tenderlove commented Apr 20, 2021

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!

eregonMar 23, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tenderlove commented Apr 20, 2021

Uh oh!

Uh oh!

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!

Uh oh!

tenderlove commented Apr 20, 2021

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!

eregonMar 23, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tenderlove commented Apr 20, 2021

Uh oh!

Uh oh!

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!

Uh oh!

tenderlove commented Apr 20, 2021

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!

eregonMar 23, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tenderlove commented Apr 20, 2021

Uh oh!

Uh oh!

Uh oh!

maximecb commented Apr 20, 2021

Uh oh!