Use SierraGenerator for testing (part 2)

[FrancoGiachetta]

Checklist

• Linked to Github Issue
• Unit tests added
• Integration tests added.
• This change requires new documentation.
  • Documentation has been added/updated.

[github-actions]

Benchmark results Main vs HEAD.

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_insert.cairo (JIT)	4.117 ± 0.118	3.920	4.292	1.06 ± 0.04
base dict_insert.cairo (AOT)	3.877 ± 0.069	3.801	3.973	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head dict_insert.cairo (JIT)	3.875 ± 0.021	3.839	3.914	1.02 ± 0.01
head dict_insert.cairo (AOT)	3.800 ± 0.015	3.778	3.825	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_snapshot.cairo (JIT)	3.928 ± 0.090	3.793	4.113	1.07 ± 0.03
base dict_snapshot.cairo (AOT)	3.668 ± 0.035	3.631	3.748	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head dict_snapshot.cairo (JIT)	3.892 ± 0.049	3.824	3.969	1.04 ± 0.02
head dict_snapshot.cairo (AOT)	3.752 ± 0.054	3.694	3.834	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base factorial_2M.cairo (JIT)	4.162 ± 0.054	4.100	4.270	1.03 ± 0.02
base factorial_2M.cairo (AOT)	4.057 ± 0.028	4.034	4.108	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head factorial_2M.cairo (JIT)	4.179 ± 0.050	4.123	4.265	1.00 ± 0.02
head factorial_2M.cairo (AOT)	4.168 ± 0.044	4.069	4.209	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base fib_2M.cairo (JIT)	3.625 ± 0.014	3.609	3.651	1.00
base fib_2M.cairo (AOT)	3.644 ± 0.087	3.571	3.857	1.01 ± 0.02

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head fib_2M.cairo (JIT)	3.693 ± 0.052	3.627	3.798	1.00 ± 0.02
head fib_2M.cairo (AOT)	3.686 ± 0.035	3.605	3.734	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base linear_search.cairo (JIT)	3.939 ± 0.052	3.855	4.033	1.04 ± 0.02
base linear_search.cairo (AOT)	3.803 ± 0.072	3.692	3.915	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head linear_search.cairo (JIT)	3.862 ± 0.033	3.820	3.904	1.04 ± 0.01
head linear_search.cairo (AOT)	3.715 ± 0.020	3.687	3.746	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base logistic_map.cairo (JIT)	4.012 ± 0.030	3.969	4.061	1.03 ± 0.01
base logistic_map.cairo (AOT)	3.904 ± 0.048	3.829	3.962	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head logistic_map.cairo (JIT)	4.026 ± 0.044	3.952	4.101	1.05 ± 0.01
head logistic_map.cairo (AOT)	3.828 ± 0.025	3.800	3.871	1.00

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 99.10448% with 3 lines in your changes missing coverage. Please review.

Project coverage is 81.30%. Comparing base (3ed2f4b) to head (70d21d2).

Files with missing lines	Patch %	Lines
src/values.rs	70.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1126      +/-   ##
==========================================
+ Coverage   81.07%   81.30%   +0.22%     
==========================================
  Files         110      110              
  Lines       29312    29506     +194     
==========================================
+ Hits        23765    23990     +225     
+ Misses       5547     5516      -31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Benchmarking results

Benchmark for program dict_insert

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	20.064 ± 0.166	19.933	20.489	5.45 ± 0.05
cairo-native (embedded AOT)	3.678 ± 0.017	3.652	3.705	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	3.757 ± 0.014	3.738	3.780	1.02 ± 0.01

Benchmark for program dict_snapshot

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	5.737 ± 0.029	5.676	5.774	1.62 ± 0.01
cairo-native (embedded AOT)	3.546 ± 0.017	3.522	3.576	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	3.653 ± 0.014	3.633	3.675	1.03 ± 0.01

Benchmark for program factorial_2M

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	13.901 ± 0.063	13.786	13.985	3.51 ± 0.02
cairo-native (embedded AOT)	3.955 ± 0.016	3.934	3.986	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	4.006 ± 0.021	3.970	4.039	1.01 ± 0.01

Benchmark for program fib_2M

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	13.802 ± 0.079	13.697	13.974	3.96 ± 0.03
cairo-native (embedded AOT)	3.489 ± 0.013	3.461	3.507	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	3.531 ± 0.012	3.516	3.547	1.01 ± 0.01

Benchmark for program linear_search

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	5.740 ± 0.030	5.679	5.777	1.60 ± 0.01
cairo-native (embedded AOT)	3.597 ± 0.020	3.576	3.638	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	3.722 ± 0.013	3.700	3.741	1.03 ± 0.01

Benchmark for program logistic_map

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	5.645 ± 0.036	5.589	5.689	1.54 ± 0.01
cairo-native (embedded AOT)	3.665 ± 0.012	3.654	3.686	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	3.826 ± 0.014	3.806	3.849	1.04 ± 0.01