carlosted / pulldown-latex Goto Github PK

Hi!

When I run the benchmarks locally with cargo +nightly bench I get the following error:

     Running benches/primitive_parsing.rs (target/release/deps/primitive_parsing-b5eb76bd89111f91)

running 3 tests
test basic_macros      ... FAILED
test match_on_greek    ... bench:      22,519.05 ns/iter (+/- 172.66)
test subscript_torture ... bench:       5,119.92 ns/iter (+/- 34.83)

failures:

---- basic_macros stdout ----
thread 'main' panicked at src/parser.rs:490:9:
assertion failed: ptr_val > input_start && ptr_val <= input_start + self.input.len() as isize
stack backtrace:
   0:     0x55c0b15191e5 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h7ba491a968dfae1d
   1:     0x55c0b1541c5b - core::fmt::write::hb4f7da2a314ef133
   2:     0x55c0b15164ff - std::io::Write::write_fmt::hd2eeb4ab41a6ecfb
   3:     0x55c0b1518fbe - std::sys_common::backtrace::print::h301edb80f46c58f8
   4:     0x55c0b151aa29 - std::panicking::default_hook::{{closure}}::h08c68fc043d12d43
   5:     0x55c0b151a696 - std::panicking::default_hook::hdf65558cafad5039
   6:     0x55c0b14ea58a - test::test_main::{{closure}}::h84b9a93cb3d03ce5
   7:     0x55c0b151b09b - std::panicking::rust_panic_with_hook::h9576f09a7a058c94
   8:     0x55c0b151addb - std::panicking::begin_panic_handler::{{closure}}::hd8f56243f630c203
   9:     0x55c0b15196a9 - std::sys_common::backtrace::__rust_end_short_backtrace::h5bed6f770a09acf8
  10:     0x55c0b151ab47 - rust_begin_unwind
  11:     0x55c0b153ff73 - core::panicking::panic_fmt::h088f9f84f0369284
  12:     0x55c0b154001c - core::panicking::panic::h41df2c6d8f6b057e
  13:     0x55c0b14aa3fd - pulldown_latex::parser::InnerParser::parse::h8721fad33043229e
  14:     0x55c0b14a979e - <pulldown_latex::parser::Parser as core::iter::traits::iterator::Iterator>::next::h9bc2f4e031107f84
  15:     0x55c0b149300e - pulldown_latex::mathml::MathmlWriter<I,W>::write::hcdf616188e8e12e4
  16:     0x55c0b14988ed - test::bench::ns_iter_inner::hdd3c7c091f7ca674
  17:     0x55c0b1498ba2 - test::bench::Bencher::iter::h77512a90c3d6c2c6
  18:     0x55c0b1497c5c - core::ops::function::FnOnce::call_once::h8c2addb6848ce4e1
  19:     0x55c0b14ec063 - test::run_test::h6c655a930b404f9a
  20:     0x55c0b14cef92 - test::console::run_tests_console::h9ec2804fcaaab62d
  21:     0x55c0b14e99f9 - test::test_main::hce91ba976c2b0793
  22:     0x55c0b14ea7eb - test::test_main_static::hef2a26e347822b42
  23:     0x55c0b1497ab3 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc6ba23b7939ed81b
  24:     0x55c0b1497ac9 - std::rt::lang_start::{{closure}}::haefa08eb3526987e
  25:     0x55c0b150fb4d - std::rt::lang_start_internal::h978095d8c1ffa0ac
  26:     0x55c0b1498265 - main
  27:     0x7fb76bbe3088 - __libc_start_call_main
  28:     0x7fb76bbe314b - __libc_start_main_impl
  29:     0x55c0b1490005 - _start
  30:                0x0 - <unknown>


failures:
    basic_macros

test result: FAILED. 0 passed; 1 failed; 0 ignored; 2 measured; 0 filtered out; finished in 0.44s

error: bench failed, to rerun pass `--bench primitive_parsing`

Expected behavior: the benchmark passes successfully.

OS: Fedora 40, Linux 6.9
Rustc version: 1.80 Nightly
Library version: main branch, a200bea991e68e5a66170f1eb7f15a741df0fec3 commit

Hi!

I was interested in optimizing the library's performance even further. I evaluated Profile-Guided Optimization (PGO) on many projects - all the results are available at https://github.com/zamazan4ik/awesome-pgo . Since this compiler optimization works well in many places, especially different parsers, I decided to apply it to the project - here are my benchmark results.

Test environment

• Fedora 40
• Linux kernel 6.9.12
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.79
• pulldown-latex version: main branch on commit 62cbc23a48ab3828dfd70e177227e7d3c03bb042
• Disabled Turbo boost

Benchmark

For benchmark purposes, I use built-in into the project benchmarks. For PGO optimization I use the cargo-pgo tool. Release bench result I got with taskset -c 0 cargo bench command. The PGO training phase is done with taskset -c 0 cargo pgo bench, PGO optimization phase - with taskset -c 0 cargo pgo optimize bench.

taskset -c 0 is used for reducing the OS scheduler's influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/9180993e98d13046a4bcf798e554d0bc
• PGO optimized compared to Release: https://gist.github.com/zamazan4ik/aacc0ec8cb158a6a078c4ea1d32327e8
• (just for reference) PGO instrumented compared to Release: https://gist.github.com/zamazan4ik/56f459df333322e9180e827699c98eb5

According to the results, PGO measurably improves the library's performance in many cases.

Further steps

I can suggest the following action points:

• Perform more PGO benchmarks with other datasets (if you are interested enough). If it shows improvements - add a note to the documentation (the README file, I guess?) about possible improvements in the library's performance with PGO.
• Probably, you can try to get some insights about how the code can be optimized further based on the changes that the compiler performed with PGO. It can be done via analyzing flamegraphs before and after applying PGO to understand the difference or checking some assembly/LLVM IR differences before and after PGO. However, this job can be boring and time-consuming - and the compiler already does all the job automatically when PGO is used.

I would be happy to answer your questions about PGO.

P.S. I created the issue just because Discussions are disabled for the repository. It's just a benchmark report, not a bug or smth like that.

Here is what is planned for future releases:

• Version 0.2:
  Support for basic math environments, such as array, matrix, and align.

• Version 0.3: Version 0.2:
  Operator Event Rework:
  The Operator event should follow what The TeXBook says, rather than the weird conventions of MathML.

• Version 0.3:
  Macro expansion.

Currently, there seems to be no library in npm that converts latex to mathml core.

It would be great if we could provide Node.js binding or wasm-wasi to javascript binding!

I believe this can promote the popularity of mathml core.