Comments (12)
This seems like a bug report (or feature request) for Miri, not this crate.
from aho-corasick.
Those already exist. The tests that are failing are specifically testing the optimized paths.
Miri tests don't succeed on x86 either.
Fixing this on my end probably means going through and adding a bunch of miri conditional compilation things to skip certain tests. I do that on some crates (like memchr and regex-automata), but it's pretty annoying to do. It also somewhat defeats the purpose, because the optimized paths are exactly where you want miri the most.
from aho-corasick.
Gotya, thank you for explaining that. That makes sense! I see the problem now.
from aho-corasick.
Well we could have non-optimized functions that would work on aarch64 miri, and also other architectures, but I can try over at Miri instead
from aho-corasick.
Miri tests don't succeed on x86 either.
Are you sure about that? Miri has decent x86_64 shims as of rather recently; with default flags the searcher fails to build (I didn't look into why yet, probably a runtime CPU feature detection fails) but this seems to work:
RUSTFLAGS=-Ctarget-cpu=x86-64-v2 cargo +nightly miri nextest run -j32
from aho-corasick.
with default flags the searcher fails to build
That's what I tried. I didn't dig into it.
from aho-corasick.
I'll reopen this for now since it looks like some further investigation might be warranted.
With that said, the original error message for aarch64 is pretty clear. In that case, miri likely needs support for neon.
from aho-corasick.
I agree with that completely. Miri doesn't have much if any neon support and would need a lot for this use case.
from aho-corasick.
Ah; the reason tests don't work on Miri + x86 by default is that aho-corasick tries to do runtime CPU feature detection (in teddy
) which in Miri doesn't get you the properties of the host, it gets you the properties of the configured target-cpu
, and the Rust x86_64 baseline according to rustc --print cfg -Ctarget-cpu=x86-64
only has sse and sse2.
Up to you if you think there's something more reasonable to do in this case I guess? I don't know if there's something more reasonable that Miri could do here; using anything other than the target baseline by default seems dubious for a cross-interpreter.
from aho-corasick.
I don't know if there's something more reasonable that Miri could do here; using anything other than the target baseline by default seems dubious for a cross-interpreter.
Would it be possible to spell this out in more detail? I totally trust what you're saying here, I just don't completely grok it. I think my underlying question is, why can't Miri let runtime CPU feature detection actually inspect the CPU features of the host? Like, what is the blocker? (I say this with curiosity and ignorance, not with combativeness.)
It is great that Miri can run with RUSTFLAGS=-Ctarget-cpu=x86-64-v2
though. That's a useful tip, thank you.
from aho-corasick.
I think one of the issues I ran into here is that I didn't want to just silently skip the tests if building the searcher failed. I wanted that to be loud. Otherwise, it's too easy for a test that should fail to just silently pass. I don't think I dug in too much at this point and kind of just ran out of steam. And I don't think I knew the tidbit about Miri not doing runtime CPU feature detection, so was perhaps confused. But now that I know that, I think I can rejigger the tests a bit to make this work more nicely.
from aho-corasick.
I think my underlying question is, why can't Miri let runtime CPU feature detection actually inspect the CPU features of the host? Like, what is the blocker?
Miri tries rather hard to be a portable cross-interpreter. I can run Miri on an M1 and ask it for --target=s390x-unknown-linux-gnu
. Without passing MIRIFLAGS=-Zmiri-disable-isolation
, every aspect of the program behavior should only depend on the target, not the host (and also be exactly the same run-to-run).
With isolation disabled is a slightly more interesting question; it might make sense to default to "whatever the host has", but then it's not entirely clear what to do when cross-interpreting because the CPU feature detection in stdarch depends on architecture and OS. We couldn't just have Miri directly run the standard library code for is_X_feature_detected
because everything gets cross-compiled for the target so that code can only possibly work if the target matches the host, including the OS. To at least solve the OS dimension, we could execute the feature detection code in the interpreter itself and return it from the macro by a shim. (and of course you can't pass through to the host for interpreting across architectures, and I wouldn't be surprised if we have targets that are the same architecture with different baseline CPU feature sets)
But then we have a UI problem where we need to figure out what people should do to give Miri access to the filesystem but not to runtime feature detection, so that we can have reproducible errors in programs that need to read a file but also need to do some SIMD. A lot of people globally disable isolation because so few real programs work with it, so it's really not clear what direction an "isolation includes CPU feature detection" flag should go.
I don't think there are any hard blockers, but passing through to the host is a sticky mess for a cross-interpreter.
from aho-corasick.
Related Issues (20)
- For Python wrapper, is it worth switching to new NFA, or sticking to DFA? HOT 2
- [Note] Investigating "transmute slower than intrinsics for vector unpacking" HOT 1
- Serializing pre-built automaton HOT 7
- RFC: review API before 1.0 release HOT 15
- Benchmarks results aren't clear HOT 11
- Default `anchored` value in a new `Input`. HOT 2
- AhoCorasick::memory_usage doctests fail on i686 (and possibly most 32-bit architectures) HOT 3
- nfa::contiguous::tests::swar test failure on s390x (and possibly all big-endian architectures) HOT 3
- Question: Suggestions for handling . wildcards within query patterns HOT 1
- Go wrapper for aho-corasick HOT 10
- A performance issue vs pyahocorasick HOT 3
- Async support HOT 3
- Document order of returned matches HOT 1
- `_mm256_loadu_si256` is failed to be inlined for ABI issues HOT 4
- issue with target_feature in Conditional Compilation Tags HOT 3
- Default automaton HOT 7
- Is it ok to replace memchr with stringzilla for better performance? HOT 1
- Make len() available on `Match` HOT 1
- question: comparisson with a regex union. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aho-corasick.