JSON indexing off about ripgrep HOT 4 CLOSED

The start and end fields represent byte offsets (see the format doc), and that particular line starts with "Reagents—The αIIbβ3", which contains 3 characters that are encoded on more than 1 byte: "—" takes 3 bytes, "α" and "β" take 2, which use 4 more bytes than if they were encoded on a single byte each, hence the difference you see.

from ripgrep.

The output you've shown is correct. Here is a verification:

$ rg --json "monoclonal antibody" PMC1395485.txt | rg '"type":"match"' | jq '.data.line_number, .data.submatches[].start, .data.submatches[].end'
52
121
140
152
611
630

$ head -n52 PMC1395485.txt | tail -n1 | dd ibs=1 skip=121 count=19 2> /dev/null
monoclonal antibody%

$ head -n152 PMC1395485.txt | tail -n1 | dd ibs=1 skip=611 count=19 2> /dev/null
monoclonal antibody%

Please read the JSON format docs closely: https://docs.rs/grep-printer/latest/grep_printer/struct.JSON.html

You'll note that the offsets are very specifically defined as byte offsets.

So when you use those offsets in some context, you need to make sure they are being interpreted as byte offsets. They may not be. Hint: Since you didn't provide any code and I thus cannot be sure, offsets for Python strings are codepoint offsets.

from ripgrep.

Wonderful. thanks guys. When in doubt, RTFM. Sorry for opening an issue!

from ripgrep.

Hi,

Thanks as always for such a great tool. I have noticed inconsistencies in the indexing of matches when using json mode. Sometimes the indexing is as expected, but sometimes the characters are a few characters off from the start of the line when the text field is loaded as a string in python. If this isn't a bug and there's a reason for the discrepancy, please let me know!

All best,
B

from ripgrep.