[Bug]: BM25Retriever retrieval nodes, top_n indexs and scores not match about llama_index HOT 1 CLOSED

Hey @huanghao128! I'm here to help you with any bugs, questions, or contributions you have for the repository. Let me know how I can assist you!

The issue in the BM25Retriever class at line 92 of the file llama_index/retrievers/bm25/base.py can be fixed by modifying the loop to correctly match the retrieval nodes, top_n indexes, and scores. The suggested fix is correct. Here is the corrected code:

for ix in top_n:
    nodes.append(NodeWithScore(node=self._nodes[ix], score=float(scores[ix])))

This ensures that the scores are correctly matched with their corresponding nodes. The updated _retrieve method should look like this:

def _retrieve(self, query_bundle: QueryBundle) -> List[NodeWithScore]:
    if query_bundle.custom_embedding_strs or query_bundle.embedding:
        logger.warning("BM25Retriever does not support embeddings, skipping...")

    query = query_bundle.query_str
    tokenized_query = self._tokenizer(query)
    scores = self.bm25.get_scores(tokenized_query)

    top_n = scores.argsort()[::-1][: self._similarity_top_k]

    nodes: List[NodeWithScore] = []
    for ix in top_n:
        nodes.append(NodeWithScore(node=self._nodes[ix], score=float(scores[ix])))

    return nodes

This change ensures that the scores are correctly associated with the nodes they correspond to ^[1].

To continue talking to Dosu, mention @dosu.

from llama_index.