[BUG] 多并发调用，偶尔出现因为embedding没加载而调用失败 about langchain-chatchat HOT 3 CLOSED

The error you're encountering, "AttributeError: 'NoneType' object has no attribute 'acquire'", indicates that the ThreadSafeFaiss object is not properly initialized at the time you're trying to acquire a lock on it. This problem arises when the load_vector_store method returns None instead of a valid ThreadSafeFaiss instance. To address this issue, ensure that the embeddings or vector stores are properly initialized and loaded before they are accessed. Here's a more focused approach to resolve the issue:

1. Check Initialization and Loading: Confirm that the vector stores are correctly initialized and loaded before any access attempts. This involves ensuring that the load_vector_store method always returns a fully initialized ThreadSafeFaiss instance.

2. Implement Error Handling and Logging: Add error handling and logging within the initialization and loading processes. This will help identify and troubleshoot issues more effectively.

3. Ensure Object Readiness with wait_for_loading: Before using the embeddings or vector store, utilize the wait_for_loading method to ensure the object is fully loaded and ready for use.

4. Review Thread Safety and Concurrency: Given the concurrent nature of your application, it's crucial to ensure that all operations on shared resources are thread-safe to avoid race conditions or deadlocks.

By following these steps, you should be able to mitigate the intermittent failures related to embeddings not being loaded properly when handling multiple concurrent requests with vllm acceleration.

To continue the conversation, mention @dosu.

from langchain-chatchat.

这个框架没做并发啊

from langchain-chatchat.

这个框架没做并发啊

有计划做并发处理吗？

from langchain-chatchat.