Custom Entity Detection Issue in Presidio Version 2 about presidio HOT 7 OPEN

@surajsonee can you please elaborate on how you did step 3? Was the yaml loaded and recognizers added? if you print the list of recognizers, are the ones from the yaml listed there?

from presidio.

sure!
Here is the yaml file which I'm using:
https://github.com/microsoft/presidio/blob/main/presidio-analyzer/conf/example_recognizers.yaml

1. Access the Presidio Container: Use the docker exec command to access the running Presidio container's shell. For example:
   sudo docker exec -it <container_id> bash

2. Navigate to Configuration Directory: Inside the container, navigate to the directory where Presidio's configuration files are stored. This is typically the /presidio/config/ directory.

3. Check Mounted YAML File: Verify that the custom entity YAML file is correctly mounted in the container's configuration directory. using ls command to list the files in the directory:

4. Access Presidio Configuration Directory: First, navigate to the directory where Presidio's configuration files are stored. which is /presidio/config/ directory within the Presidio container.

5. Review Configuration Files: Look for configuration files or scripts that are used to initialize Presidio's analyzer engine. These files often have names like config.yaml or similar.

content of config.yaml:
custom_entities:
yaml_path: /presidio/config/example_recognizers.yaml

6. Inspect Configuration Content: Open the configuration file using a text editor or command-line tools like cat or less. Look for sections or properties related to loading custom entity rules or YAML files.

7. Add Recognizers to Registry: Add the created recognizers to Presidio's recognizer registry.
   Here's a Python example demonstrating how to add recognizers after loading the YAML:

from presidio_analyzer import Pattern, PatternRecognizer, AnalyzerEngine
import yaml

# Load custom entity rules from YAML file
with open('example_recognizers.yaml', 'r') as yaml_file:
    custom_entity_rules = yaml.safe_load(yaml_file)

# Create an instance of AnalyzerEngine
analyzer = AnalyzerEngine()

# Iterate over each entity in the custom entity rules
for entity_name, entity_config in custom_entity_rules.items():
    patterns = entity_config.get('patterns', [])
    
    # Create a recognizer for each pattern defined for the entity
    for pattern_config in patterns:
        name = pattern_config.get('name')
        regex = pattern_config.get('regex')
        score = pattern_config.get('score', 0.8)  # Default score
        
        # Create a Pattern object
        pattern = Pattern(name=name, regex=regex, score=score)
        
        # Create a PatternRecognizer with the Pattern object
        recognizer = PatternRecognizer(supported_entity=entity_name, patterns=[pattern])
        
        # Add the recognizer to Presidio's recognizer registry
        analyzer.registry.add_recognizer(recognizer)

In this above example, example_recognizers.yaml is the YAML file containing the custom entity rules. The script reads this file, extracts the entity names and patterns, creates recognizers based on the extracted information, and adds them to Presidio's recognizer registry.

Please let me know where I'm doing wrong.

Thank you!

from presidio.

Hi, I'm not sure what's wrong, as you seem to add the recognizers the right way. Could it be that patterns are always empty?

BTW we have a method for adding recognizers from YAML: https://microsoft.github.io/presidio/analyzer/adding_recognizers/#reading-pattern-recognizers-from-yaml
Perhaps try to see if it makes any difference.

from presidio.

Thank you for the reference!
Could you please provide guidance on which files require modification to establish custom entity rules?

from presidio.

Sure. if you change the default configuration in app.py:

To something more similar to the tutorial:

yaml_file = "recognizers.yaml"
registry = RecognizerRegistry()
registry.load_predefined_recognizers()

registry.add_recognizers_from_yaml(yaml_file)

self.engine = AnalyzerEngine(registry=registry)

You should be able to load the yaml based recognizers into the analyzer engine, and these would be used in each call.

from presidio.