Search rule failed about fp-ngfw-smc-python HOT 25 CLOSED

Hello,

I did a quick try on my env and here is the results:

[IPv4Rule(name=Rule @262278.0)]

Which SMC version are you using?
Which SMC API version are you using?
Wich smc-python version are you using?

Thanks in advance.

/Greg

from fp-ngfw-smc-python.

Hi,

Thank you for your answer.
my SMC version is 6.8.5 [10932].
the SMC API version is 6.8 and the smc-python version is 21.3.1.
I don't think that the problem related to the firewall itself because the api commands are working in the cmd but when I tries to use this repository it doesn't work.
In the previous code I tried to clone all the files from github to my project. I have since made a few changes and today I tried to do the following steps:

1. I opened new project in Pycharm
2. I installed the smc-api package - pip install git+https://github.com/Forcepoint/fp-NGFW-SMC-python.git
3. then I got this message in the terminal: "WARNING: You are using pip version 21.1.2; however, version 21.3.1 is available"
   so I I upgraded the smc-version to the version 21.3.1.
4. I tried again to search a rule with this code:
   session.login(url=SMC_URL, api_key=API_KEY, verify=False, timeout=120, api_version=API_VERSION)
   p = FirewallPolicy("Vamsi")
   print(p.search_rule('@2187085.0'))
   session.logout()

the error I got:
Traceback (most recent call last):
File "C:\Users\smana\PycharmProjects\pythonProject1\main.py", line 5, in
from smc import session
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc_init_.py", line 3, in
from smc.api.session import SessionManager
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\api\session.py", line 24, in
from smc.base.model import ElementFactory
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 56, in
from smc.base.structs import NestedDict
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\structs.py", line 131, in
class NestedDict(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

it looks like there is a missing object in the code:(

Thank you again for your help,
Son

from fp-ngfw-smc-python.

ok thanks for your information.
It could be related to the content of your rule.

Would it be possible to somehow details more this rule?

/Greg.

from fp-ngfw-smc-python.

it should be something related to the rules because I can read other things from the SMC using the API (for example - the firewall policies names).
I have tried to search other rules using the previous code and it still didn't work.
just for an example: I have created new dummy firewall policy with the name "smana", then I added this rule to the policy:

I got the same errors like before.

in smc\base\structs.py there is some class with the name "class NestedDict(collections.MutableMapping)".
when I tried to search a rule without any changes in this class I got the error: (the error from the previous message)
Traceback (most recent call last):
File "C:\Users\smana\PycharmProjects\pythonProject1\main.py", line 5, in
from smc import session
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc_init_.py", line 3, in
from smc.api.session import SessionManager
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\api\session.py", line 24, in
from smc.base.model import ElementFactory
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 56, in
from smc.base.structs import NestedDict
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\structs.py", line 131, in
class NestedDict(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

but when I removed "collections.MutableMapping" from the class NestedDict, i.e. I changed the class name to be "class NestedDict()", I got the error from the first message I have posted here:
Traceback (most recent call last):
File "C:\Users\smana\PycharmProjects\pythonProject1\main.py", line 43, in
print(p.search_rule('@2207871.0'))
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\policy\policy.py", line 110, in search_rule
result = self.make_request(resource="search_rule", params={"filter": search})
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\mixins.py", line 26, in make_request
kwargs.update(href=self.data.get_link(kwargs.pop("resource")))
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 413, in getattr
if "typeof" not in key and key in self.data:
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 413, in getattr
if "typeof" not in key and key in self.data:
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 413, in getattr
if "typeof" not in key and key in self.data:
[Previous line repeated 989 more times]
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\decorators.py", line 86, in get
value = obj.dict[self.func.name] = self.func(obj)
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 375, in data
return LoadElement(self.href)
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 99, in LoadElement
request = SMCRequest(href=href)
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\api\common.py", line 69, in init
for k, v in kwargs.items():
RecursionError: maximum recursion depth exceeded while calling a Python object

Thanks,
Son

from fp-ngfw-smc-python.

Could you give us also python version you are using ?
Missing object collections.MutableMapping is a part of python which is defined in Python3.x/Lib/typing.py:
MutableMapping = _alias(collections.abc.MutableMapping, (KT, VT))

Could you check also this definition exists in your python library ?

Thomas.

from fp-ngfw-smc-python.

Hi,
I'm using python3.10. I checked and this definition exists in my python library:

Thanks,
Son

from fp-ngfw-smc-python.

Ok, I get the same issue using python 3.10..

 File "C:\Users\tdevulder\PycharmProjects\smc-python\smc\examples\basic_requests.py", line 6, in <module>
    from smc import session
  File "C:\Users\tdevulder\PycharmProjects\smc-python\smc\__init__.py", line 3, in <module>
    from smc.api.session import SessionManager
  File "C:\Users\tdevulder\PycharmProjects\smc-python\smc\api\session.py", line 24, in <module>
    from smc.base.model import ElementFactory
  File "C:\Users\tdevulder\PycharmProjects\smc-python\smc\base\model.py", line 56, in <module>
    from smc.base.structs import NestedDict
  File "C:\Users\tdevulder\PycharmProjects\smc-python\smc\base\structs.py", line 131, in <module>
    class NestedDict(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

We have to fix this compatibility issue with python 3.10
I am going to open an issue in our GitHub Enterprise for that. I let you know..
Meanwhile, I suggest you to downgrade your python version to 3.8 if possible.

from fp-ngfw-smc-python.

You could also try to replace collections.MutableMapping by collections.abc.MutableMapping.
It should correct your issue.. but smc-python is not currently validated with python 3.10.. it will be soon.

from fp-ngfw-smc-python.

I have tried it last week, I think that I wrote about it here. after I did this change I got the error of maximum recursion depth:
Traceback (most recent call last):
File "C:\Users\smana\PycharmProjects\pythonProject1\main.py", line 43, in
print(p.search_rule('@2207871.0'))
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\policy\policy.py", line 110, in search_rule
result = self.make_request(resource="search_rule", params={"filter": search})
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\mixins.py", line 26, in make_request
kwargs.update(href=self.data.get_link(kwargs.pop("resource")))
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 413, in getattr
if "typeof" not in key and key in self.data:
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 413, in getattr
if "typeof" not in key and key in self.data:
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 413, in getattr
if "typeof" not in key and key in self.data:
[Previous line repeated 989 more times]
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\decorators.py", line 86, in get
value = obj.dict[self.func.name] = self.func(obj)
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 375, in data
return LoadElement(self.href)
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\base\model.py", line 99, in LoadElement
request = SMCRequest(href=href)
File "C:\Users\smana\PycharmProjects\pythonProject1\venv\lib\site-packages\smc\api\common.py", line 69, in init
for k, v in kwargs.items():
RecursionError: maximum recursion depth exceeded while calling a Python object

anyway, I will try to replace my python version - which version of python do you recommend using?

Thanks,
Son

from fp-ngfw-smc-python.

Ok, I thought you just removed the extension from NestedDict
but when I removed "collections.MutableMapping" from the class NestedDict, i.e. I changed the class name to be "class NestedDict()", I got the error from the first message I have posted here:

We are using mainly python 3.8 but we are going to validate/use also version 3.9 and 3.10

from fp-ngfw-smc-python.

ok thanks, I downgraded my python version to 3.8 and now it's working.
one more question - I didn't find it in the documentation, is there a way to find all the rules/groups/aliases that contain a specific host like "Where used" in the GUI environment?

thanks again for the help

from fp-ngfw-smc-python.

Example:

host = Host("Greg")
print(list(host.referenced_by))

Results:
[Group(name=GRoup Test), IPv4Rule(name=Rule @262278.2), Layer3Firewall(name=NGFW5)]

from fp-ngfw-smc-python.

thanks:), I see that there is a history attribute only for ipv4_nat_rules. Is there a way to get history for IPv4_access_rules as well?
for example:
for rule in p.fw_ipv4_access_rules.all():
print("--------------- history --------------------")
print(list(rule.history))
print("--------------- end of history --------------------")

from fp-ngfw-smc-python.

Well it is working for me and it is available for all rule types.

print("===========")
for rule in policy.fw_ipv4_access_rules.all():
    print(list(rule.history))

print("===========")
for rule in policy.fw_ipv4_nat_rules.all():
    print(list(rule.history))
print("===========")

Result:

===========
[1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1558935326944, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1559049586166, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1559049586168, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1637309544327, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1637309389884, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1558934893072, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
===========
[1559049586166, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1559049586171, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1559052188110, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1559541463253, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1559049586166, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1559049586175, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
[1558934893044, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None, True, False, False, 1558934893194, 'http://10.37.204.81:8082/6.8/elements/admin_user/2', None]
===========

from fp-ngfw-smc-python.

when I tried this code I got the error:

it happened with some other scripts from the examples in the repository.
I think that maybe there is still issue with the versions I'm using.
this is my smc-python version:

I tried to upgrade this version but I got message that there isn't more up to date version.

from fp-ngfw-smc-python.

mm you are using very old version :)

Please uninstall it and then install:

• https://github.com/Forcepoint/fp-NGFW-SMC-python

• pip install fp-NGFW-SMC-python
  OR

• search fp-NGFW-SMC-python in Pycharm package (it is the same :))

from fp-ngfw-smc-python.

I have installed this package before, it can also be seen in the previous image in the third row.
the problem is that without downloading the smc-python 0.6.2 package - pycharm doesn't recognize the objects in this repository.
When I'm only use fp-NGFW-SMC-python package I get syntax errors:

from fp-ngfw-smc-python.

Well it looks like more a problem with your env. Even if package name has been changed the content remains the same.
Did you try to invalidate cache and restart PyCharm?

from fp-ngfw-smc-python.

yes, I have tried this and it didn't work but today I try to delete PyCharm and reinstall and now it's ok.
thank you very much and sorry for the hassle, I will close the issue.
just one more last question - given a particular host what is the simplest way to check behind which firewall it's routed using the API?

from fp-ngfw-smc-python.

Hey no problem I'm glad to help.

For you last question:
No there is no way to query it like in the GUI. But very good point. We'll try to see how his can be done. Of course it will supported only for new SMC versions. Could you open a dedicated issue so we can have a look at this please?

By this time I may have few lines that can help you (at least I hope)

engine = Engine("Algiers")
my_host = Host("Atlanta_host")
"""
Routing has 3 level

Example:
Engine 
    Interface 0
        102.168.0.1                     <--- level network
            Router 102.168.0.254        <--- level gateway
                Network 10.1.1.0/24     <--- level any
"""
engine_routing = list(engine.routing.all())
for routing_entry_net in list(engine_routing):
    for routing_entry_gw in list(routing_entry_net.all()):
        for routing_entry_any in list(routing_entry_gw.all()):
            # print(routing_entry_any.all())
            for entry in list(routing_entry_any.all()):
                if str(entry).__contains__(my_host.name):
                    print("Host {} has been found in routing".format(my_host.name))

from fp-ngfw-smc-python.

Hi,
sorry for the delay, just saw your message.
thanks for the help I will try this, I think these lines will help me with the task.
in the same context - I see that there is a use of networks in these lines so I need to ask something about this.
I an trying to make my program efficient as possible so I'm searching for a way to find all the networks contains some particular host without checking in a loop all the networks that exist in the smc.
do you know such a way using the API?

Thanks

from fp-ngfw-smc-python.

Hi,

As I said it is kind of workaround. In the GUI you can query route but it is not possible in API yet. We could implement it. But that will require time and not sure in which version it will be available.

Can you try the feature in the GUI and let me know if it does exactly what you are expecting. If yes then I'll open an enhancement internally in order to developp it.

from fp-ngfw-smc-python.

Fix included in smc-python 1.0.13

from fp-ngfw-smc-python.

Hi, thank you vey much!
do you mean to the route query or to my question about the option to get all the networks in the firewall that contain some host? what are the names of the commands in the API?

from fp-ngfw-smc-python.

We did not implement the query route for now but soon :)

from fp-ngfw-smc-python.