Comments (7)
@jburns12, this is currently an open issue in the taxii2-client
repo. oasis-open/cti-taxii-client#50
from cti.
Hi @leveled - hopefully I can help you out with this. I'm having a hard time recreating the issue, but I can think of a few things that might be going on. If you don't mind answering a couple questions that would help troubleshooting a bit:
-
Do you have the latest version of the ATT&CK content? I ask because originally (right when the October 2018 content was released) I realized there was an improperly formatted attack-pattern in enterprise-attack. I committed a fix to this repo a few hours later, but was thinking that maybe if that attack-pattern was in your content it could cause an issue.
-
If that's not the problem, can you tell me what version of the stix2 library you're using?
Thanks!
from cti.
I created a clean folder with a Python 3 virtual environment and was still getting the error
(venv)➜ attack_update_test ls
ATTACK venv
(venv)➜ attack_update_test python3
Python 3.6.5 (default, Mar 31 2018, 13:09:02)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import stix2
>>> from stix2 import FileSystemStore
>>> fs = FileSystemStore('./ATTACK')
>>> from stix2 import Filter
>>> filt = Filter('type', '=', 'attack-pattern')
>>> techniques = fs.query([filt])
>>> print(techniques)
[]
>>> quit()
(venv)➜ attack_update_test pip3 show stix2
---
Metadata-Version: 2.1
Name: stix2
Version: 1.0.2
Summary: Produce and consume STIX 2 JSON content
Home-page: https://github.com/oasis-open/cti-python-stix2
Author: OASIS Cyber Threat Intelligence Technical Committee
Author-email: [email protected]
License: BSD
Location: /Users/adi/github/attack_update_test/venv/lib/python3.6/site-packages
Requires: pytz, simplejson, stix2-patterns, six, python-dateutil, requests
Just in case it was a strange MacOS bug I checked it on a Kali VM I had running again, and got the same issue
(venv) root@kali:~/attack_update# python3
Python 3.6.6 (default, Jun 27 2018, 14:44:17)
[GCC 8.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import stix2
>>> from stix2 import FileSystemStore
>>> fs = FileSystemStore('./ATTACK')
>>> from stix2 import Filter
>>> filt = Filter('type', '=', 'attack-pattern')
>>> techniques = fs.query([filt])
>>> print(techniques)
[]
>>>
I got the same error when I tried this in a Python 2 environment. Filters on other fields seem to work just fine. Perhaps you can tell me how you managed to not get the error and I can try to replicate that.
from cti.
Hi @leveled, it took a while to figure out the problem you were experiencing.
I was looking back at the commit history for stix2
and the FileSystemSource
query code has not changed in a while. Now, this data source makes an assumption that you have a parent directory followed by directories that have the type as the name (example below). Similar to what you see in this repo. In the past, only ATTACK (or what is now attack-enterprise) data was hosted here, but now there are other data sources hosted here as well.
The FileSystemSource will recursively go deeper to find data, but another thing related to the funny results you were obtaining was that a "type" Filter is treated differently by the FileSystemSource. It essentially serves as a hint to search in a specific directory. Since below cti/ there is no attack-pattern/ directory it returns an empty list. On the hand, Filter("name", "=", ".bash_profile and .bashrc") will search naively through all directories.
parent/
attack-pattern/
campaign/
... (so on for each type including custom objects)
I believe you encountered this problem because of the USAGE.md (which needs to be updated). Now to overcome this you have various options:
Case 1: I only need to query the enterprise-attack sources
import stix2
enterprise_attack_ds = stix2.FileSystemStore("./cti/enterprise-attack")
f1 = stix2.Filter("name", "=", ".bash_profile and .bashrc")
f2 = stix2.Filter("type", "=", "attack-pattern")
print(len(enterprise_attack_ds.query(f1))) # Expect 1
print(len(enterprise_attack_ds.query(f2))) # Expect 223
Case 2: I need to query multiple data sources (e.g., mobile, enterprise, pre) at the same time.
import stix2
enterprise_attack_ds = stix2.FileSystemSource("./cti/enterprise-attack")
pre_attack_ds = stix2.FileSystemSource("./cti/pre-attack")
mobile_attack_ds = stix2.FileSystemSource("./cti/mobile-attack")
composite_ds = stix2.CompositeDataSource()
composite_ds.add_data_sources([enterprise_attack_ds, pre_attack_ds, mobile_attack_ds])
f1 = stix2.Filter("name", "=", ".bash_profile and .bashrc")
f2 = stix2.Filter("type", "=", "attack-pattern")
print(len(composite_ds.query(f1))) # Expect 1
print(len(composite_ds.query(f2))) # Expect 478
Hope this helps!
from cti.
Hey @emmanvg, that cleared up my issue with using a local datastore but I'm also having trouble with the MITRE hosted TAXII server. I'm not sure if I should bring this up in a separate issue but I'll detail the problem below as the solution may be similar.
I'm unable to use the documentation listed on this MITRE blog post to do any sorts of queries via the TAXII server.
The collections appear to be retrieved correctly
>>> from stix2 import TAXIICollectionSource
>>> from taxii2client import Server
>>> server = Server("https://cti-taxii.mitre.org/taxii/")
>>> api_root = server.api_roots[0]
>>> for collection in api_root.collections:
... print(collection.title + ": "+ collection.id)
...
Enterprise ATT&CK: 95ecc380-afe9-11e4-9b6c-751b66dd541e
PRE-ATT&CK: 062767bd-02d2-4b72-84ba-56caef0f8658
Mobile ATT&CK: 2f669986-b40b-4423-b720-4396ca6a462b
However I'm unable to perform any queries on the TAXIICollectionSource sourced from MITRE
>>> collection = Collection("https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e")
>>> tc_source = TAXIICollectionSource(collection)
>>> f1 = stix2.Filter("type","=","attack-pattern")
>>> tc_source.query(f1)
[]
>>> f2 = stix2.Filter("name","=",".bash_profile and .bashrc")
>>> tc_source.query(f2)
[]```
from cti.
Hi @leveled - I played around with this a bit and realized the issue is adding a slash at the end of the argument to the Collection API.
For example:
collection = Collection("https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/")
@emmanvg - is this expected behavior?
from cti.
Alright that seems to work, thanks for the help!
from cti.
Related Issues (20)
- Mitre Taxii Service Throwing 502 Errors
- Some revoked attack pattern miss the revoked-by relation in mobile domain HOT 2
- x_mitre_domains field for x-mitre-matrices populated only for ics
- Microsoft Defender Detection HOT 4
- [T1059.009] Cloud API - Typo in source name HOT 2
- v13.0 bundle ids match in both mitre/cti and mitre-attack/attack-stix-data, but content is different
- x_mitre_data_sources missing for Mobile ATT&CK attack-patterns HOT 2
- ICS platform information
- Some relationship missing when v12, v13 release HOT 2
- Alias of APT37 has a typo HOT 1
- The CAPEC dataset is not updated with the one available on capec.mitre.org
- Missing Some Records in 'Data Sources' HOT 1
- Request for ATT&CK version to be added to objects
- ATT&CK attack-patterns no longer have external_references to CAPEC HOT 1
- Bug: All MITRE ATT&CK ICS Techniques have "x_mitre_platforms": [ "None" ] HOT 2
- Certificate Expired
- cti-taxii.mitre.org timing out since Saturday, March 2, 2024 HOT 1
- https://cti-taxii.mitre.org seems to be down HOT 4
- Bad URL for "Dell PSP ZeuS" in "T1001.101 Junk Data" in the 11.0, 12.0, 13.0, and 14.1 MITRE ATT&CK Frameworks HOT 1
- Taxii Server seems to be timing out HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cti.