Current rules with count and other aggregations only define timeframe with the corresponding attribute. To make conditions easier to read and the syntax more consistent across all aggregation types, within is dropped.

https://github.com/Neo23x0/sigma/blob/39625208483cc2090c78aa6a10acf13e30efdbdb/rules/windows/sysmon/sysmon_rundll32_net_connections.yml#L19

https://en.wikipedia.org/wiki/Private_network
There are public IP address in this range 172.*. It should be
^172.1[6-9].
^172.2[0-9].
^172.3[0-1].
And missing loopback not sure if it's important.
^127.0.0.1

Minor issue with line piblic should be public

The schema defines condition as str.
Yet it is mentioned in the specs that

If multiple conditions are given, they are logically linked with OR.

without defining how could that be possible... the only example I found was (https://github.com/Neo23x0/sigma/blob/master/rules/network/net_susp_network_scan.yml).

So, either:

• A) Update the schema to mention that condition could be either a str or list,
• B) condition remains a str and fix the parser (sigmac?) to allow for multiple pipes/parentheses/ORs, or
• C) fix condition to be a list and update all rules accordingly (all existing rules will have a single-item list for their condition)

Sorry for being picky :D, working on something which will have a sigma rule parser and being consistent is a good base to build on...

P.S: Changing the above mentioned rule from

    condition:
        - selection | count(dst_port) by src_ip > 10
        - selection | count(dst_ip) by src_ip > 10

to
condition: ( selection | count(dst_port) by src_ip > 10 ) or ( selection | count(dst_ip) by src_ip > 10 )
broke sigmac, complaining about unmatched closing parentheses (maybe because it improperly parses after the pipe token)

P.P.S: I'll gladly update all rules to match whatever decision will be made.

Query, filter and aggregate with Powershell:

• Get-EventLog
• where for filtering
• Group-Object for aggregation
• Format-Table with fields from Sigma rule

If powershell runs "net user" then under my ElasticSearch/Kibana stack, the data is represented like so:

• parent_cmdline: "C:\WINDOWS\system32\net.exe"
• image: C:\Windows\System32\net1.exe
• cmdline: C:\WINDOWS\system32\net1 user

I think the following conditions provide more flexibility:

Image:
    - '*\net.exe'
    - '*\net1.exe'

CommandLine:
    - ' group*'
    - ' localgroup*'
    - ' user*'
    - ' view*'
    - ' share'
    - ' accounts*'
    - ' use*'

sigmac give the following error when using it to parse the rule mimikatz_inmemory_detection.yml.

[tagashy@manjaro シグマ]$ sigmac -t es-qs sigma_rule.yml 
Sigma parse error in sigma_rule.yml: Unexpected token [ Token: ID: 'dllload1' ] at 12 in aggregation expression

sigma_rule .yml is mimikatz_inmemory_detection.yml but if the content is needed it's the following :

title: Mimikatz In-Memory Detection
status: experimental
description: Detects certain DLL loads when Mimikatz gets executed
reference: https://securityriskadvisors.com/blog/post/detecting-in-memory-mimikatz/
logsource:
    product: sysmon
detection:
    selector:
        EventID: 7
        Image: 'C:\Windows\System32\rundll32.exe'
    dllload1:
        ImageLoaded: '*\vaultcli.dll'
    dllload2:
        ImageLoaded: '*\wlanapi.dll'
    exclusion:
        ImageLoaded:
            - 'ntdsapi.dll'
            - 'netapi32.dll'
            - 'imm32.dll'
            - 'samlib.dll'
            - 'combase.dll'
            - 'srvcli.dll'
            - 'shcore.dll'
            - 'ntasn1.dll'
            - 'cryptdll.dll'
            - 'logoncli.dll'
    timeframe: last 30s
    condition: selector | dllload1 and dllload2 and not exclusion
falsepositives:
    - unknown
level: medium

for my instalation i've installed it through :
pip install sigma && pip install sigmatools

Hello there,
I am creating a MISP module to export attributes/objects from MISP as sigma rules and I read your wiki to find out about the specification and so on.

At the moment I did tests to create rules and I end up with something valid for your parser, because it has the required fields, like in the following example:

title: Sigma rule test 
logsource: 
    product: MISP 
detection: 
    selection: 
        ip-src: 
            - 1.2.3.4 
    condition: selection

But instead of ip-src: 1.2.3.4, any other combination of type and value seems to be valid for the parser

I also saw there is a list for the mapping between Sigma and ATP, so I wonder if there is a list of Sigma types (or documentation about it) I should use to map misp types with ?

Splunk backend generates field=("val1" "val2" ...), should be field="val1" OR field="val2".

IN operator seems not appropriate as wildcards are used in values.

The aggregation functions are not supported in elasticsearch and kibana backend, is that because it is technically impossible to implement with those backends or it is just not implemented yet ?
Thank you

While reviewing some plaso stuff, I discovered
https://github.com/ForensicArtifacts/artifacts
"A free, community-sourced, machine-readable knowledge base of forensic artifacts that the world can use both as an information source and within other tools."
It's YAML based too and would be a nice way to reuse common definition path independently of platform.

It's already very populated like
https://github.com/ForensicArtifacts/artifacts/blob/master/definitions/linux.yaml
https://github.com/ForensicArtifacts/artifacts/blob/master/definitions/darwin.yaml
https://github.com/ForensicArtifacts/artifacts/blob/master/definitions/windows.yaml

Currently field name mappings are quite static which is not sufficient for SIEMs that normalize to different field names based on EventIDs or other criteria. A possible enhancement to cover this is to allow multiple target field names per source (Sigma) field name. Sigmac would then translate this into ORs. Example:

- fieldmappings
  SubjectAccountName:
    - caller_user
    - target_user
    - member

Would result in:

... caller_user=xxx OR target_user=xxx OR member=xxx ...

This syntax could be further enhanced by conditions:

- fieldmappings
  SubjectAccountName:
    EventID=123: caller_user
    EventID=234: target_user
    EventID=345: member

Generated queries are often unnecessarily complicated. An optimization step should be done on the parse tree. Ideas:

• Remove duplicate NOTs
• NOT ConditionNULLValue to ConditionNotNULLValue (some backends like es-qs generate improved statements)
• Parentheses around single conditions, e.g. (EventID:"5861") AND ...
• Parentheses around conditions of same type without any further conditions, e.g. (... AND ... AND ...)

Hi, really nice project.
Do you plan to support Graylog ?
Thanks in advance!

Hi, I wonder if anyone considered creating some official data dictionaries for some fields. The log source fields, for example. When the number of community created starts growing we'll see some challenges like rules referring to "Windows", "Microsoft Windows", "MS Windows", "windows-snare", etc. A place to get official dictionaries for fields like that would make it far easier to develop converters to proprietary formats.

I don't know much about YAML, so not sure about what are most appropriate ways to address this challenge. Do they do anything similar for YARA?

rulename prefix (product:service)

lnx_buffer_overflows.yml linux:unknown
lnx_shellshock.yml linux:unknown
lnx_shell_susp_activity.yml linux:unknown
net_susp_network_scan.yml unknown:unknown
proxy_exe_download_susp_tlds.yml unknown:unknown
proxy_powershell_ua.yml unknown:unknown
web_apache_segfault.yml apache:unknown
web_multiple_suspicious_resp_codes_single_source.yml unknown:unknown
web_webshell_keyword.yml unknown:unknown
win_alert_mimikatz_keywords.yml windows:unknown

also, this in case you are interested in a downstream development project for splunk

It looks like Sigma currently only supports field to value or field to pattern comparison.

This is an issue to track support for a field to field comparison.

An example here might be an output where the field values are used to compare each other such as:
not (field1 = field2)

It might be useful to date fields like date rule created or dated rule updated. That way, folks who use this down stream can run (potentially expensive) searches on old data with only new/updated rules.

We need a more explicit syntax for conditions that describe relations between event like this one. The syntax currently used can cause ambiguities in parsing of such conditions. Main requirement is the introduction of a keyword.

It would be nice to have a field allowing how to test rules be it with inline command or script (shell, bat, metasploit, empire, scapy...)
This way it would be easier to validate rule is always working and fitting the environment.
Eventually, it can be multiple test suites and false-positive test suites to ensure it doesn't match on some cases.
Clearly optional as it can demanding but a really nice to have if you want to put your platform in continuous integration mode:

Example draft syntax

title
status [optional]
description [optional]
author [optional]
[...]
testing:
  - shell: "nmap..."
     platform: unix
  - script: "test/rule1/test.ps1
     platform: win
  - script: "test/rule1/test-fp1.ps1
     platform: win
     falsepositive: yes

"not null" field values are not converted into an appropriate target expression but handled as string.

Extension of configuration by placeholders that provide replacement values. Example (with Elasticsearch Query String syntax):

[...]
placeholders:
  Workstations:
    - PC*
    - LAP*
  Servers: SRV*
[...]

Would replace:

• ComputerName: %Workstations% with ComputerName:("PC*" "LAP*")
• ComputerName: %Servers% with ComputerName:SRV*

Everything is ready for automatic testing with Travis, just need it activated in the main repository, which can only be done by a repo admin.

Hi,

I saw Florian's twitter post this week and thought I'd raise a ticket to discuss how using Splunk's data models could work: https://twitter.com/cyb3rops/status/992315782582427648?s=12

After giving it some thought, I'm now thinking it's best to update the existing backend but to have it produce SPL that leverages Splunk's data models and data model acceleration when it can. Something like the following:

1. Check that the config file associates a logsource with a data model.
2. If all the provided fields exist within the data model, then produce a query that uses the tstats command.
3. If not all the fields exist within the datamodel, then fallback to producing a query that uses the from command.
4. If the logsource isn't associated with a data model, then just output a regular search.

Doing the above would mean there'd need to be some way to associate a logsource to a data model within the config. A data model consists of a json schema that sigma could read in to see if the fields listed exist. The Common Information Model (CIM) is the one everyone should be using, but it's possible that some may have extended it or need to use custom data models, so it'd be good to make sigma account for this: https://splunkbase.splunk.com/app/1621/ (json is under default\data\models\ within the .tar.gz).

Some examples of what this might look like:

1. rules\proxy\proxy_powershell_ua.yml could be associated with the Web.Proxy data model and only uses fields within the data model, so it should produce:
   | tstats count from datamodel=Web where nodename=Web.Proxy (Web.http_user_agent="* WindowsPowerShell/*") by host
2. rules\windows\sysmon\sysmon_webshell_detection should be associated with the Application_State.Processes data model but not all fields from the sigma query are contained within this datamodel (EventID), so it should produce:
   | from datamodel:Application_State.Processes | search (EventID="1" (parent_process="*\\apache*" OR parent_process="*\\tomcat*" OR parent_process="*\\w3wp.exe" OR parent_process="*\\php-cgi.exe" OR parent_process="*\\nginx.exe" OR parent_process="*\\httpd.exe") (cmdline="whoami" OR cmdline="net user" OR cmdline="ping -n" OR cmdline="systeminfo"))
3. The final outcome is where the logsource hasn't been associated to a data model, which produces the same SPL that the current splunk backend does.

I started having a little bit of a tinker but haven't gone as far as implementing the above suggestion. Let me know what you think first.

Generate SQL queries for lnav.

An unsupported feature is required for this Sigma rule: Aggregations not implemented for this backend

Not sure what this means.

The backends module got quite big and confusing. Requires splitting in at least:

• Base classes
• Mixins
• Output classes
• Exceptions
• Backends

Do the split in a history preserving way.

In my experience, the windows event log EventID is represented as EventCode within Splunk. Might be worth considering outputting EventCode instead of EventID as part of the converter.

Examples of queries can be found here:
https://github.com/Microsoft/WindowsDefenderATP-Hunting-Queries

Would be useful to use the Sigma signatures against this.

I would love to see this support Graylog and their rest api:
http://docs.graylog.org/en/2.2/pages/configuration/rest_api.html

Expressions like cnt(foo) by bar are currently not handled precisely, as XPackWatcherBackend uses the hit count. This could be accomplished by aggregating the search result (value count metrics aggregation on foo nested in a terms bucket aggregation on bar).

the ruleset provided here is amazing and a invaluable resource for identifying new detections that need implemented in our environment. we have been mapping all of our detections to MITRE's ATT&CK matrix and it would be awesome if the sigma rule definitions already had this mapping.

thoughts?

• Empty/non-existing fields can be expressed with null (Converted by YAML parsing to None)
• Not empty/existing is expressed via condition not selection_name and selection used null syntax from above

It would be nice to create a sigma rule to monitor disabling of windows automatic defragmentation
https://securelist.com/apt-slingshot/84312/

Schema in the specs defines "reference" and "falsepositives" to be str; yet sometimes they're str, sometimes they are list in the sample rules.

If I may suggest going with "list" and update the rules accordingly, changing single strs to a list with one item for the sake of consistency (I'd gladly update all the rules and submit a PR), or if it will be decided that they could be either a str or list, then update the specs.

Windows service install event id is 4697. But you use 7045 in that article:
https://github.com/Neo23x0/sigma/blob/master/rules/windows/builtin/win_mal_service_installs.yml

There is no 7045 windows event id.
Reference:
https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4697

It would be helpful if there were a utility that allowed us to convert existing query catalogs in Splunk/ELK/etc... to Sigma notation so that a uniform documentation standard can be built and stored in a git repo.

In most rules, you use the logsource.product field to describe the operating system type and the logsource.service field to describe the log sourcetype.

With powershell, you use logsource.platform to describe the operating system type and logsource.product to describe the log sourcetype.

This seems inconsistent. Is platform going to be a thing or should the powershell rules be updated for consistency?

Hy guys!
How are you?
I have some questions about the system... sorry for my "inconvinient question"

First... How I integrate this solution with Ossec?
Second... This solution have any agent (on client side?)?

Tks guys!

Hi,

When I use the sigmac tool, I've an error syntax:

Traceback (most recent call last):
  File "tools/sigmac", line 27, in <module>
    import sigma.backends as backends
  File "sigma-master/tools/sigma/backends.py", line 681
    **min_doc_count
     ^
SyntaxError: invalid syntax

Host is debian os, and I installed sigma with pip3.

Backends which require an index pattern (e.g. Kibana) currently assume hard coded default which may not be appropriate in specific environments.

During the usage of sigma-language I got stuck describing cases when there is "NOT" and ("TRUE" and "TRUE") logic. I thought of suggested way of handling them as more like a workaround so I tried to add more readability to it. Also, the rules which are "united" by the common attack technique, but devided by different logsources of events, are tricky to combine into one rule. So, the main downsides of current realisation, as I see it, are:

1. Not obvious implementation of "NOT" directive.
2. The concept of putting "logsource" field on top of the rule isn't quite flexible - the events provided by various logs sources can't live "under the same roof".
3. Hard to filter the cases, when few fildes have to contain specific values simultaneously(!) and only then they are to be false positives.
   Here's the example which describes abusing bitstransfer technique:

title: BITS Jobs
description: 
    general: Detects usage of bitsadmin downloading a file
    mitre:
        id: T1197
        tactic:
          - Defense Evasion
          - Persistence  

detection:
    selection1:
        Logsource: 'Microsoft-Windows-Sysmon/Operational'
        EventID: 1
        Image: '*\bitsadmin.exe'

    selection2:
        Logsource: 'Microsoft-Windows-Bits-Client/Operational'
        EventID:
          - 3      
          - 59
          - 60
    
    selection3:
        Logsource: 'System'
        EventID: 7040
        Service: 'Background Intelligent Transfer Service'

    condition: 1 of selection*

level: medium

Presumably, the engineer who'd be analysing it shouldn't experience inconvenience - the rule is brief and readable. I don't see why we should devide it into pieces given the fact that each part is telling the same thing. Regarding the discomfort with current "not" in the condition filed:

title: 
description: 
    general: Running code in the context of another process may allow access to the process's memory, system/network resources, and possibly elevated privileges
    mitre:
        id: T1055 
        tactic:
          - Defense Evasion
          - Privilege Escalation

detection:
    selection1:
        Logsource: 'Microsoft-Windows-Sysmon/Operational'
        EventID: 8
        SourceImage: 
          - NOT '*\VBoxTray.exe'
        TargetImage: 
          - NOT '*\csrss.exe'
       #(StartFunction: NOT 'EtwpNotificationThread' AND SourceImage: NOT '*\\rundll32.exe')
    
    condition: 1 of selection*

level: medium

Here's when the problem pops up - I don't know how to describe the last line in the selection1 yet. Of course, we already can solve it in directive "condition", but what if we have a dozen selections, then my "condition" would be bulky.
Can we consider some of my thoughts? The appearance of sigma-rules would be more simplistic, I believe.

Thanks a lot for the contribution you're making with Sigma, I think it will solve problems for lots of people!

I am trying to integrate Sigma into Yeti (rule storage, validation, and exporting) and I was wondering if there was a way to easily parse a given rule (only for validation purposes). The code I have today is:

yaml_stream = cStringIO.StringIO(self.yaml)   # self.yaml is the string representation of the yaml rule
sigmaconfig = SigmaConfiguration()

try:
    SigmaCollectionParser(yaml_stream, sigmaconfig, None)
except (yaml.parser.ParserError, yaml.scanner.ScannerError) as e:
    raise IndicatorValidationError("YAML parsing error: {}".format(e))
except (SigmaParseError, SigmaCollectionParseError) as e:
    raise IndicatorValidationError("Sigma parsing error: {}".format(e))

The problem I have is using sigma (SigmaCollectionParser, etc.) from another app is a bit tricky since I haven't found any setup.py or pypy repo. Is there a cleaner way to "install" sigma besides cloning the repo and manually copying files?

Are there any plans to create an API for sigma so that other tools can interact with it?

Thanks!

Hi,

Just wanted to point you out to a research paper of JPCERT in regards to "Detecting Lateral Movement through Tracking Event Logs".
Windows Event Logs generated because of 44 typical tools and commands, used for lateral movement, were analysed.

I think it will be a great resource for creating Sigma rules for detecting lateral movement in Windows environments.

Here is the link to the article: http://blog.jpcert.or.jp/2017/06/1-ae0d.html
And here is the link to the PDF: https://www.jpcert.or.jp/english/pub/sr/20170612ac-ir_research_en.pdf

• Indentation
• Colorization

The condition statements any of them or 1 of them should combine all sub elements of detections with an OR.

Hello,

Any plans for suggested actions in the schema?
I know it's pretty broad and environment dependent but it could be an interesting optional field
for example

• retrieve executable on lnx_buffer_overflows and validate hash against VT/MHR
• retrieve process tree
• dump full memory
• kill process
• change network vlan or any other containment actions

Depending on implementation, it could be leveraged to automate part of the incident response with tools like ansible, puppet & co
Also LimaCharlie (https://github.com/refractionPOINT/limacharlie) is doing automated response based on python

kind of related to CoA mentionned on MISP project perspective issue.

Thanks a lot for this great contribution!

Generate OSSEC rules from sigmac

Relevant documentation:
OSSEC Documentation
Rules and Decoders
Rule Syntax

Inspired by #51

Hi,

just checked out the project on a RHEL system. Now I'm in the process of figuring out what packages and even what python version is recommended/required. This is a bit bothersome

Would be really helpful if you could make that information easily available. (Not specifically for RHEL, but generally). I would issue a pull request but I still haven't make it work yet.

cheers

wlet

Feedback from a MISP project perspective, the Sigma intermediate language for the signature creation for different SIEM is really a great idea.

We (@iglocska and @adulau) quickly reviewed it and we can propose two steps (on a @MISPProject perspective):

• Creating a specific attribute type in MISP called Sigma which can be used just like YARA or Snort rule types. This is an easy step but It might not fully cover the full interesting use-case of Sigma. When your repository will be public, we will add the attribute in the next release of MISP.

• As as second step creating rules from CoA (Course of Action) could be really nifty. We need some additional work in MISP to support the objects and especially the ability to make a list of chain events. This requires more work on our side especially to see how we could generate such generic search rules from a series of misp-objects. But it's definitely doable.

I hope this helps.

Don't hesitate to provide feedback or tell us if you need specific requirements in MISP to support the use of Sigma.

PS: Do you think that we could propose in the future some additional fields in Sigma? If we see a specific need in MISP communities?