valitydev/SigmaHQ
14
0
Fork 0
mirror of https://github.com/valitydev/SigmaHQ.git synced 2024-11-08 10:13:57 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,673 Commits 20 Branches 25 Tags 21 MiB
4a775650a2
Commit Graph

404 Commits

Author SHA1 Message Date
Thomas Patzke
690807c846 Sigma tools release 0.8 2019-02-28 09:08:22 +01:00
Thomas Patzke
c922f7d73f Merge branch 'master' into project-1 2019-02-26 00:24:46 +01:00
christophetd
1a6faf385c Add HTTP POST alert type to the Elastalert backend 2019-02-23 14:12:14 +01:00
christophetd
3a7160d52b Accept backend options from a configuration file (closes #213) 2019-02-23 13:20:20 +01:00
Thomas Patzke
9ef314486e Grep backend escapes + 2019-02-19 14:49:06 +01:00
Florian Roth
004497075d fix: spark source config bug 2019-02-12 23:27:38 +01:00
Thomas Patzke
01dfc23a26
Merge pull request #234 from juju4/devel-sumo 
Sumologic support update
 2019-02-09 23:54:23 +01:00
Thomas Patzke
5866d8eb71
Merge pull request #238 from sisecbe/patch-1 
Adapt count function when aggfield not present
 2019-02-09 23:38:20 +01:00
juju4
4429d7564f remove 'escape' of '_' - not needed 2019-02-09 12:57:43 -05:00
juju4
a815b7eb9b add custom cleanValue function for wildcards in keyvalue: OK with lists, NOK with string 2019-02-09 12:57:07 -05:00
neu5ron
046510f021 updated HELK Destination IP name 2019-02-05 13:11:06 -05:00
sisecbe
5d94b9f0bc
Changed stats to eventstats 
Changed 'stats' to 'eventstats' when using aggregation, this keeps the original data of the event in the result.
 2019-02-05 17:36:46 +01:00
sisecbe
2f5eb08b41
Adapt count function when aggfield not present 
When no field is present, use "count" , when field is present use "dc(field)". As described in the Sigma specifications.
Splunk throws errors when using "count()" with empy fields. use "count" instead.
 2019-02-05 15:44:05 +01:00
Florian Roth
a276d3083d DHCP log source in sigmac configs 2019-02-05 14:35:23 +01:00
juju4
7d159fb980 sumologic backend: review with inspiration from arcsight 2019-02-03 12:53:58 -05:00
Thomas Patzke
6215a694a8 Remove escaping from '\\*' in es-dsl backend 2019-02-02 23:51:11 +01:00
Thomas Patzke
8a0784ad33 Fixed escaping of \\* 2019-02-02 00:18:58 +01:00
Thomas Patzke
516bfc88ff Added rule: RDP login from localhost 2019-01-28 22:43:22 +01:00
Thomas Patzke
3eaf83cf5a Improved configurations 
Added Security/4688 field mappings
 2019-01-16 23:37:18 +01:00
Thomas Patzke
ba64f485ac Added generic Windows audit log configuration 2019-01-16 22:41:42 +01:00
Thomas Patzke
4bc4c94a91 sigma2genericsigma: preserve dict order 2019-01-16 22:37:32 +01:00
Thomas Patzke
2fd88c837d Added generic sigma rule support to WDATP backend 
* Process creation rules
 2019-01-14 23:54:05 +01:00
Thomas Patzke
4e83bfeb16 Fixed merge bugs 2019-01-14 22:54:26 +01:00
Thomas Patzke
a9cf14438c Merge branch 'master' into project-1 2019-01-14 22:36:15 +01:00
Thomas Patzke
8336b47530 Merge branch 'master' of https://github.com/Neo23x0/sigma 2019-01-14 22:12:37 +01:00
Thomas Patzke
cc4b806b94 Sigma tools release 0.7.1 2019-01-14 00:26:03 +01:00
Thomas Patzke
5cba0b9946
Merge pull request #223 from m0jtaba/master 
extending the qradar backend to allow for timeframe query
 2019-01-13 23:55:55 +01:00
Thomas Patzke
7634128143 Generate list of converted file in conversion to generic rules 2019-01-13 23:53:11 +01:00
Thomas Patzke
e585858128 Optimization in conversion to generic rules 
* only create necessary output files in directory output mode
* delete empty detections and empty detection sections
* Merge equal documents
* Merge reduced collections into one YAML document in common case
 2019-01-13 23:45:11 +01:00
Mo Amiri
aa37ef2559 extending the qradar backend to allow for timeframe query 2019-01-11 03:33:49 +00:00
Adrien Vergé
44f18db80d Fix YAML errors reported by yamllint 
Especially the config for ArcSight, that was invalid:

    tools/config/arcsight.yml
      89:5      error    duplication of key "product" in mapping  (key-duplicates)
      90:5      error    duplication of key "conditions" in mapping  (key-duplicates)

    rules/windows/builtin/win_susp_commands_recon_activity.yml
      10:9      error    too many spaces after colon  (colons)
 2019-01-10 09:51:39 +01:00
Thomas Patzke
9f56b9e99b Output all YAML documents if one changed 
Some Sigma rule collections contain YAML documents that reduce to almost
nothing because they only contain EventID definitions. Previous behavior
would filter the part with the remaining selection.
 2019-01-08 23:27:16 +01:00
Thomas Patzke
bf9a567afd Fixed issues in converter 2019-01-06 23:57:09 +01:00
Thomas Patzke
faeaf1dfef Added first version of generic sigma rules conversion tool 2019-01-06 23:46:23 +01:00
Thomas Patzke
73b0c3a25b Fixed wildcard issue for es-dsl backend 
Moved field mapping code into mixin shared by es-qs and es-dsl.
 2018-12-21 14:10:45 +01:00
Thomas Patzke
75c7d65240
Merge pull request #211 from Cyb3rWard0g/master 
Field-Index Mapping File & SIGMA Rules Field names fix
 2018-12-19 00:38:06 +01:00
Thomas Patzke
ffd43823cf Fixed wildcard issue in es-qs backend and depending 
See GitHub issue #194. Fix for es-dsl is pending.
 2018-12-19 00:33:12 +01:00
Roberto Rodriguez
a0486edeea Field-Index Mapping File & SIGMA Rules Field names fix 
+ Updated HELK field-index mapping file
+ After going through all the fields with 'fieldlist' output, I found a few rules that fixed.
 2018-12-11 09:27:26 +03:00
Thomas Patzke
68866433e8 Merge branch 'juju4-devel-sumo' 2018-12-10 22:37:58 +01:00
Thomas Patzke
4175d0cdd5 Fixed config and added index field 
* Added index field _index to backend implementation
* Fixed index values in config
 2018-12-10 22:37:39 +01:00
Roberto Rodriguez
93d1d700d4 Merge remote-tracking branch 'upstream/master' 2018-12-10 07:04:30 +03:00
juju4
1f707cb37c Adding Sumologic backend 2018-12-09 17:55:51 -05:00
Thomas Patzke
2091c90538 Fixed ElastAlert *_key options 
* Always use .keyword field instead of analyzed one
* Fixed 'null' value if group field was not set
 2018-12-09 22:33:23 +01:00
Roberto Rodriguez
8c577a329f Improve Rule & Updated HELK SIGMA Standardization Config 
Rule should be focusing on the 'process_command_line' field and not just on any value of any event generated by powershell.exe.

SIGMA HELK standardization config updated to match latest HELK Common Information Model
 2018-12-08 11:30:21 +03:00
Thomas Patzke
246ad7c59a Revert "Fixed wildcards in es-qs backend" 
This reverts commit 49d464f979.

The partial fix for issue #194 broke the generation of many other rules,
see #203.
 2018-12-05 09:07:07 +01:00
Thomas Patzke
f9d9d653dc
Merge pull request #199 from sisecbe/patch-1 
Distinct count in aggragation function
 2018-12-04 23:42:16 +01:00
Florian Roth
2bf0170956
Merge pull request #202 from tuckner/master 
Fixed backslash escape
 2018-12-03 22:22:53 +01:00
tuckner
2c5c92ab0a fixed backslash escape 2018-12-03 15:09:29 -06:00
Thomas Patzke
0a5caae5df Merge branch 'master' of https://github.com/lsoumille/sigma into lsoumille-master 2018-11-28 23:53:15 +01:00
Florian Roth
99e0a4defb fix: SPARK config duplicate identifier 2018-11-27 14:05:13 +01:00
lsoumille
50c74b94bc add elastalert backend support 2018-11-23 20:39:15 +01:00
sisecbe
c848c473a3
Error when empty fields attribute 2018-11-23 15:37:42 +01:00
sisecbe
31eae25756
Indentation error 2018-11-23 15:20:17 +01:00
sisecbe
e43909678e
Added the fields attribute parser 
Make a table with the fields present in the fields attribute
 2018-11-23 15:11:12 +01:00
sisecbe
c2eb87133d
Distinct count in aggragation function 
Added dc() instead of count() when group-by field is present. Because count() doesn't do a distinct count in Splunk. Must be the dc() function instead.
 2018-11-23 15:04:08 +01:00
Thomas Patzke
aa1a953a65 Moved node dumping code to generic location 2018-11-21 23:22:38 +01:00
Thomas Patzke
26d888aec3 Removed "not null" handling code 
Feature was removed some time ago.
 2018-11-21 22:56:48 +01:00
Thomas Patzke
9e28669c33 Backend es-qs return quotes on empty or whitespace-only string 2018-11-21 22:29:12 +01:00
Thomas Patzke
49d464f979 Fixed wildcards in es-qs backend 2018-11-20 23:23:54 +01:00
Thomas Patzke
396a030ed1 Removed duplicate code 2018-11-07 22:52:12 +01:00
Thomas Patzke
116a0e9f03 Merge branch 'master' of https://github.com/tuckner/sigma into tuckner-master 2018-11-07 22:27:41 +01:00
Thomas Patzke
5053cc4e95 Fixed optimizing of not conditions with subexpressions 
Optimization pass traversal is cut at ConditionNOT nodes.
 2018-11-07 13:54:45 +01:00
Thomas Patzke
a88b1e81ec Optimizer debugging code cleanup 
* Removed commented debugging code
* Output to stdin
* Coverage exception for _dumpNode
 2018-11-07 13:49:08 +01:00
Thomas Patzke
42ed8acec9 Improved test coverage 
* Adding tests
* Removal of coverage measurement for debugging code
 2018-11-04 23:28:40 +01:00
Thomas Patzke
418f8d10a3 Wrap conditions generated by mappings into sub-expression 2018-11-04 23:00:04 +01:00
Thomas Patzke
0e4842962b Added tests 2018-11-04 22:16:20 +01:00
tuckner
ca6ba4a85b Added NetWitness backend and tests 2018-10-31 14:24:14 -05:00
tuckner
26f73d60fa Added NetWitness backend and tests 2018-10-31 14:07:59 -05:00
Thomas Patzke
eacfaa7460 Check for forbidden null values in list items in Splunk backend 2018-10-27 01:07:03 +02:00
Thomas Patzke
423a73efd5 Dropped .py suffix 2018-10-22 23:02:05 +02:00
Thomas Patzke
b2d6d73034 Added requirements 2018-10-22 22:43:59 +02:00
Thomas Patzke
16e3838a90 Renamed script 2018-10-19 21:23:33 +02:00
Thomas Patzke
6b14930302 Recursive path traversal 2018-10-19 21:21:33 +02:00
Thomas Patzke
67b416379f Improved import of multiple rules 2018-10-19 19:53:00 +02:00
Thomas Patzke
0cc8b77307 Merge branch 'master' of https://github.com/pivotforensics/sigma into pivotforensics-master 2018-10-18 15:56:26 +02:00
ntim
e501c4a5b9 Added additional output type 'json' to the xpack-watcher backend which prints each watcher as compress json, one watcher per line 2018-10-17 10:38:56 +02:00
Thomas Patzke
265ce115a0 Fixed conditional field mapping usage in mapping chains 2018-10-16 13:57:51 +02:00
Thomas Patzke
a61b3d352a Added test cases 
* Generic log sources
* Splunk index queries
 2018-10-15 15:24:18 +02:00
Michael H
5b33713ef8 Quick fix for string formatting bug 2018-10-13 20:21:37 -05:00
Michael H
38ec257f7e Re-doing LogName formatting 2018-10-13 20:18:57 -05:00
Michael H
9f48265eb1 Adding re.sub for LogName that accounts for expression grouping 2018-10-13 20:09:54 -05:00
Michael H
7e184f01c6 Removing invalid fieldmapping 2018-10-13 19:53:39 -05:00
Michael H
bbb67fbba4 Adding support for reading sigma rule from stdin in sigmac 2018-10-07 10:11:47 -05:00
Michael H
aabaa0257b Merge branch 'master' of https://github.com/Neo23x0/sigma 2018-10-06 20:12:15 -05:00
Michael H
4b85a34b34 Added CSV option to powershell backend 2018-10-06 20:08:20 -05:00
Thomas Patzke
e28bc35cad Apply field mappings in generation of log source condition 2018-10-06 23:38:35 +02:00
Daniel Roethlisberger
fc45df144c Improve the comments on the optimizer 2018-10-03 13:44:03 +02:00
Daniel Roethlisberger
87aa1b5521 Move optimizer to sigma.parser.condition to enable it for all backends 2018-10-03 00:24:31 +02:00
Daniel Roethlisberger
cd3661b60c Fix optimization of NOT corner cases 2018-10-02 22:48:33 +02:00
Daniel Roethlisberger
bed88cf813 Make uniq work for lists within definitions 2018-10-02 22:12:54 +02:00
Daniel Roethlisberger
7165128fa5 Remove None from AST - fixes None-related test failures 2018-10-02 21:44:37 +02:00
Daniel Roethlisberger
2242fc5ac8 Optimize the boolean expressions in the AST before generating output 
Add code optimizing the boolean expressions in the abstract syntax tree
before generating output using the backend.

The main idea behind optimizing the AST is that less repeated terms is
generally better for backend performance.  This is especially relevant
to backends that do not perform any query language optimization down
the road, such as those that generate code.

The following optimizations are currently performed:

-   Removal of empty OR(), AND()
-   OR(X), AND(X)                 =>  X
-   OR(X, X, ...), AND(X, X, ...) =>  OR(X, ...), AND(X, ...)
-   OR(X, OR(Y))                  =>  OR(X, Y)
-   OR(AND(X, ...), AND(X, ...))  =>  AND(X, OR(AND(...), AND(...)))
-   NOT(NOT(X))                   =>  X

A common example for when these suboptimal rules actually occur in
practice is when a rule has multiple alternative detections that are
OR'ed together in the condition, and all of the detections include a
common element, such as the same EventID.

This implementation is not optimized for performance and will perform
poorly on very large expressions.
 2018-10-02 21:14:25 +02:00
Karneades
468af42de5 Add missing event id list handling in PowerShell backend 2018-09-29 14:43:28 +02:00
Karneades
c289484c5c Improve default field handling in PowerShell backend 2018-09-29 12:29:44 +02:00
Florian Roth
1c2431f33b
Merge pull request #169 from Karneades/fix-aggregation-exeption 
Add rule filename to "not implemented" exception output
 2018-09-26 11:50:25 +02:00
Karneades
c66b00356d Add initial version of PowerShell backend 
* Add PowerShell backend
* Add PowerShell config file

State: Work in progress :)

See https://github.com/Neo23x0/sigma/issues/94
 2018-09-23 21:41:48 +02:00
Karneades
fe6f4c7475 Add rule filename to exception output for unsupported aggregation 2018-09-23 19:12:50 +02:00
Thomas Patzke
1d12fc290c Added Winlogbeat configuration 2018-09-20 12:08:11 +02:00
Thomas Patzke
2fbf17ff34 Addition and resolution of field mapping chains explicitely checks for list 2018-09-13 16:22:29 +02:00
Thomas Patzke
41a8ef2fd9 Implemented resolve_fieldname in FieldMappingChain 2018-09-13 14:56:31 +02:00