valitydev/SigmaHQ
14
0
Fork 0
mirror of https://github.com/valitydev/SigmaHQ.git synced 2024-11-08 10:13:57 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,673 Commits 20 Branches 25 Tags 21 MiB
4a775650a2
Commit Graph

404 Commits

Author SHA1 Message Date
lsoumille
50c74b94bc add elastalert backend support 2018-11-23 20:39:15 +01:00
sisecbe
c848c473a3
Error when empty fields attribute 2018-11-23 15:37:42 +01:00
sisecbe
31eae25756
Indentation error 2018-11-23 15:20:17 +01:00
sisecbe
e43909678e
Added the fields attribute parser 
Make a table with the fields present in the fields attribute
 2018-11-23 15:11:12 +01:00
sisecbe
c2eb87133d
Distinct count in aggragation function 
Added dc() instead of count() when group-by field is present. Because count() doesn't do a distinct count in Splunk. Must be the dc() function instead.
 2018-11-23 15:04:08 +01:00
Thomas Patzke
aa1a953a65 Moved node dumping code to generic location 2018-11-21 23:22:38 +01:00
Thomas Patzke
26d888aec3 Removed "not null" handling code 
Feature was removed some time ago.
 2018-11-21 22:56:48 +01:00
Thomas Patzke
9e28669c33 Backend es-qs return quotes on empty or whitespace-only string 2018-11-21 22:29:12 +01:00
Thomas Patzke
49d464f979 Fixed wildcards in es-qs backend 2018-11-20 23:23:54 +01:00
Thomas Patzke
396a030ed1 Removed duplicate code 2018-11-07 22:52:12 +01:00
Thomas Patzke
116a0e9f03 Merge branch 'master' of https://github.com/tuckner/sigma into tuckner-master 2018-11-07 22:27:41 +01:00
Thomas Patzke
5053cc4e95 Fixed optimizing of not conditions with subexpressions 
Optimization pass traversal is cut at ConditionNOT nodes.
 2018-11-07 13:54:45 +01:00
Thomas Patzke
a88b1e81ec Optimizer debugging code cleanup 
* Removed commented debugging code
* Output to stdin
* Coverage exception for _dumpNode
 2018-11-07 13:49:08 +01:00
Thomas Patzke
42ed8acec9 Improved test coverage 
* Adding tests
* Removal of coverage measurement for debugging code
 2018-11-04 23:28:40 +01:00
Thomas Patzke
418f8d10a3 Wrap conditions generated by mappings into sub-expression 2018-11-04 23:00:04 +01:00
Thomas Patzke
0e4842962b Added tests 2018-11-04 22:16:20 +01:00
tuckner
ca6ba4a85b Added NetWitness backend and tests 2018-10-31 14:24:14 -05:00
tuckner
26f73d60fa Added NetWitness backend and tests 2018-10-31 14:07:59 -05:00
Thomas Patzke
eacfaa7460 Check for forbidden null values in list items in Splunk backend 2018-10-27 01:07:03 +02:00
Thomas Patzke
423a73efd5 Dropped .py suffix 2018-10-22 23:02:05 +02:00
Thomas Patzke
b2d6d73034 Added requirements 2018-10-22 22:43:59 +02:00
Thomas Patzke
16e3838a90 Renamed script 2018-10-19 21:23:33 +02:00
Thomas Patzke
6b14930302 Recursive path traversal 2018-10-19 21:21:33 +02:00
Thomas Patzke
67b416379f Improved import of multiple rules 2018-10-19 19:53:00 +02:00
Thomas Patzke
0cc8b77307 Merge branch 'master' of https://github.com/pivotforensics/sigma into pivotforensics-master 2018-10-18 15:56:26 +02:00
ntim
e501c4a5b9 Added additional output type 'json' to the xpack-watcher backend which prints each watcher as compress json, one watcher per line 2018-10-17 10:38:56 +02:00
Thomas Patzke
265ce115a0 Fixed conditional field mapping usage in mapping chains 2018-10-16 13:57:51 +02:00
Thomas Patzke
a61b3d352a Added test cases 
* Generic log sources
* Splunk index queries
 2018-10-15 15:24:18 +02:00
Michael H
5b33713ef8 Quick fix for string formatting bug 2018-10-13 20:21:37 -05:00
Michael H
38ec257f7e Re-doing LogName formatting 2018-10-13 20:18:57 -05:00
Michael H
9f48265eb1 Adding re.sub for LogName that accounts for expression grouping 2018-10-13 20:09:54 -05:00
Michael H
7e184f01c6 Removing invalid fieldmapping 2018-10-13 19:53:39 -05:00
Michael H
bbb67fbba4 Adding support for reading sigma rule from stdin in sigmac 2018-10-07 10:11:47 -05:00
Michael H
aabaa0257b Merge branch 'master' of https://github.com/Neo23x0/sigma 2018-10-06 20:12:15 -05:00
Michael H
4b85a34b34 Added CSV option to powershell backend 2018-10-06 20:08:20 -05:00
Thomas Patzke
e28bc35cad Apply field mappings in generation of log source condition 2018-10-06 23:38:35 +02:00
Daniel Roethlisberger
fc45df144c Improve the comments on the optimizer 2018-10-03 13:44:03 +02:00
Daniel Roethlisberger
87aa1b5521 Move optimizer to sigma.parser.condition to enable it for all backends 2018-10-03 00:24:31 +02:00
Daniel Roethlisberger
cd3661b60c Fix optimization of NOT corner cases 2018-10-02 22:48:33 +02:00
Daniel Roethlisberger
bed88cf813 Make uniq work for lists within definitions 2018-10-02 22:12:54 +02:00
Daniel Roethlisberger
7165128fa5 Remove None from AST - fixes None-related test failures 2018-10-02 21:44:37 +02:00
Daniel Roethlisberger
2242fc5ac8 Optimize the boolean expressions in the AST before generating output 
Add code optimizing the boolean expressions in the abstract syntax tree
before generating output using the backend.

The main idea behind optimizing the AST is that less repeated terms is
generally better for backend performance.  This is especially relevant
to backends that do not perform any query language optimization down
the road, such as those that generate code.

The following optimizations are currently performed:

-   Removal of empty OR(), AND()
-   OR(X), AND(X)                 =>  X
-   OR(X, X, ...), AND(X, X, ...) =>  OR(X, ...), AND(X, ...)
-   OR(X, OR(Y))                  =>  OR(X, Y)
-   OR(AND(X, ...), AND(X, ...))  =>  AND(X, OR(AND(...), AND(...)))
-   NOT(NOT(X))                   =>  X

A common example for when these suboptimal rules actually occur in
practice is when a rule has multiple alternative detections that are
OR'ed together in the condition, and all of the detections include a
common element, such as the same EventID.

This implementation is not optimized for performance and will perform
poorly on very large expressions.
 2018-10-02 21:14:25 +02:00
Karneades
468af42de5 Add missing event id list handling in PowerShell backend 2018-09-29 14:43:28 +02:00
Karneades
c289484c5c Improve default field handling in PowerShell backend 2018-09-29 12:29:44 +02:00
Florian Roth
1c2431f33b
Merge pull request #169 from Karneades/fix-aggregation-exeption 
Add rule filename to "not implemented" exception output
 2018-09-26 11:50:25 +02:00
Karneades
c66b00356d Add initial version of PowerShell backend 
* Add PowerShell backend
* Add PowerShell config file

State: Work in progress :)

See https://github.com/Neo23x0/sigma/issues/94
 2018-09-23 21:41:48 +02:00
Karneades
fe6f4c7475 Add rule filename to exception output for unsupported aggregation 2018-09-23 19:12:50 +02:00
Thomas Patzke
1d12fc290c Added Winlogbeat configuration 2018-09-20 12:08:11 +02:00
Thomas Patzke
2fbf17ff34 Addition and resolution of field mapping chains explicitely checks for list 2018-09-13 16:22:29 +02:00
Thomas Patzke
41a8ef2fd9 Implemented resolve_fieldname in FieldMappingChain 2018-09-13 14:56:31 +02:00
Thomas Patzke
2330306db1 Added merged field mapping and log sources dict to config chain 2018-09-13 14:55:05 +02:00
Thomas Patzke
ba76f04fe6 Merging of raw configurations in configuration chains 2018-09-13 13:49:36 +02:00
Thomas Patzke
d81946df39 Stacked configurations 
- Added log source rewriting
- Removed log source merging condition type setting
- Simplified SigmaLogsourceConfiguration constructor
- Condition is generated in SigmaParser instead of SigmaLogsourceConfiguration

Missing:
- Merging of raw config dict for backends that rely on this (es-dsl)
 2018-09-12 23:40:22 +02:00
Thomas Patzke
210f7ac044 Rewrote logsource definition merging to set generator 2018-09-12 22:29:51 +02:00
Thomas Patzke
f3c60a6309 Added tag filtering to sigmac 2018-09-06 00:57:54 +02:00
Thomas Patzke
7f875af1ca Fixed WDATP backend 
It never generated any output due to missing return in generate()
method.
 2018-09-06 00:31:40 +02:00
Thomas Patzke
1d7722c1cb Added configuration and field mapping chains 
Missing: field name mapping of log source conditions.
 2018-08-27 00:17:27 +02:00
James Dickenson
29bed766dd removed re-introduced output class from qradar backend. fixed list handling error. 2018-08-21 22:45:12 -07:00
James Dickenson
468f040c0a Merge branch 'qradar-dev' 2018-08-20 21:54:30 -07:00
James Dickenson
9a61f40cef added support flor flow data in qradar backend 2018-08-16 21:44:17 -07:00
Thomas Patzke
320bb9f8c4 Added rewrite config to generic sysmon configuration 2018-08-14 21:34:54 +02:00
Thomas Patzke
430972231f Added generic sysmon configuration with process_execution config 2018-08-14 21:34:54 +02:00
James Dickenson
a8d1831382 Added aggregation support for qradar backend 2018-08-13 23:04:10 -07:00
Thomas Patzke
dce4b4825d Fixed aggregations without field name 
Generated query contained field name "None".
 2018-08-10 15:07:07 +02:00
Thomas Patzke
e0b3f91b2a Removed empty line 2018-08-08 23:15:13 +02:00
Thomas Patzke
f8246e9f49 Removed "not implemented" hints for available options in sigmac 2018-08-04 23:31:29 +02:00
Thomas Patzke
af9f636199 Removal of backend output classes 
Breaking change: Instead of feeding the output class with the results,
they are now returned as strings (*Backend.generate()) or list
(SigmaCollectionParser.generate()). Users of the library must now take
care of the output to the terminal, files or wherever Sigma rules should
be pushed to.
 2018-08-02 22:41:32 +02:00
Thomas Patzke
1c9d0a176e Moved const_start into class definition 2018-07-28 23:51:33 +02:00
Thomas Patzke
df74460629 Fixed imports after config split 2018-07-27 23:54:18 +02:00
Thomas Patzke
e02af9aa37 Merge config split branches 2018-07-27 23:16:50 +02:00
Thomas Patzke
eb440b3357 Split config - code removal from configuration 2018-07-27 23:02:35 +02:00
Thomas Patzke
36ada66007 Split config - Copy configuration 2018-07-27 23:01:41 +02:00
Thomas Patzke
920c4b061d Split config - code removal from filter 2018-07-27 22:35:30 +02:00
Thomas Patzke
d235a9e017 Split config - Copy filter 2018-07-27 00:23:22 +02:00
Thomas Patzke
50a6a92d20 Split config - code removal from exceptions 2018-07-27 00:17:35 +02:00
Thomas Patzke
405bc4a0d1 Split config - Copy exception 2018-07-27 00:17:13 +02:00
Thomas Patzke
096bc35447 Split config - code removal from mapping 2018-07-27 00:15:14 +02:00
Thomas Patzke
4ffbb25960 Split config - Copy mapping 2018-07-27 00:13:19 +02:00
Thomas Patzke
1c4c67053c Fixes for parser split 
* Fixed imports
* Rename
 2018-07-27 00:02:07 +02:00
Thomas Patzke
88a4a5d36a Merge parser split branches 2018-07-26 23:42:09 +02:00
Thomas Patzke
595327ace4 Split parser - code removal from condition 2018-07-26 23:40:22 +02:00
Thomas Patzke
c8043368bd Split parser - code removal from rule 2018-07-26 22:43:49 +02:00
Thomas Patzke
294ca20350 Split parser - code removal from collection 2018-07-26 22:28:33 +02:00
Thomas Patzke
3a0de01bad Split parser - code removal from base 2018-07-26 22:22:21 +02:00
Thomas Patzke
b9425d13df Split parser - code removal from exceptions 2018-07-26 22:18:21 +02:00
Thomas Patzke
e550bf5c3b Split parser - Copy base 2018-07-26 22:15:04 +02:00
Thomas Patzke
a2329de03c Split parser - Copy rule 2018-07-26 22:07:38 +02:00
Thomas Patzke
1abb13c5d9 Split parser - Copy condition 2018-07-24 00:13:37 +02:00
Thomas Patzke
a8501cb446 Split parser - Copy exceptions 2018-07-24 00:08:23 +02:00
Thomas Patzke
983ee6eeb9 Splitting parser - copying collections 2018-07-24 00:06:02 +02:00
Thomas Patzke
54f5870658 Removed debugging code 2018-07-24 00:04:24 +02:00
Thomas Patzke
b76fa884ec Changed copyright notices accordingly 2018-07-24 00:01:16 +02:00
Thomas Patzke
fbde251ebc Added missing exception import in ES backend 2018-07-22 09:26:25 +02:00
Thomas Patzke
91e6b8ca6b Merging refactoring changes into master 2018-07-22 09:23:07 +02:00
Thomas Patzke
cf175d7b7e Removal from sigma.backends.qradar 2018-07-22 09:14:50 +02:00
Thomas Patzke
097660c678 Splitting backends - Copy qradar.py 2018-07-22 09:12:29 +02:00
Thomas Patzke
c8e21b3f24 Fixing after split 
* Fixing imports
* Discovery in new sub modules
 2018-07-21 01:09:02 +02:00
Thomas Patzke
b85aec6157 Merging backend split branches 2018-07-21 00:59:50 +02:00
Thomas Patzke
3e2184ac61 Removal from sigma.backends.elasticsearch 2018-07-21 00:37:36 +02:00
Thomas Patzke
a9257c32c6 Sigma tools release 0.6 2018-07-17 23:12:23 +02:00
nikotin
b5f27d75be Added Qradar backend 2018-07-17 15:25:06 +03:00
Thomas Patzke
c2b1a58813 Removal from sigma.backends.wdatp 2018-07-10 23:49:39 +02:00
Thomas Patzke
45782c6328 Removal from sigma.backends.splunk 2018-07-10 23:48:47 +02:00
Thomas Patzke
46f29d2eb2 Removal from sigma.backends.output 2018-07-10 23:47:41 +02:00
Thomas Patzke
2d4145cfe8 Removal from sigma.backends.discovery 2018-07-10 23:46:52 +02:00
Thomas Patzke
83acff6859 Splitting backends - Copy discovery.py 2018-07-10 23:46:16 +02:00
Thomas Patzke
d340487e94 Removal from sigma.backends.base 2018-07-10 23:44:14 +02:00
Thomas Patzke
2e7d366da5 Removal from sigma.backends.mixins 2018-07-10 23:42:38 +02:00
Thomas Patzke
bb78c1428e Removal from sigma.backends.logpoint 2018-07-10 23:41:15 +02:00
Thomas Patzke
2edeaee748 Removal from sigma.backends.graylog 2018-07-10 23:40:17 +02:00
Thomas Patzke
e5baca0ac4 Removal from sigma.backends.qualys 2018-07-10 23:39:18 +02:00
Thomas Patzke
fdfe346adc Removal from sigma.backends.exceptions 2018-07-10 23:37:59 +02:00
Thomas Patzke
7fbc3a35a3 Removal from sigma.backends.cli 2018-07-10 23:33:40 +02:00
Thomas Patzke
881f72e418 Removal from sigma.backends.tools 2018-07-10 23:32:42 +02:00
Thomas Patzke
09ac41949c Removal from sigma.backends.archsight 2018-07-10 23:22:36 +02:00
Thomas Patzke
04b89befce Splitting backends - Copy elasticsearch.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
bb9bef4deb Splitting backends - Copy wdatp.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
72480d304b Splitting backends - Copy splunk.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
c5d5c52850 Splitting backends - Copy output.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
0c93040da5 Splitting backends - Copy base.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
a8e19bb4ba Splitting backends - Copy mixins.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
116fe16512 Splitting backends - Copy logpoint.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
b621e9c3a8 Splitting backends - Copy graylog.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
a2ee36eac7 Splitting backends - Copy qualys.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
32c70b26d8 Splitting backends - Copy exceptions.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
43d951b173 Splitting backends - Copy cli.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
a6cd7a3d6b Splitting backends - Copy tools.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
7a2b1ae790 Splitting backends - Copy arcsight.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
d064d24fbe Sigmac WDATP backend: renamed action types 2018-07-10 22:49:38 +02:00
Thomas Patzke
0cdfc776de Sigma tools release 0.5 2018-07-03 00:07:43 +02:00
Thomas Patzke
67158ba1d2 Merge branch 'master' of https://github.com/SaltyHash123/sigma into SaltyHash123-master 2018-07-02 23:14:04 +02:00
Florian Roth
2a74a62c67 Config file for SPARK scanner 2018-06-29 16:42:16 +02:00
Roey
14464f8c79 Added support of splunk dashboards (xml) 2018-06-22 14:17:58 +02:00
Thomas Patzke
7d1b801858 Merge branch 'devel-sigmac-wdatp' 2018-06-22 00:43:23 +02:00
Thomas Patzke
d8e036f737 sigmac: Parameter for ignoring "not supported" errors 
Used to pass tests with complete rule set that would fail for backends
which target systems don't support required features.
 2018-06-22 00:23:59 +02:00
Thomas Patzke
31727b3b25 Added Windows Defender ATP backend 
Missing:
* Aggregations
 2018-06-22 00:03:10 +02:00
Thomas Patzke
e72c0d5de4 SingleTextQueryBackend ignores empty components in composed queries 
Example: one component of a AND-composition is ignored if invoked
generate* call returns None.
 2018-06-21 23:59:41 +02:00
Thomas Patzke
d8a7bcad39 Reordered rule generation 
Generation of query parts before and after main query gives access to
information possibly gathered while main query generation.
 2018-06-21 23:50:13 +02:00
Florian Roth
3d52030391 Changed help text for -r flag 2018-06-13 00:08:46 +02:00
Florian Roth
7edd95744a Windows NTLM 2018-06-13 00:08:46 +02:00
Florian Roth
c9658074dd Removed "not yet implemented" comment from -r flag 2018-06-13 00:08:46 +02:00
Thomas Patzke
f6d5e5dd99 Sigmac parameter -I now ignores all backend errors 
New backends introduced further exceptions and the intention of -I is to
get a successful run.
 2018-06-07 23:33:12 +02:00
Thomas Patzke
8ddb369df3 Integration of Qualys backend 
* Changed description text to one-liner
* Output to intended class
* Minor code optimizations
 2018-06-07 23:31:09 +02:00
Thomas Patzke
ce9db548ff Integration of ArcSight backend 
* Rename
* Changed description to one line to beautify output of backend list
* Small bugfix in handling of numeric values
 2018-06-07 23:04:36 +02:00
nikotin
d13e8d7bd3 Added ArcSight & Qualys backends 2018-06-07 16:18:23 +03:00
Florian Roth
4eabc5ea5c
Sigmac Usage 2018-06-01 10:33:11 +02:00
Florian Roth
65cc78f9e8 Windows Config Update - DNS logs 2018-05-22 16:59:58 +02:00
Paul Dutot
715a88542d Graylog backend added 2018-05-17 15:51:25 +01:00
milkmix
37ee355a77 patched es-dsl 2018-05-17 08:44:50 +02:00
Thomas Patzke
738d03c751 Fixed position of line separation if rulecomment and verbose is active 2018-05-13 22:36:51 +02:00