valitydev/SigmaHQ
14
0
Fork 0
mirror of https://github.com/valitydev/SigmaHQ.git synced 2024-11-07 09:48:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,018 Commits 20 Branches 25 Tags 21 MiB
4d65b62063
Commit Graph

495 Commits

Author SHA1 Message Date
Thomas Patzke
396a030ed1 Removed duplicate code 2018-11-07 22:52:12 +01:00
Thomas Patzke
116a0e9f03 Merge branch 'master' of https://github.com/tuckner/sigma into tuckner-master 2018-11-07 22:27:41 +01:00
Thomas Patzke
5053cc4e95 Fixed optimizing of not conditions with subexpressions 
Optimization pass traversal is cut at ConditionNOT nodes.
 2018-11-07 13:54:45 +01:00
Thomas Patzke
a88b1e81ec Optimizer debugging code cleanup 
* Removed commented debugging code
* Output to stdin
* Coverage exception for _dumpNode
 2018-11-07 13:49:08 +01:00
Thomas Patzke
42ed8acec9 Improved test coverage 
* Adding tests
* Removal of coverage measurement for debugging code
 2018-11-04 23:28:40 +01:00
Thomas Patzke
418f8d10a3 Wrap conditions generated by mappings into sub-expression 2018-11-04 23:00:04 +01:00
Thomas Patzke
0e4842962b Added tests 2018-11-04 22:16:20 +01:00
tuckner
ca6ba4a85b Added NetWitness backend and tests 2018-10-31 14:24:14 -05:00
tuckner
26f73d60fa Added NetWitness backend and tests 2018-10-31 14:07:59 -05:00
Thomas Patzke
eacfaa7460 Check for forbidden null values in list items in Splunk backend 2018-10-27 01:07:03 +02:00
Thomas Patzke
423a73efd5 Dropped .py suffix 2018-10-22 23:02:05 +02:00
Thomas Patzke
b2d6d73034 Added requirements 2018-10-22 22:43:59 +02:00
Thomas Patzke
16e3838a90 Renamed script 2018-10-19 21:23:33 +02:00
Thomas Patzke
6b14930302 Recursive path traversal 2018-10-19 21:21:33 +02:00
Thomas Patzke
67b416379f Improved import of multiple rules 2018-10-19 19:53:00 +02:00
Thomas Patzke
0cc8b77307 Merge branch 'master' of https://github.com/pivotforensics/sigma into pivotforensics-master 2018-10-18 15:56:26 +02:00
ntim
e501c4a5b9 Added additional output type 'json' to the xpack-watcher backend which prints each watcher as compress json, one watcher per line 2018-10-17 10:38:56 +02:00
Thomas Patzke
265ce115a0 Fixed conditional field mapping usage in mapping chains 2018-10-16 13:57:51 +02:00
Thomas Patzke
a61b3d352a Added test cases 
* Generic log sources
* Splunk index queries
 2018-10-15 15:24:18 +02:00
Michael H
5b33713ef8 Quick fix for string formatting bug 2018-10-13 20:21:37 -05:00
Michael H
38ec257f7e Re-doing LogName formatting 2018-10-13 20:18:57 -05:00
Michael H
9f48265eb1 Adding re.sub for LogName that accounts for expression grouping 2018-10-13 20:09:54 -05:00
Michael H
7e184f01c6 Removing invalid fieldmapping 2018-10-13 19:53:39 -05:00
Michael H
bbb67fbba4 Adding support for reading sigma rule from stdin in sigmac 2018-10-07 10:11:47 -05:00
Michael H
aabaa0257b Merge branch 'master' of https://github.com/Neo23x0/sigma 2018-10-06 20:12:15 -05:00
Michael H
4b85a34b34 Added CSV option to powershell backend 2018-10-06 20:08:20 -05:00
Thomas Patzke
e28bc35cad Apply field mappings in generation of log source condition 2018-10-06 23:38:35 +02:00
Daniel Roethlisberger
fc45df144c Improve the comments on the optimizer 2018-10-03 13:44:03 +02:00
Daniel Roethlisberger
87aa1b5521 Move optimizer to sigma.parser.condition to enable it for all backends 2018-10-03 00:24:31 +02:00
Daniel Roethlisberger
cd3661b60c Fix optimization of NOT corner cases 2018-10-02 22:48:33 +02:00
Daniel Roethlisberger
bed88cf813 Make uniq work for lists within definitions 2018-10-02 22:12:54 +02:00
Daniel Roethlisberger
7165128fa5 Remove None from AST - fixes None-related test failures 2018-10-02 21:44:37 +02:00
Daniel Roethlisberger
2242fc5ac8 Optimize the boolean expressions in the AST before generating output 
Add code optimizing the boolean expressions in the abstract syntax tree
before generating output using the backend.

The main idea behind optimizing the AST is that less repeated terms is
generally better for backend performance.  This is especially relevant
to backends that do not perform any query language optimization down
the road, such as those that generate code.

The following optimizations are currently performed:

-   Removal of empty OR(), AND()
-   OR(X), AND(X)                 =>  X
-   OR(X, X, ...), AND(X, X, ...) =>  OR(X, ...), AND(X, ...)
-   OR(X, OR(Y))                  =>  OR(X, Y)
-   OR(AND(X, ...), AND(X, ...))  =>  AND(X, OR(AND(...), AND(...)))
-   NOT(NOT(X))                   =>  X

A common example for when these suboptimal rules actually occur in
practice is when a rule has multiple alternative detections that are
OR'ed together in the condition, and all of the detections include a
common element, such as the same EventID.

This implementation is not optimized for performance and will perform
poorly on very large expressions.
 2018-10-02 21:14:25 +02:00
Karneades
468af42de5 Add missing event id list handling in PowerShell backend 2018-09-29 14:43:28 +02:00
Karneades
c289484c5c Improve default field handling in PowerShell backend 2018-09-29 12:29:44 +02:00
Florian Roth
1c2431f33b
Merge pull request #169 from Karneades/fix-aggregation-exeption 
Add rule filename to "not implemented" exception output
 2018-09-26 11:50:25 +02:00
Karneades
c66b00356d Add initial version of PowerShell backend 
* Add PowerShell backend
* Add PowerShell config file

State: Work in progress :)

See https://github.com/Neo23x0/sigma/issues/94
 2018-09-23 21:41:48 +02:00
Karneades
fe6f4c7475 Add rule filename to exception output for unsupported aggregation 2018-09-23 19:12:50 +02:00
Thomas Patzke
1d12fc290c Added Winlogbeat configuration 2018-09-20 12:08:11 +02:00
Thomas Patzke
2fbf17ff34 Addition and resolution of field mapping chains explicitely checks for list 2018-09-13 16:22:29 +02:00
Thomas Patzke
41a8ef2fd9 Implemented resolve_fieldname in FieldMappingChain 2018-09-13 14:56:31 +02:00
Thomas Patzke
2330306db1 Added merged field mapping and log sources dict to config chain 2018-09-13 14:55:05 +02:00
Thomas Patzke
ba76f04fe6 Merging of raw configurations in configuration chains 2018-09-13 13:49:36 +02:00
Thomas Patzke
d81946df39 Stacked configurations 
- Added log source rewriting
- Removed log source merging condition type setting
- Simplified SigmaLogsourceConfiguration constructor
- Condition is generated in SigmaParser instead of SigmaLogsourceConfiguration

Missing:
- Merging of raw config dict for backends that rely on this (es-dsl)
 2018-09-12 23:40:22 +02:00
Thomas Patzke
210f7ac044 Rewrote logsource definition merging to set generator 2018-09-12 22:29:51 +02:00
Thomas Patzke
f3c60a6309 Added tag filtering to sigmac 2018-09-06 00:57:54 +02:00
Thomas Patzke
7f875af1ca Fixed WDATP backend 
It never generated any output due to missing return in generate()
method.
 2018-09-06 00:31:40 +02:00
Thomas Patzke
1d7722c1cb Added configuration and field mapping chains 
Missing: field name mapping of log source conditions.
 2018-08-27 00:17:27 +02:00
James Dickenson
29bed766dd removed re-introduced output class from qradar backend. fixed list handling error. 2018-08-21 22:45:12 -07:00
James Dickenson
468f040c0a Merge branch 'qradar-dev' 2018-08-20 21:54:30 -07:00
James Dickenson
9a61f40cef added support flor flow data in qradar backend 2018-08-16 21:44:17 -07:00
Thomas Patzke
320bb9f8c4 Added rewrite config to generic sysmon configuration 2018-08-14 21:34:54 +02:00
Thomas Patzke
430972231f Added generic sysmon configuration with process_execution config 2018-08-14 21:34:54 +02:00
James Dickenson
a8d1831382 Added aggregation support for qradar backend 2018-08-13 23:04:10 -07:00
Thomas Patzke
dce4b4825d Fixed aggregations without field name 
Generated query contained field name "None".
 2018-08-10 15:07:07 +02:00
Thomas Patzke
e0b3f91b2a Removed empty line 2018-08-08 23:15:13 +02:00
Thomas Patzke
f8246e9f49 Removed "not implemented" hints for available options in sigmac 2018-08-04 23:31:29 +02:00
Thomas Patzke
af9f636199 Removal of backend output classes 
Breaking change: Instead of feeding the output class with the results,
they are now returned as strings (*Backend.generate()) or list
(SigmaCollectionParser.generate()). Users of the library must now take
care of the output to the terminal, files or wherever Sigma rules should
be pushed to.
 2018-08-02 22:41:32 +02:00
Thomas Patzke
1c9d0a176e Moved const_start into class definition 2018-07-28 23:51:33 +02:00
Thomas Patzke
df74460629 Fixed imports after config split 2018-07-27 23:54:18 +02:00
Thomas Patzke
e02af9aa37 Merge config split branches 2018-07-27 23:16:50 +02:00
Thomas Patzke
eb440b3357 Split config - code removal from configuration 2018-07-27 23:02:35 +02:00
Thomas Patzke
36ada66007 Split config - Copy configuration 2018-07-27 23:01:41 +02:00
Thomas Patzke
920c4b061d Split config - code removal from filter 2018-07-27 22:35:30 +02:00
Thomas Patzke
d235a9e017 Split config - Copy filter 2018-07-27 00:23:22 +02:00
Thomas Patzke
50a6a92d20 Split config - code removal from exceptions 2018-07-27 00:17:35 +02:00
Thomas Patzke
405bc4a0d1 Split config - Copy exception 2018-07-27 00:17:13 +02:00
Thomas Patzke
096bc35447 Split config - code removal from mapping 2018-07-27 00:15:14 +02:00
Thomas Patzke
4ffbb25960 Split config - Copy mapping 2018-07-27 00:13:19 +02:00
Thomas Patzke
1c4c67053c Fixes for parser split 
* Fixed imports
* Rename
 2018-07-27 00:02:07 +02:00
Thomas Patzke
88a4a5d36a Merge parser split branches 2018-07-26 23:42:09 +02:00
Thomas Patzke
595327ace4 Split parser - code removal from condition 2018-07-26 23:40:22 +02:00
Thomas Patzke
c8043368bd Split parser - code removal from rule 2018-07-26 22:43:49 +02:00
Thomas Patzke
294ca20350 Split parser - code removal from collection 2018-07-26 22:28:33 +02:00
Thomas Patzke
3a0de01bad Split parser - code removal from base 2018-07-26 22:22:21 +02:00
Thomas Patzke
b9425d13df Split parser - code removal from exceptions 2018-07-26 22:18:21 +02:00
Thomas Patzke
e550bf5c3b Split parser - Copy base 2018-07-26 22:15:04 +02:00
Thomas Patzke
a2329de03c Split parser - Copy rule 2018-07-26 22:07:38 +02:00
Thomas Patzke
1abb13c5d9 Split parser - Copy condition 2018-07-24 00:13:37 +02:00
Thomas Patzke
a8501cb446 Split parser - Copy exceptions 2018-07-24 00:08:23 +02:00
Thomas Patzke
983ee6eeb9 Splitting parser - copying collections 2018-07-24 00:06:02 +02:00
Thomas Patzke
54f5870658 Removed debugging code 2018-07-24 00:04:24 +02:00
Thomas Patzke
b76fa884ec Changed copyright notices accordingly 2018-07-24 00:01:16 +02:00
Thomas Patzke
fbde251ebc Added missing exception import in ES backend 2018-07-22 09:26:25 +02:00
Thomas Patzke
91e6b8ca6b Merging refactoring changes into master 2018-07-22 09:23:07 +02:00
Thomas Patzke
cf175d7b7e Removal from sigma.backends.qradar 2018-07-22 09:14:50 +02:00
Thomas Patzke
097660c678 Splitting backends - Copy qradar.py 2018-07-22 09:12:29 +02:00
Thomas Patzke
c8e21b3f24 Fixing after split 
* Fixing imports
* Discovery in new sub modules
 2018-07-21 01:09:02 +02:00
Thomas Patzke
b85aec6157 Merging backend split branches 2018-07-21 00:59:50 +02:00
Thomas Patzke
3e2184ac61 Removal from sigma.backends.elasticsearch 2018-07-21 00:37:36 +02:00
Thomas Patzke
a9257c32c6 Sigma tools release 0.6 2018-07-17 23:12:23 +02:00
nikotin
b5f27d75be Added Qradar backend 2018-07-17 15:25:06 +03:00
Thomas Patzke
c2b1a58813 Removal from sigma.backends.wdatp 2018-07-10 23:49:39 +02:00
Thomas Patzke
45782c6328 Removal from sigma.backends.splunk 2018-07-10 23:48:47 +02:00
Thomas Patzke
46f29d2eb2 Removal from sigma.backends.output 2018-07-10 23:47:41 +02:00
Thomas Patzke
2d4145cfe8 Removal from sigma.backends.discovery 2018-07-10 23:46:52 +02:00
Thomas Patzke
83acff6859 Splitting backends - Copy discovery.py 2018-07-10 23:46:16 +02:00
Thomas Patzke
d340487e94 Removal from sigma.backends.base 2018-07-10 23:44:14 +02:00
Thomas Patzke
2e7d366da5 Removal from sigma.backends.mixins 2018-07-10 23:42:38 +02:00
Thomas Patzke
bb78c1428e Removal from sigma.backends.logpoint 2018-07-10 23:41:15 +02:00
Thomas Patzke
2edeaee748 Removal from sigma.backends.graylog 2018-07-10 23:40:17 +02:00
Thomas Patzke
e5baca0ac4 Removal from sigma.backends.qualys 2018-07-10 23:39:18 +02:00
Thomas Patzke
fdfe346adc Removal from sigma.backends.exceptions 2018-07-10 23:37:59 +02:00
Thomas Patzke
7fbc3a35a3 Removal from sigma.backends.cli 2018-07-10 23:33:40 +02:00
Thomas Patzke
881f72e418 Removal from sigma.backends.tools 2018-07-10 23:32:42 +02:00
Thomas Patzke
09ac41949c Removal from sigma.backends.archsight 2018-07-10 23:22:36 +02:00
Thomas Patzke
04b89befce Splitting backends - Copy elasticsearch.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
bb9bef4deb Splitting backends - Copy wdatp.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
72480d304b Splitting backends - Copy splunk.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
c5d5c52850 Splitting backends - Copy output.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
0c93040da5 Splitting backends - Copy base.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
a8e19bb4ba Splitting backends - Copy mixins.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
116fe16512 Splitting backends - Copy logpoint.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
b621e9c3a8 Splitting backends - Copy graylog.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
a2ee36eac7 Splitting backends - Copy qualys.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
32c70b26d8 Splitting backends - Copy exceptions.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
43d951b173 Splitting backends - Copy cli.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
a6cd7a3d6b Splitting backends - Copy tools.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
7a2b1ae790 Splitting backends - Copy arcsight.py 2018-07-10 23:15:04 +02:00
Thomas Patzke
d064d24fbe Sigmac WDATP backend: renamed action types 2018-07-10 22:49:38 +02:00
Thomas Patzke
0cdfc776de Sigma tools release 0.5 2018-07-03 00:07:43 +02:00
Thomas Patzke
67158ba1d2 Merge branch 'master' of https://github.com/SaltyHash123/sigma into SaltyHash123-master 2018-07-02 23:14:04 +02:00
Florian Roth
2a74a62c67 Config file for SPARK scanner 2018-06-29 16:42:16 +02:00
Roey
14464f8c79 Added support of splunk dashboards (xml) 2018-06-22 14:17:58 +02:00
Thomas Patzke
7d1b801858 Merge branch 'devel-sigmac-wdatp' 2018-06-22 00:43:23 +02:00
Thomas Patzke
d8e036f737 sigmac: Parameter for ignoring "not supported" errors 
Used to pass tests with complete rule set that would fail for backends
which target systems don't support required features.
 2018-06-22 00:23:59 +02:00
Thomas Patzke
31727b3b25 Added Windows Defender ATP backend 
Missing:
* Aggregations
 2018-06-22 00:03:10 +02:00
Thomas Patzke
e72c0d5de4 SingleTextQueryBackend ignores empty components in composed queries 
Example: one component of a AND-composition is ignored if invoked
generate* call returns None.
 2018-06-21 23:59:41 +02:00
Thomas Patzke
d8a7bcad39 Reordered rule generation 
Generation of query parts before and after main query gives access to
information possibly gathered while main query generation.
 2018-06-21 23:50:13 +02:00
Florian Roth
3d52030391 Changed help text for -r flag 2018-06-13 00:08:46 +02:00
Florian Roth
7edd95744a Windows NTLM 2018-06-13 00:08:46 +02:00
Florian Roth
c9658074dd Removed "not yet implemented" comment from -r flag 2018-06-13 00:08:46 +02:00
Thomas Patzke
f6d5e5dd99 Sigmac parameter -I now ignores all backend errors 
New backends introduced further exceptions and the intention of -I is to
get a successful run.
 2018-06-07 23:33:12 +02:00
Thomas Patzke
8ddb369df3 Integration of Qualys backend 
* Changed description text to one-liner
* Output to intended class
* Minor code optimizations
 2018-06-07 23:31:09 +02:00
Thomas Patzke
ce9db548ff Integration of ArcSight backend 
* Rename
* Changed description to one line to beautify output of backend list
* Small bugfix in handling of numeric values
 2018-06-07 23:04:36 +02:00
nikotin
d13e8d7bd3 Added ArcSight & Qualys backends 2018-06-07 16:18:23 +03:00
Florian Roth
4eabc5ea5c
Sigmac Usage 2018-06-01 10:33:11 +02:00
Florian Roth
65cc78f9e8 Windows Config Update - DNS logs 2018-05-22 16:59:58 +02:00
Paul Dutot
715a88542d Graylog backend added 2018-05-17 15:51:25 +01:00
milkmix
37ee355a77 patched es-dsl 2018-05-17 08:44:50 +02:00
Thomas Patzke
738d03c751 Fixed position of line separation if rulecomment and verbose is active 2018-05-13 22:36:51 +02:00
Thomas Patzke
f60e7e125f Sigma tools release 0.4 
* Various bug fixes in quoting of specific characters
* New backend es-dsl
 2018-05-01 00:50:07 +02:00
Thomas Patzke
7647587a8b Fixed quoting of backslashes in generated queries 2018-05-01 00:45:59 +02:00
Thomas Patzke
de2ed08695 Merge branch 'ci-es' 2018-05-01 00:34:11 +02:00
Thomas Patzke
e411039b56 Fixed escaping of \u in Elasticsearch Query String queries 2018-05-01 00:05:16 +02:00
Thomas Patzke
aeda30a389 Python rewrite of es-qs query test 2018-04-11 23:59:44 +02:00
milkmix
0b3b0c3aaf imported es-dsl code from repo 2018-04-06 17:36:11 +02:00
Thomas Patzke
4183b1b59e Sigma tools release 0.3.3 2018-03-29 11:17:03 +02:00
Thomas Patzke
22ee6f4521 sigmac: escaped wildcards (\* and \?) are passed in generated query 2018-03-29 11:15:20 +02:00
Thomas Patzke
17c1c1adff Added field name mappings to HELK configuration 2018-03-27 14:41:02 +02:00
Thomas Patzke
a3e02ea70f Various rule fixes 
* Field name: LogonProcess -> LogonProcessName
* Field name: Message -> AuditPolicyChanges
* Field name: ProcessCommandLine -> CommandLine
* Removed Type match in Kerberos RC4 encryption rule
  Problematic because text representation not unified and audit failures are possibly interesting events
* Removed field 'Severity' from rules (Redundant)
* Rule decomposition of win_susp_failed_logons_single_source) because of different field names
* Field name: SubjectAccountName -> SubjectUserName
* Field name: TargetProcess -> TargetImage
* Field name: TicketEncryption -> TicketEncryptionType
* Field name: TargetFileName -> TargetFilename
 2018-03-27 14:35:49 +02:00
Thomas Patzke
5f8b60cc24 sigmac: Improved fieldlist backend 
* Unique list of fields for multiple rules
* Aggregation support
 2018-03-22 00:03:51 +01:00
Thomas Patzke
5c0f811f4a Sigma tools release 0.3.2 2018-03-21 01:15:19 +01:00
Thomas Patzke
0018503501 sigmac: Fixed rulecommend backend option 2018-03-21 01:13:10 +01:00
Thomas Patzke
7360a68741 Sigma tools release 0.3.1 2018-03-21 00:59:23 +01:00
Thomas Patzke
4a9849b161 sigmac: improved backend options 
* parsing in main class
* help
 2018-03-21 00:53:44 +01:00
Thomas Patzke
bd20ffdad9 sigmac/kibana: curl URL quoted 2018-03-21 00:22:00 +01:00
Thomas Patzke
3f5f3a8d50 sigmac: Remove problematic characters from rule identifiers 2018-03-17 00:44:50 +01:00
Thomas Patzke
f6858c436a sigmac: Kibana curl output generates one index pattern line per pattern 2018-03-16 23:53:12 +01:00
Thomas Patzke
578118315c Merge branch 'devel-sigmac' into helk 2018-03-16 23:48:13 +01:00
Thomas Patzke
e162ba0155 Added HELK configuration 2018-03-16 23:42:31 +01:00
Thomas Patzke
13ec4c3e3b sigmac: Kibana curl importer script 2018-03-11 00:25:12 +01:00
Thomas Patzke
54d9e52527 Sigma tools release 0.3 2018-03-06 23:21:13 +01:00
Thomas Patzke
3b8b04fe09 Merge branch 'devel-sigmac' 2018-03-06 23:19:45 +01:00
Thomas Patzke
7141729ffc sigma/parser: Introduced new conditions 
* Any definition: 1 of them
* All definitions: all of them
* Any of selected definitions: 1 of def* (wildcard)
* All of selected definitions: all of def* (wildcard)
 2018-03-06 23:13:42 +01:00
Thomas Patzke
5a97befea0 Sigma tools release 0.2 2018-03-04 23:03:19 +01:00
Thomas Patzke
647fc6187a sigmac: Added proper 'Content-Type' header for xpack-watcher backend 2018-03-04 22:58:15 +01:00
Thomas Patzke
89aa300bbc Improved xpack-watcher actions 
* Log and mail
* Details in message
 2018-02-09 00:03:41 +01:00
Thomas Patzke
8336929d76 XPack Watcher Backend: Improved aggregation capabilities 
* Aggregation with "...count(field)...", "...by field..." and
  combination of both
* Still only count() supported
 2018-02-08 22:17:35 +01:00
Thomas Patzke
4762a1cc30 Removed abandoned SigmaAggregationParser.trans_timeframe() method 2018-02-05 23:30:00 +01:00
Thomas Patzke
ec3f0f6d60 Fixed before/after logic 
If nothing was generated "None" was printed.
 2018-02-01 22:49:02 +01:00
Thomas Patzke
76bdcba71f Added rulecomment option to all single-query output backends 
Prints comment with rule before output.
 2018-01-27 23:48:10 +01:00
Thomas Patzke
7708a538f4 New PyPI release 2017-12-14 22:40:31 +01:00
Thomas Patzke
fc2dd90aaf Skipping dotfiles 2017-12-14 22:39:51 +01:00
Thomas Patzke
497496fdf1 New release 2017-12-13 00:28:50 +01:00
Thomas Patzke
f3d19f394e Fixed encoding issues 
Some OS environments don't use UTF-8 as default encoding. Enforced it
for output files and stdout.
 2017-12-13 00:12:56 +01:00
Thomas Patzke
19cc299c57 Added PyPI README 2017-12-09 22:13:25 +01:00
Thomas Patzke
fd7b7bb438 Fixed build 
Reference to main README
 2017-12-09 08:57:51 +01:00
Thomas Patzke
da9127276c PyPI release documentation 2017-12-09 00:23:34 +01:00
Thomas Patzke
d6526387d3 Renamed PyPI package 2017-12-09 00:15:34 +01:00
Thomas Patzke
d82a78fa3d Finalizing PyPI release 
* Removed .py suffix from command line tools
* sigmac tells when it does nothing and prints usage notice
* Makefile upload target
* minor changes
 2017-12-08 23:50:08 +01:00
Thomas Patzke
09d40ab2da Finished packaging and refactoring 2017-12-08 22:32:39 +01:00
Thomas Patzke
68d8afe4e6 Intermediate refactoring commit: moving code into package 
Further splitting sigma.py into smaller parts.
 2017-12-08 21:45:05 +01:00
Thomas Patzke
11f52b981b Merge branch 'lgpl' into packaging 2017-12-08 17:15:23 +01:00
Thomas Patzke
764e064f8c First (untested) packaging 2017-12-08 00:32:41 +01:00
Thomas Patzke
2ce0be1f2d Re-licensing toolchain under LGPLv3 
Thanks to Ben de Haan and Devin Ferguson for permission for this change.
 2017-12-07 21:55:43 +01:00
Thomas Patzke
3b9ff57a38 Added merge_sigma tool 
* Tests
* Restructured Makefile
 2017-11-14 22:17:18 +01:00
Thomas Patzke
f478cffb41 Added default index configs for usual ELK setups 
* Added test case for defaultindex with kibana backend
 2017-11-09 10:05:41 +01:00
Thomas Patzke
46f1ce35a8 sigmac/kibana backend: added index fallback if none determined 2017-11-09 10:02:23 +01:00
Florian Roth
1bea284280 Added Windows Driver Framework log source to configs 2017-11-09 08:42:58 +01:00
Florian Roth
e83e3a0c07 Bugfixes in Splunk config 2017-11-09 08:41:07 +01:00
Thomas Patzke
b03f9359ec sigmac: Added rule filter 2017-11-02 00:02:15 +01:00
Thomas Patzke
732f01878f Sigma rule collection YAML action documents 2017-11-01 00:17:55 +01:00
Thomas Patzke
d0b2bd9875 Multiple rules per file 
* New wrapper class SigmaCollectionParser parses all YAML documents
  contained in file and handles multiple SigmaParser instantiation.
* Exemplary extended one security/4688 rule to security/4688 + sysmon/1
 2017-10-31 23:06:18 +01:00
Thomas Patzke
5743e25931 Added logging framework 2017-10-31 22:13:20 +01:00
Thomas Patzke
720c992573 Dropped within keyword 
Covered by timeframe attribute.

Fixes issue #26.
 2017-10-30 00:25:56 +01:00
Thomas Patzke
012cb6227f Added proper handling of null/not null values 
Fixes issue #25
 2017-10-29 23:57:39 +01:00
Thomas Patzke
5fa9e685b1 Splitted parts of generate to generateQuery in backend code 2017-10-25 00:03:03 +02:00
Thomas Patzke
6d0e85fcfa Fixed Splunk backend (#50) 2017-10-24 23:48:47 +02:00
Thomas Patzke
65e1f8ec2b Increased test coverage 
* more tests
* removed unneeded code
* increased coverage fail threshold
 2017-10-23 23:30:44 +02:00
Thomas Patzke
3389656a5b Added ELK default index config 2017-10-23 00:45:33 +02:00
Thomas Patzke
7f93d3ca47 Kibana backend throws exception when multiple indices appear 
* Introduced backend errors with handling in sigmac
 2017-10-23 00:45:01 +02:00
Thomas Patzke
cb9aeac7d9 Added default index handling 
* Removed default index handling from backend code
* Added default indices to config templates
 2017-10-23 00:08:39 +02:00
Thomas Patzke
ec996e7353 Improved test coverage 2017-10-19 17:42:56 +02:00
Thomas Patzke
5449a12a14 Added GrepBackend 
Moved field quoting/filtering into QuoteCharMixin
 2017-10-18 19:03:38 +02:00
Thomas Patzke
54cf9af0c9 Removed ELK Sysmon config 
It's contained in ELK Windows config
 2017-10-18 15:23:55 +02:00
Thomas Patzke
b8eedfe3f0 Fixes and refactoring of KibanaBackend and XPackWatcherBackend 
* Moved unnecessary code out of condition loop
* Index specific rule-name not appended to rulename variable used later
  from other rule/index.
* Merged condition loop
 2017-09-30 23:22:05 +02:00
Thomas Patzke
1d314e326e sigmac: MultiRuleOutputMixin 
* Moved rule name generation into mixin
* KibanaBackend and XPackWatcherBackend now use this mixin instead of
  doing the same thing in both classes.
 2017-09-30 01:03:08 +02:00
Thomas Patzke
b47e3e45a8 Merge branch 'devel-sigmac' 2017-09-22 00:31:22 +02:00
Thomas Patzke
d410adb397 sigmac: X-Pack Watcher backend improvements 
* Renamed backend class according to convention
* Output types: curl (default) and plain
* Prefix of rule names
* Indices from configuration
* Support for multiple conditions per rule
* Usage of parsed condition
* Support for all condition operators
* Fixed bug preventing from passing multiple options to backend
* Added to CI tests
 2017-09-22 00:28:35 +02:00
Thomas Patzke
62eb3b2923 Merge branch 'devel-sigmac' of https://github.com/megadevx/sigma into devel-sigmac-watcher 2017-09-19 23:08:04 +02:00
Thomas Patzke
545e05370f Added first config for logstash-linux project 
URL: https://github.com/thomaspatzke/logstash-linux
 2017-09-17 00:36:04 +02:00
Thomas Patzke
a18b8eca52 sigmac: changed backend description for kibana backend 2017-09-17 00:31:25 +02:00
Thomas Patzke
270ab9ba78 Added backend options 
* generic support for backend-specific options
* kibana backend option for title prefix
 2017-09-16 23:46:40 +02:00
Thomas Patzke
c8a66e48b6 sigmac: improved Kibana backend 
* added fields from rules
* default index if none is matching
 2017-09-16 00:39:37 +02:00
Thomas Patzke
d3201229b0 sigmac: Fixed matching of log sources between rules and configuration 2017-09-16 00:32:31 +02:00
devife
9bc8e12a4f Created a X-Pack Watcher output. 
This is has only been tested slightly.
 2017-09-15 09:49:57 -05:00
devife
135e389334 Created a X-Pack Watcher output. 
This is has only been tested slightly.
 2017-09-15 09:46:37 -05:00
Thomas Patzke
e5da26578d sigmac/kibana backend: index names from configuration 2017-09-11 00:30:01 +02:00
Thomas Patzke
77a3e7ed91 Code cleanup 2017-09-11 00:27:14 +02:00
Thomas Patzke
be3c0cfb89 sigmac: Kibana backend, first version 
* totally untested!
* only supports searches
* no visualizations/aggregation expressions
* some fields are filled with default values (see code comments)
 2017-09-05 00:14:13 +02:00
Thomas Patzke
c5fc74f440 Further backend changes 
* backends get complete SigmaParser objects instead of condition
* addition of finalize step for backends
* Renaming of output classes
 2017-09-04 00:56:04 +02:00
Thomas Patzke
39381305d8 sigmac: Generic Text File Output 
Moved output logic into generic class.
 2017-08-29 00:05:59 +02:00
Florian Roth
edf2787402 Removed some spaces and added Win 10 WMI eventlog 2017-08-22 10:04:56 +02:00
Thomas Patzke
487ab99507 Changed sigmac error behavior on I/O errors 2017-08-07 08:54:18 +02:00
Thomas Patzke
d84f9dcc1c Aggregation 'near' raises NotImplementedError in backends splunk and logpoint 2017-08-05 23:48:28 +02:00
Thomas Patzke
f5b07dc9af Added semantic parsing of near expressions 2017-08-05 00:28:22 +02:00
Thomas Patzke
d17604d007 Merge branch 'master' into travis-test 2017-08-03 00:11:08 +02:00
Thomas Patzke
5706361464 Parsing of "near ... within" aggregation operator 
* Operator is only parsed. No processing or passing of parsed data to
  backends.
* Changed rule sysmon_mimikatz_inmemory_detection.yml accordingly.
 2017-08-03 00:05:48 +02:00
Thomas Patzke
52525236a5 sigmac: added parameter to control error behavior 
* --defer-abort
* --ignore-not-implemented
 2017-08-02 00:56:22 +02:00
Thomas Patzke
3495bac9cb sigmac: return error codes 2017-07-31 00:31:49 +02:00
Ben de Haan
43c4486de0 Added LogPoint aggregation 
Added generateAggregation function for LogPoint
 2017-06-19 15:21:29 +02:00
Florian Roth
c1f5bd1540 Sigmac bugfix: showing faulty condition 2017-06-12 10:07:15 +02:00
Thomas Patzke
9d49daecea Restructured backends 
Moved most logic into generic base class SingleTextQueryBackend which is
configured by class variables.
 2017-06-02 23:43:45 +02:00
Thomas Patzke
6a29884615 Structured backends module with comments 2017-05-26 23:42:49 +02:00
Thomas Patzke
998bb0079d Fixed Splunk config for sigmac again 2017-05-26 22:40:06 +02:00
Thomas Patzke
18a9fd18ef Fixed Splunk configuration 
Substituted source: with sourcetype:
 2017-05-26 00:13:30 +02:00
Florian Roth
f66085b198 Added eventlog source DNS Server to configs 2017-05-08 13:09:17 +02:00
Thomas Patzke
05e9d1e1e9 Check if aggregation is present in BaseBackend 
Caused NotImplementedError in ElasticsearchQueryStringBackend.
 2017-04-17 00:11:20 +02:00
Ben de Haan
dddb83393d Added field mappings for events with logins 2017-03-30 10:49:36 +02:00
Ben de Haan
cb9a9bc2ff Added LogPoint conditional username mapping 
Conditional mapping of SubjectAccountName based on EventID. Not a comprehensive list, but should include most relevant Event IDs.
 2017-03-30 09:51:32 +02:00
Thomas Patzke
c43166d5b9 Fixed log source configuration matching 2017-03-29 23:33:26 +02:00
Thomas Patzke
a22fe58ac9 Aggregation support for Splunk backend 2017-03-29 23:18:47 +02:00
Thomas Patzke
b62de742d7 Aggregation expression parsing 2017-03-29 23:17:43 +02:00
Thomas Patzke
ae5ae8f763 Verbose mode prints tokens if parsing failed 2017-03-29 22:21:40 +02:00
Thomas Patzke
9698e8fdf7 Changed Logpoint SubjectAccountName mapping to conditional mapping 2017-03-25 00:27:29 +01:00
Thomas Patzke
c978e19d88 Conditional field mappings 2017-03-25 00:21:44 +01:00
Thomas Patzke
a4465ce844 Added 1:n field mapping 
MultiFieldMapping
 2017-03-24 00:58:11 +01:00
Thomas Patzke
5009794591 Changes to field mappings 
* Introduced field mapping objects
* moved mapping from backends into parse tree generation
  (SigmaParser.parse_definition)
 2017-03-24 00:48:32 +01:00
Florian Roth
7e180365ab PowerShell Classic Log in Splunk Config Example 2017-03-22 11:17:46 +01:00