valitydev/SigmaHQ
14
0
Fork 0
mirror of https://github.com/valitydev/SigmaHQ.git synced 2024-11-07 09:48:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,473 Commits 20 Branches 25 Tags 21 MiB
e79e99c4aa
Commit Graph

556 Commits

Author SHA1 Message Date
milkmix
37ee355a77 patched es-dsl 2018-05-17 08:44:50 +02:00
Thomas Patzke
738d03c751 Fixed position of line separation if rulecomment and verbose is active 2018-05-13 22:36:51 +02:00
Thomas Patzke
f60e7e125f Sigma tools release 0.4 
* Various bug fixes in quoting of specific characters
* New backend es-dsl
 2018-05-01 00:50:07 +02:00
Thomas Patzke
7647587a8b Fixed quoting of backslashes in generated queries 2018-05-01 00:45:59 +02:00
Thomas Patzke
de2ed08695 Merge branch 'ci-es' 2018-05-01 00:34:11 +02:00
Thomas Patzke
e411039b56 Fixed escaping of \u in Elasticsearch Query String queries 2018-05-01 00:05:16 +02:00
Thomas Patzke
aeda30a389 Python rewrite of es-qs query test 2018-04-11 23:59:44 +02:00
milkmix
0b3b0c3aaf imported es-dsl code from repo 2018-04-06 17:36:11 +02:00
Thomas Patzke
4183b1b59e Sigma tools release 0.3.3 2018-03-29 11:17:03 +02:00
Thomas Patzke
22ee6f4521 sigmac: escaped wildcards (\* and \?) are passed in generated query 2018-03-29 11:15:20 +02:00
Thomas Patzke
17c1c1adff Added field name mappings to HELK configuration 2018-03-27 14:41:02 +02:00
Thomas Patzke
a3e02ea70f Various rule fixes 
* Field name: LogonProcess -> LogonProcessName
* Field name: Message -> AuditPolicyChanges
* Field name: ProcessCommandLine -> CommandLine
* Removed Type match in Kerberos RC4 encryption rule
  Problematic because text representation not unified and audit failures are possibly interesting events
* Removed field 'Severity' from rules (Redundant)
* Rule decomposition of win_susp_failed_logons_single_source) because of different field names
* Field name: SubjectAccountName -> SubjectUserName
* Field name: TargetProcess -> TargetImage
* Field name: TicketEncryption -> TicketEncryptionType
* Field name: TargetFileName -> TargetFilename
 2018-03-27 14:35:49 +02:00
Thomas Patzke
5f8b60cc24 sigmac: Improved fieldlist backend 
* Unique list of fields for multiple rules
* Aggregation support
 2018-03-22 00:03:51 +01:00
Thomas Patzke
5c0f811f4a Sigma tools release 0.3.2 2018-03-21 01:15:19 +01:00
Thomas Patzke
0018503501 sigmac: Fixed rulecommend backend option 2018-03-21 01:13:10 +01:00
Thomas Patzke
7360a68741 Sigma tools release 0.3.1 2018-03-21 00:59:23 +01:00
Thomas Patzke
4a9849b161 sigmac: improved backend options 
* parsing in main class
* help
 2018-03-21 00:53:44 +01:00
Thomas Patzke
bd20ffdad9 sigmac/kibana: curl URL quoted 2018-03-21 00:22:00 +01:00
Thomas Patzke
3f5f3a8d50 sigmac: Remove problematic characters from rule identifiers 2018-03-17 00:44:50 +01:00
Thomas Patzke
f6858c436a sigmac: Kibana curl output generates one index pattern line per pattern 2018-03-16 23:53:12 +01:00
Thomas Patzke
578118315c Merge branch 'devel-sigmac' into helk 2018-03-16 23:48:13 +01:00
Thomas Patzke
e162ba0155 Added HELK configuration 2018-03-16 23:42:31 +01:00
Thomas Patzke
13ec4c3e3b sigmac: Kibana curl importer script 2018-03-11 00:25:12 +01:00
Thomas Patzke
54d9e52527 Sigma tools release 0.3 2018-03-06 23:21:13 +01:00
Thomas Patzke
3b8b04fe09 Merge branch 'devel-sigmac' 2018-03-06 23:19:45 +01:00
Thomas Patzke
7141729ffc sigma/parser: Introduced new conditions 
* Any definition: 1 of them
* All definitions: all of them
* Any of selected definitions: 1 of def* (wildcard)
* All of selected definitions: all of def* (wildcard)
 2018-03-06 23:13:42 +01:00
Thomas Patzke
5a97befea0 Sigma tools release 0.2 2018-03-04 23:03:19 +01:00
Thomas Patzke
647fc6187a sigmac: Added proper 'Content-Type' header for xpack-watcher backend 2018-03-04 22:58:15 +01:00
Thomas Patzke
89aa300bbc Improved xpack-watcher actions 
* Log and mail
* Details in message
 2018-02-09 00:03:41 +01:00
Thomas Patzke
8336929d76 XPack Watcher Backend: Improved aggregation capabilities 
* Aggregation with "...count(field)...", "...by field..." and
  combination of both
* Still only count() supported
 2018-02-08 22:17:35 +01:00
Thomas Patzke
4762a1cc30 Removed abandoned SigmaAggregationParser.trans_timeframe() method 2018-02-05 23:30:00 +01:00
Thomas Patzke
ec3f0f6d60 Fixed before/after logic 
If nothing was generated "None" was printed.
 2018-02-01 22:49:02 +01:00
Thomas Patzke
76bdcba71f Added rulecomment option to all single-query output backends 
Prints comment with rule before output.
 2018-01-27 23:48:10 +01:00
Thomas Patzke
7708a538f4 New PyPI release 2017-12-14 22:40:31 +01:00
Thomas Patzke
fc2dd90aaf Skipping dotfiles 2017-12-14 22:39:51 +01:00
Thomas Patzke
497496fdf1 New release 2017-12-13 00:28:50 +01:00
Thomas Patzke
f3d19f394e Fixed encoding issues 
Some OS environments don't use UTF-8 as default encoding. Enforced it
for output files and stdout.
 2017-12-13 00:12:56 +01:00
Thomas Patzke
19cc299c57 Added PyPI README 2017-12-09 22:13:25 +01:00
Thomas Patzke
fd7b7bb438 Fixed build 
Reference to main README
 2017-12-09 08:57:51 +01:00
Thomas Patzke
da9127276c PyPI release documentation 2017-12-09 00:23:34 +01:00
Thomas Patzke
d6526387d3 Renamed PyPI package 2017-12-09 00:15:34 +01:00
Thomas Patzke
d82a78fa3d Finalizing PyPI release 
* Removed .py suffix from command line tools
* sigmac tells when it does nothing and prints usage notice
* Makefile upload target
* minor changes
 2017-12-08 23:50:08 +01:00
Thomas Patzke
09d40ab2da Finished packaging and refactoring 2017-12-08 22:32:39 +01:00
Thomas Patzke
68d8afe4e6 Intermediate refactoring commit: moving code into package 
Further splitting sigma.py into smaller parts.
 2017-12-08 21:45:05 +01:00
Thomas Patzke
11f52b981b Merge branch 'lgpl' into packaging 2017-12-08 17:15:23 +01:00
Thomas Patzke
764e064f8c First (untested) packaging 2017-12-08 00:32:41 +01:00
Thomas Patzke
2ce0be1f2d Re-licensing toolchain under LGPLv3 
Thanks to Ben de Haan and Devin Ferguson for permission for this change.
 2017-12-07 21:55:43 +01:00
Thomas Patzke
3b9ff57a38 Added merge_sigma tool 
* Tests
* Restructured Makefile
 2017-11-14 22:17:18 +01:00
Thomas Patzke
f478cffb41 Added default index configs for usual ELK setups 
* Added test case for defaultindex with kibana backend
 2017-11-09 10:05:41 +01:00
Thomas Patzke
46f1ce35a8 sigmac/kibana backend: added index fallback if none determined 2017-11-09 10:02:23 +01:00
Florian Roth
1bea284280 Added Windows Driver Framework log source to configs 2017-11-09 08:42:58 +01:00
Florian Roth
e83e3a0c07 Bugfixes in Splunk config 2017-11-09 08:41:07 +01:00
Thomas Patzke
b03f9359ec sigmac: Added rule filter 2017-11-02 00:02:15 +01:00
Thomas Patzke
732f01878f Sigma rule collection YAML action documents 2017-11-01 00:17:55 +01:00
Thomas Patzke
d0b2bd9875 Multiple rules per file 
* New wrapper class SigmaCollectionParser parses all YAML documents
  contained in file and handles multiple SigmaParser instantiation.
* Exemplary extended one security/4688 rule to security/4688 + sysmon/1
 2017-10-31 23:06:18 +01:00
Thomas Patzke
5743e25931 Added logging framework 2017-10-31 22:13:20 +01:00
Thomas Patzke
720c992573 Dropped within keyword 
Covered by timeframe attribute.

Fixes issue #26.
 2017-10-30 00:25:56 +01:00
Thomas Patzke
012cb6227f Added proper handling of null/not null values 
Fixes issue #25
 2017-10-29 23:57:39 +01:00
Thomas Patzke
5fa9e685b1 Splitted parts of generate to generateQuery in backend code 2017-10-25 00:03:03 +02:00
Thomas Patzke
6d0e85fcfa Fixed Splunk backend (#50) 2017-10-24 23:48:47 +02:00
Thomas Patzke
65e1f8ec2b Increased test coverage 
* more tests
* removed unneeded code
* increased coverage fail threshold
 2017-10-23 23:30:44 +02:00
Thomas Patzke
3389656a5b Added ELK default index config 2017-10-23 00:45:33 +02:00
Thomas Patzke
7f93d3ca47 Kibana backend throws exception when multiple indices appear 
* Introduced backend errors with handling in sigmac
 2017-10-23 00:45:01 +02:00
Thomas Patzke
cb9aeac7d9 Added default index handling 
* Removed default index handling from backend code
* Added default indices to config templates
 2017-10-23 00:08:39 +02:00
Thomas Patzke
ec996e7353 Improved test coverage 2017-10-19 17:42:56 +02:00
Thomas Patzke
5449a12a14 Added GrepBackend 
Moved field quoting/filtering into QuoteCharMixin
 2017-10-18 19:03:38 +02:00
Thomas Patzke
54cf9af0c9 Removed ELK Sysmon config 
It's contained in ELK Windows config
 2017-10-18 15:23:55 +02:00
Thomas Patzke
b8eedfe3f0 Fixes and refactoring of KibanaBackend and XPackWatcherBackend 
* Moved unnecessary code out of condition loop
* Index specific rule-name not appended to rulename variable used later
  from other rule/index.
* Merged condition loop
 2017-09-30 23:22:05 +02:00
Thomas Patzke
1d314e326e sigmac: MultiRuleOutputMixin 
* Moved rule name generation into mixin
* KibanaBackend and XPackWatcherBackend now use this mixin instead of
  doing the same thing in both classes.
 2017-09-30 01:03:08 +02:00
Thomas Patzke
b47e3e45a8 Merge branch 'devel-sigmac' 2017-09-22 00:31:22 +02:00
Thomas Patzke
d410adb397 sigmac: X-Pack Watcher backend improvements 
* Renamed backend class according to convention
* Output types: curl (default) and plain
* Prefix of rule names
* Indices from configuration
* Support for multiple conditions per rule
* Usage of parsed condition
* Support for all condition operators
* Fixed bug preventing from passing multiple options to backend
* Added to CI tests
 2017-09-22 00:28:35 +02:00
Thomas Patzke
62eb3b2923 Merge branch 'devel-sigmac' of https://github.com/megadevx/sigma into devel-sigmac-watcher 2017-09-19 23:08:04 +02:00
Thomas Patzke
545e05370f Added first config for logstash-linux project 
URL: https://github.com/thomaspatzke/logstash-linux
 2017-09-17 00:36:04 +02:00
Thomas Patzke
a18b8eca52 sigmac: changed backend description for kibana backend 2017-09-17 00:31:25 +02:00
Thomas Patzke
270ab9ba78 Added backend options 
* generic support for backend-specific options
* kibana backend option for title prefix
 2017-09-16 23:46:40 +02:00
Thomas Patzke
c8a66e48b6 sigmac: improved Kibana backend 
* added fields from rules
* default index if none is matching
 2017-09-16 00:39:37 +02:00
Thomas Patzke
d3201229b0 sigmac: Fixed matching of log sources between rules and configuration 2017-09-16 00:32:31 +02:00
devife
9bc8e12a4f Created a X-Pack Watcher output. 
This is has only been tested slightly.
 2017-09-15 09:49:57 -05:00
devife
135e389334 Created a X-Pack Watcher output. 
This is has only been tested slightly.
 2017-09-15 09:46:37 -05:00
Thomas Patzke
e5da26578d sigmac/kibana backend: index names from configuration 2017-09-11 00:30:01 +02:00
Thomas Patzke
77a3e7ed91 Code cleanup 2017-09-11 00:27:14 +02:00
Thomas Patzke
be3c0cfb89 sigmac: Kibana backend, first version 
* totally untested!
* only supports searches
* no visualizations/aggregation expressions
* some fields are filled with default values (see code comments)
 2017-09-05 00:14:13 +02:00
Thomas Patzke
c5fc74f440 Further backend changes 
* backends get complete SigmaParser objects instead of condition
* addition of finalize step for backends
* Renaming of output classes
 2017-09-04 00:56:04 +02:00
Thomas Patzke
39381305d8 sigmac: Generic Text File Output 
Moved output logic into generic class.
 2017-08-29 00:05:59 +02:00
Florian Roth
edf2787402 Removed some spaces and added Win 10 WMI eventlog 2017-08-22 10:04:56 +02:00
Thomas Patzke
487ab99507 Changed sigmac error behavior on I/O errors 2017-08-07 08:54:18 +02:00
Thomas Patzke
d84f9dcc1c Aggregation 'near' raises NotImplementedError in backends splunk and logpoint 2017-08-05 23:48:28 +02:00
Thomas Patzke
f5b07dc9af Added semantic parsing of near expressions 2017-08-05 00:28:22 +02:00
Thomas Patzke
d17604d007 Merge branch 'master' into travis-test 2017-08-03 00:11:08 +02:00
Thomas Patzke
5706361464 Parsing of "near ... within" aggregation operator 
* Operator is only parsed. No processing or passing of parsed data to
  backends.
* Changed rule sysmon_mimikatz_inmemory_detection.yml accordingly.
 2017-08-03 00:05:48 +02:00
Thomas Patzke
52525236a5 sigmac: added parameter to control error behavior 
* --defer-abort
* --ignore-not-implemented
 2017-08-02 00:56:22 +02:00
Thomas Patzke
3495bac9cb sigmac: return error codes 2017-07-31 00:31:49 +02:00
Ben de Haan
43c4486de0 Added LogPoint aggregation 
Added generateAggregation function for LogPoint
 2017-06-19 15:21:29 +02:00
Florian Roth
c1f5bd1540 Sigmac bugfix: showing faulty condition 2017-06-12 10:07:15 +02:00
Thomas Patzke
9d49daecea Restructured backends 
Moved most logic into generic base class SingleTextQueryBackend which is
configured by class variables.
 2017-06-02 23:43:45 +02:00
Thomas Patzke
6a29884615 Structured backends module with comments 2017-05-26 23:42:49 +02:00
Thomas Patzke
998bb0079d Fixed Splunk config for sigmac again 2017-05-26 22:40:06 +02:00
Thomas Patzke
18a9fd18ef Fixed Splunk configuration 
Substituted source: with sourcetype:
 2017-05-26 00:13:30 +02:00
Florian Roth
f66085b198 Added eventlog source DNS Server to configs 2017-05-08 13:09:17 +02:00
Thomas Patzke
05e9d1e1e9 Check if aggregation is present in BaseBackend 
Caused NotImplementedError in ElasticsearchQueryStringBackend.
 2017-04-17 00:11:20 +02:00
Ben de Haan
dddb83393d Added field mappings for events with logins 2017-03-30 10:49:36 +02:00
Ben de Haan
cb9a9bc2ff Added LogPoint conditional username mapping 
Conditional mapping of SubjectAccountName based on EventID. Not a comprehensive list, but should include most relevant Event IDs.
 2017-03-30 09:51:32 +02:00
Thomas Patzke
c43166d5b9 Fixed log source configuration matching 2017-03-29 23:33:26 +02:00
Thomas Patzke
a22fe58ac9 Aggregation support for Splunk backend 2017-03-29 23:18:47 +02:00
Thomas Patzke
b62de742d7 Aggregation expression parsing 2017-03-29 23:17:43 +02:00
Thomas Patzke
ae5ae8f763 Verbose mode prints tokens if parsing failed 2017-03-29 22:21:40 +02:00
Thomas Patzke
9698e8fdf7 Changed Logpoint SubjectAccountName mapping to conditional mapping 2017-03-25 00:27:29 +01:00
Thomas Patzke
c978e19d88 Conditional field mappings 2017-03-25 00:21:44 +01:00
Thomas Patzke
a4465ce844 Added 1:n field mapping 
MultiFieldMapping
 2017-03-24 00:58:11 +01:00
Thomas Patzke
5009794591 Changes to field mappings 
* Introduced field mapping objects
* moved mapping from backends into parse tree generation
  (SigmaParser.parse_definition)
 2017-03-24 00:48:32 +01:00
Florian Roth
7e180365ab PowerShell Classic Log in Splunk Config Example 2017-03-22 11:17:46 +01:00
Ben de Haan
c3c405a95e LogPoint windows mapping 2017-03-20 16:57:19 +01:00
Thomas Patzke
1bf11dc471 Merge pull request #17 from benno001/master 
Fixed LogPoint list behaviour
 2017-03-20 08:58:16 +01:00
Ben de Haan
c94b539b14 Fixed LogPoint list behaviour 2017-03-20 08:41:29 +01:00
Thomas Patzke
d0bed75eb9 Added --output/-o parameter to sigmac 2017-03-18 23:15:03 +01:00
Florian Roth
f34156138f Bugfix - Index 2017-03-18 13:57:42 +01:00
Florian Roth
8403e8072c Merge pull request #14 from benno001/master 
Added LogPoint backend
 2017-03-18 13:30:35 +01:00
Florian Roth
f292a259a5 Adjusted Windows Splunk Config 2017-03-18 13:12:31 +01:00
Ben de Haan
d18751a0ea Added LogPoint backend 2017-03-18 11:12:06 +01:00
Thomas Patzke
17c484163d Improved examples 2017-03-18 00:03:21 +01:00
Thomas Patzke
b4f52d9cfb Windows index in Splunk example configuration 2017-03-17 23:30:11 +01:00
Thomas Patzke
b865a858aa Generation of conditions for configured indices 2017-03-17 23:28:06 +01:00
Florian Roth
dc00baacda Splunk Windows Configuration Example 2017-03-17 10:00:56 +01:00
Thomas Patzke
d2a9a91175 Log source conditions are integrated in generated expressions 
Indices not yet included
 2017-03-14 23:22:32 +01:00
Thomas Patzke
52d7e9fc07 Parsing log sources in configuration files 2017-03-12 23:12:21 +01:00
Thomas Patzke
12e825783b Merge branch 'master' into devel-sigmac 2017-03-11 23:49:56 +01:00
Thomas Patzke
63e23af63c Merge branch 'devel-sigmac-config' into devel-sigmac 2017-03-11 23:49:41 +01:00
Florian Roth
ad9f73a178 Merge branch 'devel-sigmac' 2017-03-07 10:49:03 +01:00
Florian Roth
b93379a6a9 Config example: sysmon / logstash index 2017-03-07 10:09:43 +01:00
Florian Roth
cd445f8ae9 Bugfix: non-recursive list not pathlib.Path elements but strings 2017-03-07 09:41:46 +01:00
Thomas Patzke
dae88fbcfa Error and warning messages are printed to stderr 2017-03-06 23:01:33 +01:00
Thomas Patzke
d1030ec053 Fieldlist backend 
Lists all fields used in given rules.
 2017-03-06 22:47:30 +01:00
Thomas Patzke
05df298d45 Field mappings 2017-03-06 22:07:04 +01:00
Thomas Patzke
6ddc15c972 Merge branch 'devel-sigmac' into devel-sigmac-config 2017-03-06 21:32:58 +01:00
Thomas Patzke
896b8fb56e Finished path recursion 2017-03-06 21:26:56 +01:00
Florian Roth
da6c5c19ae Update README.md 2017-03-06 09:37:44 +01:00
Florian Roth
362ff157ba Update README.md 2017-03-06 09:37:31 +01:00
Florian Roth
df39dee702 Sigmac recursive feature 2017-03-06 09:36:24 +01:00
Thomas Patzke
8864647e04 Parsing of sigmac configuration files 
* field mappings
* log sources
 2017-03-05 23:44:52 +01:00
Thomas Patzke
f092333bb4 Sigmac configuration parsing 2017-03-05 00:56:45 +01:00
Thomas Patzke
4aaa22fd6d Made not implemented sigmac features obvious 
* added notes to help message
* error if not implemented option is used
 2017-03-04 23:36:46 +01:00
Florian Roth
47bfe82cc4 Splunk specifics 2017-03-04 10:37:40 +01:00
Florian Roth
9971192bff Create README.md 2017-03-03 13:45:55 +01:00
Florian Roth
b984d83685 Typo in help text 2017-03-03 12:47:20 +01:00
Thomas Patzke
8f3541f0a0 Added Splunk backend 2017-03-02 23:34:12 +01:00
Thomas Patzke
2dd1c7cd12 Deactivated not implemented backends 2017-03-02 22:55:45 +01:00
Thomas Patzke
9556e73cd1 Fix: automatic escaping of * and ? in es-qs backend removed 2017-03-02 12:07:07 +01:00
Thomas Patzke
10ee9c64fe Moved node output into dedicated backend class methods 2017-03-01 21:47:51 +01:00
Thomas Patzke
0d470af0e7 Set sigmac default backend to 'es-qs' 2017-03-01 09:40:51 +01:00
Thomas Patzke
e0f813ebbb Conversion to Elasticsearch Query Strings 
First version of sigmac that converts Sigma YAMLs without aggregations
into ES Query Strings suitable for Kibana or other tools.
 2017-03-01 00:03:34 +01:00
Thomas Patzke
58f2118ef4 Parsing of search expressions 
* Tokenization
* Building a parse tree
* Aggregations not yet implemented
 2017-02-24 23:36:19 +01:00
Thomas Patzke
ec9f42410a Intermediate backup state: Parsing of most conditions 
* Conditions with parentheses cause exceptions
 2017-02-22 22:43:35 +01:00
Thomas Patzke
0543ef7e75 sigmac: Condition Tokenizer 2017-02-16 23:58:44 +01:00
Thomas Patzke
ce43dce7ef Parsing of detections 
Transformation of detections into internal data structures. Parsing must
be changed later to on-demand parsing because condition can change
default behavior of lists.
 2017-02-16 00:40:08 +01:00
Thomas Patzke
980ed9c5c7 Moved YAML parsing in SigmaParser class 2017-02-13 23:31:42 +01:00
Thomas Patzke
1498d787e7 Added Sigma converter skeleton 
* YAML parsing
* argument parsing
* empty backend classes
 2017-02-13 23:28:53 +01:00