rdump

With rdump you can read, write, interact, and manipulate records from stdin or from record files saved on disk.

Hint

Don’t know yet what a record is? Read more here for a detailed explanation.

Records are the primary output type when using the various functions of target-query. Keep in mind that not all functions in target-query output records.

You can check the output type of a function by using target-query -l and searching for your function. The output type is specified on the end, and looks something like this:

runkeys - Iterate various run key locations. See source for all locations. (output: records)
                                                                           ^^^^^^^^^^^^^^^^^

If the output type is records you can use rdump!

Basic usage

Since target-query primarily outputs records it is often used in conjunction with rdump. Common usage is to pipe the record output from target-query into rdump:

$ target-query -f runkeys targets/EXAMPLE.vmx | rdump
<windows/registry/run hostname='EXAMPLE' domain='EXAMPLE.local' ts=2022-12-09 12:06:20.037806+00:00 name='OneDriveSetup' path='C:/Windows/SysWOW64/OneDriveSetup.exe /thfirstsetup' key='HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Run' hive_filepath='C:\\Windows/ServiceProfiles/LocalService/ntuser.dat' username='LocalService' user_sid='S-1-5-19' user_home='%systemroot%\\ServiceProfiles\\LocalService'>
<...>

By default, records will be serialized into a binary format when piped to a different command or file so that they can be deserialized and consumed by another command. This is how rdump interacts with the records from target-query.

Without specifying any arguments to rdump the resulting output will be grep-able text. NOTE: The same can be achieved by passing target-query the -s flag, so it’s recommended you use that if that is all you want.

You can use rdump on (multiple) files as well. Just point rdump to the record file(s) in question:

$ target-query -f runkeys targets/EXAMPLE.vmx > runkeys.rec
$ target-query -f services targets/EXAMPLE.vmx > services.rec
$ rdump runkeys.rec
<windows/registry/run hostname='EXAMPLE' domain='EXAMPLE.local' ts=2022-12-09 12:06:20.037806+00:00 name='OneDriveSetup' path='C:/Windows/SysWOW64/OneDriveSetup.exe /thfirstsetup' key='HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Run' hive_filepath='C:\\Windows/ServiceProfiles/LocalService/ntuser.dat' username='LocalService' user_sid='S-1-5-19' user_home='%systemroot%\\ServiceProfiles\\LocalService'>
[...]
$ rdump *.rec
<windows/service hostname='EXAMPLE' domain='EXAMPLE.local' ts=2016-11-21 08:06:04.528015 name='Wof' displayname='Windows Overlay File System Filter Driver' servicedll=None imagepath=None imagepath_args=None objectname=None start='Boot (0)' type='File System Driver (0x2)' errorcontrol='Normal (1)'>
<windows/registry/run hostname='EXAMPLE' domain='EXAMPLE.local' ts=2022-12-09 12:06:20.037806+00:00 name='OneDriveSetup' path='C:/Windows/SysWOW64/OneDriveSetup.exe /thfirstsetup' key='HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Run' hive_filepath='C:\\Windows/ServiceProfiles/LocalService/ntuser.dat' username='LocalService' user_sid='S-1-5-19' user_home='%systemroot%\\ServiceProfiles\\LocalService'>
[...]

The file extension of the record file can be anything you want.

Timeline & datetime fields

target-query functions that have a record with the fieldtype datetime are outputed in a single record. As shown below with the function mft:

$ target-query -f mft -t targets/EXAMPLE.tar --limit 1 | rdump
<filesystem/ntfs/mft/std hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411+00:00 last_modification_time=2019-03-19 21:52:25.169411+00:00 last_change_time=2019-03-19 21:52:25.169411+00:00 last_access_time=2019-03-19 21:52:25.169411+00:00 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>

The single record contains four different events that occured on the filesystem:

  • creation_time

  • last_modification_time

  • last_change_time

  • last_access_time

To analyze a timeline of events that occured every record needs a single datetime field on which can be filtered to view records in chronological order.

For this purpose the argument --multi-timestamp can be used to output multiple ts enriched records based on the datetime fields of the original record.

$ target-query -f mft -t targets/EXAMPLE.tar --limit 1 | rdump --multi-timestamp
[reading from stdin]
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='creation_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='last_modification_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='last_change_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='last_access_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>

Filtering & manipulating records

One of the things you can do with rdump is filtering records. This can be done with the -s or --selector argument. This argument takes a Python statement which must evaluate to True or False. This statement is used to filter records. You can interact with the fields of a record using the magic r variable.

$ rdump services.rec -s '"%systemroot%" not in r.imagepath.lower()'
<windows/service hostname='EXAMPLE' domain='EXAMPLE.local' ts=2021-06-01 12:55:07.103594 name='ACPI' displayname='@acpi.inf,%ACPI.SvcDesc%;Microsoft ACPI Driver' servicedll=None imagepath='System32/drivers/ACPI.sys' imagepath_args='' objectname=None start='Boot (0)' type='Kernel Device Driver (0x1)' errorcontrol='Critical (3)'>
[...]

To manipulate the output generated by rdump the -f (--format) and -F (--fields) arguments can be used. The -f behaves as a Python format-string, record fields can be referred to by their field name in braces (e.g. {path}). For example, we can output just the hostname, name and image path of a Windows service:

$ rdump services.rec -f '{hostname} - {name}:{imagepath}'
EXAMPLE - 1394ohci:/SystemRoot/System32/drivers/1394ohci.sys
EXAMPLE - 3ware:System32/drivers/3ware.sys
EXAMPLE - ACPI:System32/drivers/ACPI.sys
EXAMPLE - AcpiDev:/SystemRoot/System32/drivers/AcpiDev.sys
[...]

See also

Please refer to Use-cases for more examples of how to use rdump.

Writing records

Something about writing records, e.g. auto detection of filename for compression.

Usage

rdump - CLI interface

Record dumper, a tool that can read, write and filter records

rdump [-h] [--version] [-v] [-a] [-l] [-n] [--record-source RECORD_SOURCE]
      [--record-classification RECORD_CLASSIFICATION] [-F FIELDS] [-X FIELDS] [-s SELECTOR]
      [-f FORMAT] [-c COUNT] [--skip COUNT] [-w OUTPUT]
      [-m {csv,csv-no-header,json,jsonlines,line,line-verbose}] [--split COUNT]
      [--suffix-length LEN] [--multi-timestamp] [-p] [-t TOTAL] [--stats]
      [-E EXEC_EXPRESSION] [-j] [-J] [-C] [-L] [-Lv] [-Cn]
      [SOURCE ...]

rdump positional arguments

  • SOURCE - Record source (default: ['-'])

rdump options

  • -h, --help - show this help message and exit

  • --version - show program’s version number and exit

  • -v, --verbose - Increase verbosity (default: 0)

rdump miscellaneous

  • -a, --list-adapters - List (un)available adapters that can be used for reading or writing

  • -l, --list - List unique Record Descriptors

  • -n, --no-compile - Don’t use a compiled selector (safer, but slower)

  • --record-source RECORD_SOURCE - Overwrite the record source field (default: None)

  • --record-classification RECORD_CLASSIFICATION - Overwrite the record classification field (default: None)

rdump selection

  • -F FIELDS, --fields FIELDS - Fields (comma seperated) to output in dumping (default: None)

  • -X FIELDS, --exclude FIELDS - Fields (comma seperated) to exclude in dumping (default: None)

  • -s SELECTOR, --selector SELECTOR - Only output records matching Selector (default: None)

rdump output control

  • -f FORMAT, --format FORMAT - Format string (default: None)

  • -c COUNT, --count COUNT - Exit after COUNT records (default: None)

  • --skip COUNT - Skip the first COUNT records (default: 0)

  • -w OUTPUT, --writer OUTPUT - Write records to output (default: None)

  • -m MODE, --mode MODE - Output mode (default: None)

  • --split COUNT - Write record files smaller than COUNT records (default: None)

  • --suffix-length LEN - Generate suffixes of length LEN for splitted output files (default: 2)

  • --multi-timestamp - Create records for datetime fields

  • -p, --progress - Show progress bar (requires tqdm)

  • -t TOTAL, --total TOTAL - The number of expected records, used for progress bar (requires tqdm) (default: None)

  • --stats - Show count of processed records

rdump advanced

  • -E EXEC_EXPRESSION, --exec-expression EXEC_EXPRESSION - execute a (Python) expression for each record AFTER selector matching, can be used to assign new fields (default: None)

rdump aliases