rdump

With rdump you can read, write, interact, and manipulate records from stdin or from record files saved on disk.

Hint

Don’t know yet what a record is? Read more here for a detailed explanation.

Records are the primary output type when using the various functions of target-query. Keep in mind that not all functions in target-query output records.

You can check the output type of a function by using target-query -l and searching for your function. The output type is specified on the end, and looks something like this:

runkeys - Iterate various run key locations. See source for all locations. (output: records)
                                                                           ^^^^^^^^^^^^^^^^^

If the output type is records you can use rdump!

Basic usage

Since target-query primarily outputs records it is often used in conjunction with rdump. Common usage is to pipe the record output from target-query into rdump:

$ target-query -f runkeys targets/EXAMPLE.vmx | rdump
<windows/registry/run hostname='EXAMPLE' domain='EXAMPLE.local' ts=2022-12-09 12:06:20.037806+00:00 name='OneDriveSetup' path='C:/Windows/SysWOW64/OneDriveSetup.exe /thfirstsetup' key='HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Run' hive_filepath='C:\\Windows/ServiceProfiles/LocalService/ntuser.dat' username='LocalService' user_sid='S-1-5-19' user_home='%systemroot%\\ServiceProfiles\\LocalService'>
<...>

By default, records will be serialized into a binary format when piped to a different command or file so that they can be deserialized and consumed by another command. This is how rdump interacts with the records from target-query.

Without specifying any arguments to rdump the resulting output will be grep-able text. NOTE: The same can be achieved by passing target-query the -s flag, so it’s recommended you use that if that is all you want.

You can use rdump on (multiple) files as well. Just point rdump to the record file(s) in question:

$ target-query -f runkeys targets/EXAMPLE.vmx > runkeys.rec
$ target-query -f services targets/EXAMPLE.vmx > services.rec
$ rdump runkeys.rec
<windows/registry/run hostname='EXAMPLE' domain='EXAMPLE.local' ts=2022-12-09 12:06:20.037806+00:00 name='OneDriveSetup' path='C:/Windows/SysWOW64/OneDriveSetup.exe /thfirstsetup' key='HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Run' hive_filepath='C:\\Windows/ServiceProfiles/LocalService/ntuser.dat' username='LocalService' user_sid='S-1-5-19' user_home='%systemroot%\\ServiceProfiles\\LocalService'>
[...]
$ rdump *.rec
<windows/service hostname='EXAMPLE' domain='EXAMPLE.local' ts=2016-11-21 08:06:04.528015 name='Wof' displayname='Windows Overlay File System Filter Driver' servicedll=None imagepath=None imagepath_args=None objectname=None start='Boot (0)' type='File System Driver (0x2)' errorcontrol='Normal (1)'>
<windows/registry/run hostname='EXAMPLE' domain='EXAMPLE.local' ts=2022-12-09 12:06:20.037806+00:00 name='OneDriveSetup' path='C:/Windows/SysWOW64/OneDriveSetup.exe /thfirstsetup' key='HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Run' hive_filepath='C:\\Windows/ServiceProfiles/LocalService/ntuser.dat' username='LocalService' user_sid='S-1-5-19' user_home='%systemroot%\\ServiceProfiles\\LocalService'>
[...]

The file extension of the record file can be anything you want.

Timeline & datetime fields

target-query functions that have a record with the fieldtype datetime are outputed in a single record. As shown below with the function mft:

$ target-query -f mft -t targets/EXAMPLE.tar --limit 1 | rdump
<filesystem/ntfs/mft/std hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411+00:00 last_modification_time=2019-03-19 21:52:25.169411+00:00 last_change_time=2019-03-19 21:52:25.169411+00:00 last_access_time=2019-03-19 21:52:25.169411+00:00 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>

The single record contains four different events that occured on the filesystem:

  • creation_time

  • last_modification_time

  • last_change_time

  • last_access_time

To analyze a timeline of events that occured every record needs a single datetime field on which can be filtered to view records in chronological order.

For this purpose the argument --multi-timestamp can be used to output multiple ts enriched records based on the datetime fields of the original record.

$ target-query -f mft -t targets/EXAMPLE.tar --limit 1 | rdump --multi-timestamp
[reading from stdin]
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='creation_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='last_modification_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='last_change_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>
<filesystem/ntfs/mft/std ts=2019-03-19 21:52:25.169411 ts_description='last_access_time' hostname='MSEDGEWIN10' domain=None creation_time=2019-03-19 21:52:25.169411 last_modification_time=2019-03-19 21:52:25.169411 last_change_time=2019-03-19 21:52:25.169411 last_access_time=2019-03-19 21:52:25.169411 segment=0 path='c:/$MFT' owner='S-1-5-18' filesize=0.12 GB resident=False inuse=True volume_uuid=None>

Filtering & manipulating records

One of the things you can do with rdump is filtering records. This can be done with the -s or --selector argument. This argument takes a Python statement which must evaluate to True or False. This statement is used to filter records. You can interact with the fields of a record using the magic r variable.

$ rdump services.rec -s '"%systemroot%" not in r.imagepath.lower()'
<windows/service hostname='EXAMPLE' domain='EXAMPLE.local' ts=2021-06-01 12:55:07.103594 name='ACPI' displayname='@acpi.inf,%ACPI.SvcDesc%;Microsoft ACPI Driver' servicedll=None imagepath='System32/drivers/ACPI.sys' imagepath_args='' objectname=None start='Boot (0)' type='Kernel Device Driver (0x1)' errorcontrol='Critical (3)'>
[...]

To manipulate the output generated by rdump the -f (--format) and -F (--fields) arguments can be used. The -f behaves as a Python format-string, record fields can be referred to by their field name in braces (e.g. {path}). For example, we can output just the hostname, name and image path of a Windows service:

$ rdump services.rec -f '{hostname} - {name}:{imagepath}'
EXAMPLE - 1394ohci:/SystemRoot/System32/drivers/1394ohci.sys
EXAMPLE - 3ware:System32/drivers/3ware.sys
EXAMPLE - ACPI:System32/drivers/ACPI.sys
EXAMPLE - AcpiDev:/SystemRoot/System32/drivers/AcpiDev.sys
[...]

See also

Please refer to Use-cases for more examples of how to use rdump.

Writing records

rdump can write records to a file, which can be used as input for rdump at a later stage. To write records to a file, the -w or --writer argument can be used.

The output format and compression type are automatically detected based on the filename extension. For example, to write to a gzip compressed file, simply use the .gz extension in your output file.

$ rdump services.rec -w services.rec.gz

This will read the records from services.rec and write them to a new gzip compressed file named services.rec.gz. Other supported compression formats are .bz2, .zst (zstandard), and .lz4.

If you want to write the records to a file without any compression, just use a filename without a compression extension.

$ rdump services.rec -w services.rec.out

When the -w argument is omitted, rdump prints the string representation of the records to standard output, which is useful for piping to tools like grep or less.

If you want to output the record output to another rdump process you need to use -w -, which will write the records in binary stream format to standard output.

For example, you can chain rdump with common Linux command-line tools to analyze records. In this example, we extract image paths from a record source, sort them, count occurrences, and display the top 5 most common paths:

$ rdump services.rec -w - | rdump -f "{imagepath}" | sort | uniq -c | sort -rn | head -n 5
[reading from stdin]
 104 %SystemRoot%\system32\svchost.exe
  71 %SystemRoot%\System32\svchost.exe
  28 %systemroot%\system32\svchost.exe
  12 None
   3 %SystemRoot%\system32\lsass.exe

Usage

rdump - CLI interface

Record dumper, a tool that can read, write and filter records

rdump [-h] [--version] [-v] [-a] [-l] [-n] [--record-source RECORD_SOURCE]
      [--record-classification RECORD_CLASSIFICATION] [-F FIELDS] [-X FIELDS] [-s SELECTOR]
      [-f FORMAT] [-c COUNT] [--skip COUNT] [-w OUTPUT]
      [-m {csv,csv-no-header,json,jsonlines,line,line-verbose}] [--split COUNT]
      [--suffix-length LEN] [--multi-timestamp] [-p] [-t TOTAL] [--stats]
      [-E EXEC_EXPRESSION] [-j] [-J] [-C] [-L] [-Lv] [-Cn]
      [SOURCE ...]

rdump positional arguments

  • SOURCE - Record source (default: ['-'])

rdump options

  • -h, --help - show this help message and exit

  • --version - show program’s version number and exit

  • -v, --verbose - Increase verbosity (default: 0)

rdump miscellaneous

  • -a, --list-adapters - List (un)available adapters that can be used for reading or writing

  • -l, --list - List unique Record Descriptors

  • -n, --no-compile - Don’t use a compiled selector (safer, but slower)

  • --record-source RECORD_SOURCE - Overwrite the record source field (default: None)

  • --record-classification RECORD_CLASSIFICATION - Overwrite the record classification field (default: None)

rdump selection

  • -F FIELDS, --fields FIELDS - Fields (comma seperated) to output in dumping (default: None)

  • -X FIELDS, --exclude FIELDS - Fields (comma seperated) to exclude in dumping (default: None)

  • -s SELECTOR, --selector SELECTOR - Only output records matching Selector (default: None)

rdump output control

  • -f FORMAT, --format FORMAT - Format string (default: None)

  • -c COUNT, --count COUNT - Exit after COUNT records (default: None)

  • --skip COUNT - Skip the first COUNT records (default: 0)

  • -w OUTPUT, --writer OUTPUT - Write records to output (default: None)

  • -m MODE, --mode MODE - Output mode (default: None)

  • --split COUNT - Write record files smaller than COUNT records (default: None)

  • --suffix-length LEN - Generate suffixes of length LEN for splitted output files (default: 2)

  • --multi-timestamp - Create records for datetime fields

  • -p, --progress - Show progress bar (requires tqdm)

  • -t TOTAL, --total TOTAL - The number of expected records, used for progress bar (requires tqdm) (default: None)

  • --stats - Show count of processed records

rdump advanced

  • -E EXEC_EXPRESSION, --exec-expression EXEC_EXPRESSION - execute a (Python) expression for each record AFTER selector matching, can be used to assign new fields (default: None)

rdump aliases