:py:mod:`flow.record.selector` ============================== .. py:module:: flow.record.selector Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: flow.record.selector.NoneObject flow.record.selector.SelectorResult flow.record.selector.Selector flow.record.selector.WrappedRecord flow.record.selector.CompiledSelector flow.record.selector.TypeMatcher flow.record.selector.TypeMatcherInstance flow.record.selector.RecordContextMatcher Functions ~~~~~~~~~ .. autoapisummary:: :nosignatures: flow.record.selector.lower flow.record.selector.upper flow.record.selector.names flow.record.selector.name flow.record.selector.get_type flow.record.selector.has_field flow.record.selector.field_regex flow.record.selector.field_equals flow.record.selector.field_contains flow.record.selector.resolve_attr_path flow.record.selector.make_selector Attributes ~~~~~~~~~~ .. autoapisummary:: flow.record.selector.HAVE_ASTOR flow.record.selector.AST_OPERATORS flow.record.selector.AST_COMPARATORS flow.record.selector.NONE_OBJECT flow.record.selector.FUNCTION_WHITELIST .. py:data:: HAVE_ASTOR :value: True .. py:data:: AST_OPERATORS .. py:data:: AST_COMPARATORS .. py:class:: NoneObject Returned in the Selector matching if a field does not exist on the Record. NoneObject is used to override some comparators like __contains__. .. py:method:: __eq__(b: object) -> bool .. py:method:: __ne__(b: object) -> bool .. py:method:: __lt__(b: object) -> bool .. py:method:: __gt__(b: object) -> bool .. py:method:: __lte__(b: object) -> bool .. py:method:: __gte__(b: object) -> bool .. py:method:: __noteq__(b: object) -> bool .. py:method:: __contains__(b: object) -> bool .. py:method:: __len__() -> int .. py:data:: NONE_OBJECT .. py:exception:: InvalidSelectorError Bases: :py:obj:`Exception` Common base class for all non-exit exceptions. .. py:exception:: InvalidOperation Bases: :py:obj:`Exception` Common base class for all non-exit exceptions. .. py:function:: lower(s: str | Any) -> str Return lowercased string, otherwise `s` if not string type. .. py:function:: upper(s: str | Any) -> str | Any Return uppercased string, otherwise `s` if not string type. .. py:function:: names(r: flow.record.base.Record | WrappedRecord | flow.record.base.GroupedRecord) -> set[str] Return the available names as a set in the Record otherwise ['UnknownRecord']. .. py:function:: name(r: flow.record.base.Record | WrappedRecord) -> str Return the name of the Record otherwise 'UnknownRecord'. .. py:function:: get_type(obj: Any) -> str Return the type of the Object as 'str'. .. py:function:: has_field(r: flow.record.base.Record, field: str) -> bool Check if field exists on Record object. :param r: Record to match on. :param field_name: Field name :returns: True if field exists, otherwise False :rtype: (bool) .. py:function:: field_regex(r: flow.record.base.Record, fields: list[str], regex: str) -> bool Check a regex against fields of a Record object. :param r: The record to match on. :param fields: The fields in the Record to match. :param regex: The regex pattern to search for. :returns: True or False :rtype: (bool) .. py:function:: field_equals(r: flow.record.base.Record, fields: list[str], strings: list[str], nocase: bool = True) -> bool Check for exact string matches on fields of a Record object. :param r: The record to match on. :param fields: The fields in the Record to match. :param strings: The strings to search for. :param nocase: Should the matching be case insensitive. :returns: True or False :rtype: (bool) .. py:function:: field_contains(r: flow.record.base.Record, fields: list[str], strings: list[str], nocase: bool = True, word_boundary: bool = False) -> bool Check if the string matches on fields of a Record object. Only supports strings for now and partial matches using the __contains__ operator. * `fields` is a list of field names to check * `strings` is a list of strings to check on the fields * `word_boundary` is a boolean. True if matching required only word boundary matches. * Non existing fields on the Record object are skipped. * Defaults to case-insensitive matching, use `nocase=False` if you want to be case sensitive. .. py:data:: FUNCTION_WHITELIST .. py:function:: resolve_attr_path(node: ast.Call) -> str Resolve a node attribute to full path, eg: net.ipv4.Subnet. .. py:class:: SelectorResult(expression_str: str, match_result: Any, backtrace: list[tuple[int, Any]], referenced_fields: list) .. py:attribute:: expresssion_str .. py:attribute:: result .. py:attribute:: backtrace_info .. py:attribute:: referenced_fields .. py:method:: backtrace() -> str .. py:class:: Selector(expression: str) .. py:attribute:: VERBOSITY_ALL :value: 1 .. py:attribute:: VERBOSITY_BRANCHES :value: 2 .. py:attribute:: VERBOSITY_NONE :value: 3 .. py:attribute:: expression_str .. py:attribute:: expression .. py:attribute:: matcher :value: None .. py:method:: __str__() -> str .. py:method:: __repr__() -> str .. py:method:: __contains__(record: flow.record.base.Record) -> bool .. py:method:: explain_selector(record: flow.record.base.Record, verbosity: int = VERBOSITY_ALL) -> SelectorResult .. py:method:: match(record: flow.record.base.Record) -> bool .. py:class:: WrappedRecord(record: flow.record.base.Record) WrappedRecord wraps a Record but will return a NoneObject for non existing attributes. .. py:attribute:: __slots__ :value: ('record',) .. py:attribute:: record .. py:method:: __getattr__(k: str) -> Any .. py:method:: __str__() -> str .. py:method:: __repr__() -> str .. py:class:: CompiledSelector(expression: str) CompiledSelector is faster than Selector but unsafe if you don't trust the query. .. py:attribute:: expression .. py:attribute:: code :value: None .. py:attribute:: ns .. py:method:: __str__() -> str .. py:method:: __repr__() -> str .. py:method:: __contains__(record: flow.record.base.Record) -> bool .. py:method:: match(record: flow.record.base.Record) -> bool .. py:class:: TypeMatcher(rec: flow.record.base.Record) Helper to get and check fields of a certain type. Types can be selected using `Type.`. Attributes can be selected using `Type..`. For example `Type.uri.filename` will retrieve all the filenames from all uri's in a record. These selectors can also still be used in other helper functions, as they will unwrap to resulting fieldnames. So for example, you can still do `field_contains(r, Type.string, ['something'])`, which will check all `string` fields. Membership tests also work. `'something' in Type.string` will perform a membership test in each string value and return True if there are any. Reverse membership tests are trickier, and only work with a non-compiled Selector. For example, `Type.net.ipv4.Address in net.ipv4.Subnet('10.0.0.0/8')` requires the TypeMatcher to unroll its values, which is only possible when overriding this behaviour. .. py:method:: __getattr__(attr: str) -> TypeMatcherInstance | NoneObject .. py:class:: TypeMatcherInstance(rec: flow.record.base.Record, ftypeparts: list[str] | None = None, attrs: list[str] | None = None) .. py:method:: __getattr__(attr: str) -> TypeMatcherInstance | NoneObject .. py:method:: __iter__() -> collections.abc.Iterator[str] .. py:method:: __eq__(other: object) -> bool .. py:method:: __ne__(other: object) -> bool .. py:method:: __lt__(other: object) -> bool .. py:method:: __gt__(other: object) -> bool .. py:method:: __lte__(other: object) -> bool .. py:method:: __gte__(other: object) -> bool .. py:method:: __noteq__(other: object) -> bool .. py:method:: __contains__(other: object) -> bool .. py:class:: RecordContextMatcher(expr: ast.Expression, expr_str: str, backtrace_verbosity: int = Selector.VERBOSITY_NONE) .. py:attribute:: expression .. py:attribute:: expression_str .. py:attribute:: selector_backtrace :value: [] .. py:attribute:: selector_backtrace_verbosity :value: 3 .. py:attribute:: data .. py:attribute:: rec :value: None .. py:method:: matches(rec: flow.record.base.Record) -> bool .. py:method:: eval(node: ast.expr) -> Any .. py:function:: make_selector(selector: str | Selector | None, force_compiled: bool = False) -> Selector | CompiledSelector | None Return a Selector object (either CompiledSelector or Selector).