Crate retina_filtergen

A macro for defining subscriptions in Retina.

Specifying Subscriptions in a .rs File

filter is an attribute macro that parses a user-defined filter expression to generate sub-filters that are statically inlined at each processing layer. This verifies filter expressions at compile time and avoids overheads associated with interpreting filters at runtime. The filter attribute macro must be tied to the callback requested.

retina_main is an attribute macro that must be used alongside filter. It indicates to the framework how many subscriptions to expect.

Usage

use retina_core::config::default_config;
use retina_core::Runtime;
use retina_filtergen::{filter, retina_main};
use retina_datatypes::*;

#[filter("ipv4 and tcp.port >= 100")]
fn log_tcp(conn: &ConnRecord) { // Callback; datatype(s) by reference
   println!("{:#?}", conn);
}

#[filter("tls.sni ~ 'netflix'")]
fn log_tls(conn: &ConnRecord, tls: &TlsHandshake) {
   println!("{:#?}, {:#?}", conn, tls);
}

#[retina_main(2)] // 2 subscriptions expected
fn main() {
   let cfg = default_config();
   // \note SubscribedWrapper is the structure built at compile-time to
   // 'wrap' all requested data. It must be specified as the generic parameter to Runtime.
   let mut runtime: Runtime<SubscribedWrapper> = Runtime::new(cfg, filter).unwrap();
   runtime.run();
}

Specifying Subscriptions in TOML File

subscription is an attribute macro that allows users to specify a list of subscriptions from an input file. This may be useful, for example, for programs that require a large number of filters to be applied to the same callback.

Input TOML files should be a list of subscriptions, each of which has filter, datatypes, and callback specified. The datatypes in the TOML file must match the order and names of the datatypes in the callbacks in code.

Usage

[[subscriptions]]
filter = "(http.user_agent = '/mm.jpg') and (tcp.dst_port = 80 or tcp.dst_port = 8080)"
datatypes = [
    "HttpTransaction",
    "FilterStr",
]
callback = "http_cb"

[[subscriptions]]
filter = "http.user_agent = 'HttpBrowser/1.0'"
datatypes = [
    "HttpTransaction",
    "FilterStr",
]
callback = "http_cb"

use retina_core::config::default_config;
use retina_core::Runtime;
use retina_filtergen::subscription;
use retina_datatypes::*;

fn http_cb(http: &HttpTransaction, matched_filter: &FilterStr) {
   println!("Matched {} with {:#?}", matched_filter, http);
}

#[subscription("examples/XX/spec.toml")]
fn main() {
   let cfg = default_config();
   let mut runtime: Runtime<SubscribedWrapper> = Runtime::new(cfg, filter).unwrap();
   runtime.run();
}

Datatype syntax

All subscribed datatypes – parameters to callbacks – must be requested by reference. Supported datatypes are defined in the retina_datatypes crate.

Filter syntax

The Retina filter syntax is similar to that of Wireshark display filters. However, Retina is capable of filtering on live traffic for online analysis, whereas display filters can only be applied for offline analysis.

A filter expression is a logical expression of constraints on attributes of the input traffic. Each constraint is either a binary predicate that compares the value of an entity’s attribute with a constant, or a unary predicate that matches against the entity itself.

Protocols

All protocol identifiers are valid as long as Retina contains an appropriate protocol module of the same name. The filter macro automatically generates filtering code using structs defined in the protocol’s corresponding parser module. The exception to this is ethernet, which Retina filters for by default.

For example, ipv4 and tls are filterable protocols because they are both protocol modules included in Retina.

Retina will also automatically expand filter expressions to their fully-qualified form. For example, the filter tcp is equivalent to (ipv4 and tcp) or (ipv6 and tcp).

Fields

All field identifiers are valid as long as Retina exposes a public accessor method for the corresponding protocol struct of the same name, and the method returns a supported RHS field type.

For example, ipv4.src_addr and tls.sni are both filterable fields because src_addr() is a public method associated with the Ipv4 struct that returns an Ipv4Addr, and sni() is a public method associated with the Tls struct that returns a String.

Retina also supports two combined fields: addr and port. Logically, these are equivalent to src_addr or dst_addr and src_port or dst_port, respectively, except in predicates that use the != comparison operator (details below).

Field types (RHS values)

Type	Example
IPv4 address	1.2.3.4
IPv4 prefix	1.2.3.4/24
IPv6 address	2001:db8::1
IPv6 prefix	2001:db8::1/64
Integer	443
String	'Safari'
Integer range	1024..5000

Binary comparison operators

Operator	Alias	Description	Example
=		Equals	ipv4.addr = 127.0.0.1
!=	ne	Not equals	udp.dst_port != 53
>=	ge	Greater than or equals	tcp.port >= 1024
<=	le	Less than or equals	tls.version <= 771
>	gt	Greater than	ipv4.time_to_live > 64
<	lt	Less than	ipv6.payload_length < 500
in		In a range, or in a subnet	ipv4.src_addr in 1.2.3.4/16
~	matches	Regular expression match	tls.sni ~ 'netflix\\.com$'

Possible pitfalls involving !=

Retina differs from Wireshark behavior regarding the != operator. When applied to combined fields (i.e., addr or port), Retina follows connection-level semantics instead of packet-level semantics. For example, tcp.port != 80 is equivalent to tcp.src_port != 80 and tcp.dst_port != 80.

Regular expressions

Retina compiles regular expressions exactly once, using the regex and lazy_static crates. As of this writing, proc-macro crates like this one cannot export any items other than procedural macros, thus requiring applications that wish to use Retina’s regular expression filtering to specify regex and lazy_static as dependencies. Also note that regular expressions are written as normal strings in Rust, and not as raw string literals. They are allowed to match anywhere in the text, unless start (^) and end ($) anchors are used.

Logical operators

Operator	Alias	Description	Example
and	AND	Logical AND	tcp.port = 443 and tls
or	OR	Logical OR	http.method = 'GET' or http.method = 'POST'

AND takes precedence over OR in the absence of parentheses.

Retina does not yet support the logical NOT operator. For some expressions, it can be approximated using the != binary comparison operator, taking the above mentioned pitfall into consideration.

Attribute Macros

• filter
  Generate a Retina program without a specification file. This expects a #[filter(“…”)] macro followed by the expected callback. It must be used with #[retina_main(X)], where X = number of subscriptions.
• retina_main
• subscription
  Generate a Retina program from a specification file. This expects an input TOML file with the subscription specifications.