Snort errors: ERROR: rules/snort.rules:3230 unknown rule keyword: pcre

@efahl @xxxx - my snort setup stopped reading my rules files on a rebuild. You guys have any thoughts? First error is:

ERROR: /etc/snort/snort.lua: can't find detection.pcre_to_regex

Complete output:

# snort -c /etc/snort/snort.lua --tweaks local
--------------------------------------------------
o")~   Snort++ 3.1.84.0
--------------------------------------------------
Loading /etc/snort/snort.lua:
Loading homenet.lua:
Finished homenet.lua:
Loading snort_defaults.lua:
Finished snort_defaults.lua:
Loading local.lua:
Finished local.lua:
	snort
	ssh
	host_cache
	pop
	so_proxy
	stream_tcp
	mms
	smtp
	gtp_inspect
	packets
	dce_http_proxy
	detection
ERROR: /etc/snort/snort.lua: can't find detection.pcre_to_regex
	alert_fast
	suppress
	cip
	ips
	stream_icmp
	hosts
	normalizer
	binder
	wizard
	appid
	js_norm
	file_id
	http2_inspect
	http_inspect
	stream_udp
	ftp_data
	ftp_server
	search_engine
	port_scan
	dce_http_server
	dce_tcp
	dce_smb
	iec104
	telnet
	ssl
	sip
	rpc_decode
	netflow
	modbus
	host_tracker
	stream_user
	stream_ip
	process
	back_orifice
	classifications
	dnp3
	active
	trace
	ftp_client
	decode
	alerts
	stream
	references
	daq
	arp_spoof
	output
	network
	dns
	dce_udp
	imap
	file_policy
	s7commplus
	stream_file
Finished /etc/snort/snort.lua:
Loading file_id.rules_file:
Loading file_magic.rules:
Finished file_magic.rules:
Finished file_id.rules_file:
Loading rules/snort.rules:
ERROR: rules/snort.rules:9 unknown rule keyword: pcre.
ERROR: rules/snort.rules:13 unknown rule keyword: pcre.
...
ERROR: rules/snort.rules:3237 syntax error
ERROR: rules/snort.rules:3237 Unable to process the IP address: max-detect-ips.
FATAL: rules/snort.rules:3237 ***Rule--PortVar Parse error: (pos=1,error=not a number)
>>drop;
>>^
Fatal Error, Quitting..
/etc/init.d/snort
#!/bin/sh /etc/rc.common

START=99
STOP=10

USE_PROCD=1
PROG=/usr/bin/snort

validate_snort_section() {
	uci_validate_section snort snort "${1}" \
		'config_dir:string' \
		'interface:string'
}

start_service() {
	local config_file interface

	validate_snort_section snort || {
		echo "validation failed"
		return 1
	}

	procd_open_instance
	procd_set_param env SNORT_LUA_PATH="$config_dir"
	procd_set_param command nice -n -20 $PROG -c "${config_dir%/}/snort.lua" --tweaks local
	fw4 reload
	procd_set_param file $CONFIGFILE
	procd_set_param respawn
	procd_set_param stdout 1
	procd_set_param stderr 1
	procd_close_instance
}

stop_service()
{
	service_stop ${PROG}
	nft delete chain inet fw4 IPS_output
	nft delete chain inet fw4 IPS_input
}

service_triggers()
{
	procd_add_reload_trigger "snort"
	procd_add_validation validate_snort_section
}
/etc/snort/local.lua
snort  = {
	['-Q'] = true,
	['--max-packet-threads'] = 4,
}

suppress = {
	-- this kills stuff in lxc
	{
		gid = 1, sid = 650, track = 'by_dst', ip = '10.9.8.101'
	},
}

network = {
	checksum_eval = 'none',
}

daq = {
	module_dirs = { '/usr/lib/daq' },
	inputs = { '4', '5', '6', '7' },
	snaplen = 65531,
	modules = {
		{
			name = 'nfq',
			mode = 'inline',
			variables = {
				'queue_maxlen=8192',
				'fail_open',
				'device=eth1'
			}
		}
	}
}

ips = {
	mode = inline,
	variables = default_variables,
	action_override = 'drop',
	include = RULE_PATH .. '/snort.rules',
	--include = RULE_PATH .. '/test',
}

output.logdir = '/mnt/data'
alert_fast = {
	file = true,
	packet = false,
}

file_policy = {
	enable_type = true,
	enable_signature = true,
	rules = {
		use = {
			verdict = 'log', enable_file_type = true, enable_file_signature = true
		}
	}
}

search_engine = { 
	search_method = "hyperscan",
	offload_search_method ="hyperscan",
	detect_raw_tcp = true,
}

detection = { 
	hyperscan_literals = true,
	pcre_to_regex = true,
}

Probably caused by the move from the deprecated pcre to pcre2 a week ago

Thanks for pointing that out @hnyman. Seems like this needs to get merged: https://github.com/snort3/snort3/pull/326

@efahl and @xxxx - can you guys take a look at that PR linked? Is there something obvious needing to be set from a config. prespective?

Sorry I probably can't help you there I haven't even managed to build snort with pcre2 yet I end up with the error message: Package snort3 is missing dependencies for the following libraries: libpcre2-8.so.0

When weird stuff like that happens, I usually:

rm -rf feeds
./scripts/feeds update -a ; ./scripts/feeds install -a

Unfortunately that didn't really help me I had to set up the complete build system again now I got it working if you change detection = { hyperscan_literals = true, pcre_to_regex = true } in detection = { hyperscan_literals = true, pcre2_to_regex = true } in snort.lua it should work but the rules themselves still contain pcre keywords and I don't know if we can change them that easily

Nice, yes, confirmed that this change removes the error message:

detection = {
  hyperscan_literals = true,
-  pcre_to_regex = true,
+  pcre2_to_regex = true,
}

But as you pointed out, the rules themselves are now causing errors. Almost seems like our change to pcre2 was premature. Seems like upstream needs to merge and then roll out rules that are compatible. Am I understanding things correctly?

# snort -c /etc/snort/snort.lua --tweaks local
...
Loading rules/snort.rules:
ERROR: rules/snort.rules:9 unknown rule keyword: pcre.
ERROR: rules/snort.rules:13 unknown rule keyword: pcre.
ERROR: rules/snort.rules:14 unknown rule keyword: pcre.
...
ERROR: rules/snort.rules:3232 unknown rule keyword: pcre.
ERROR: rules/snort.rules:3237 unknown rule keyword: pcre.
ERROR: rules/snort.rules:3237 syntax error
ERROR: rules/snort.rules:3237 Unable to process the IP address: max-detect-ips.
FATAL: rules/snort.rules:3237 ***Rule--PortVar Parse error: (pos=1,error=not a number)
>>drop;
>>^
Fatal Error, Quitting..

@Ansuel - do you have any thoughts?

can someone post the rules/snort.rules file ?

The problem is the rules themselves you have to rewrite the rules with sed -i 's/pcre/pcre2/g' /etc/snort/rules/* and change pcre2_to_regex = true to pcre2_to_regex = false because some rules now cause compile errors. I don't know if they can be fixed because they are probably due to the changes from pcre to pcre2.

Yes, try here.

Ah, setting that to false in addition to the sed allows normal loading. Is there a performance hit as a function of pcre2_to_regex = false?

i can modify some name to keep the compatibility but if i'm not wrong some script had to be updated anyway

No, you have to set that, otherwise it can happen that something like this comes out:
ERROR: /tmp/rules/snort3-browser-plugins.rules:1309 can't compile regex '(<object\s*[^>]*\s*id\s*=\s*(?P<m1>\x22|\x27|)(?P<id1>.+?)(?P=m1)(\s|>)[^>]*\s*classid\s*=\s*(?P<q1>\x22|\x27|)\s*clsid\s*\x3a\s*{?\s*3895DD35-7573-11D2-8FED-00606730D3AA\s*}?\s*(?P=q1)(\s|>).*(?P=id1)\s*\.\s*(Run)|<object\s*[^>]*\s*classid\s*=\s*(?P<q2>\x22|\x27|)\s*clsid\s*\x3a\s*{?\s*3895DD35-7573-11D2-8FED-00606730D3AA\s*}?\s*(?P=q2)(\s|>)[^>]*\s*id\s*=\s*(?P<m2>\x22|\x27|)(?P<id2>.+?)(?P=m2)(\s|>).*(?P=id2)\.(Run))\s*\('
Out of 34000 rules or so, it has hit around 1800, some still work without problems but others can no longer be compiled.

This should not solve the problem because Snort can no longer compile some of the rules in regex as you can see, probably because of the changes to pcre2. After all, you could change it so that the rules no longer have to be rewritten with regex false it works.

i'm not following... disabling pcre is wrong. So the current fix would be to change any entry of pcre to pcre2

Yes, in the snort rule files because they change every time depending on the update frequency, they have to be rewritten every time they are updated. And I also doubt whether the whole thing has a positive effect on performance.

//edit/ A small performance test did not reveal any abnormalities, I even had the impression that the pcre2 version was slightly faster. All that remains is the problem with the Snort rules.

@darksky Have you tested the performance to see if it is faster or slower with the switch to Pcre2? I had a look at the statistics of Snort and I noticed that only 6000 rules would be affected, which are not converted from pcre to regex. So the performance loss, if there is one at all (pcre2 will certainly be faster), should not be great.

I saw no difference just running a speed test.

Neither do I, so you could leave it as it is for now. The problem with the rules can easily be solved in the updater - it doesn't take long to rewrite the rules.

1 Like