[PVE-User] ipfilter functionality

[Mark Schouten]

Hi,

We've been struggling with ipfilter for a few days, thinking it doesn't
work, because inbound connections kept working, even though there was
not a single IP in the ipfilter-net0 IPSet.

But, it looks like only outbound connections are dropped, but inbound
connections work. While this is functional, it doesn't prevent anyone
from spoofing a neighbours address, so it's not completely functional.

Am I missing something?

[Wolfgang Bumiller]

Post by Mark Schouten
Hi,
We've been struggling with ipfilter for a few days, thinking it doesn't
work, because inbound connections kept working, even though there was
not a single IP in the ipfilter-net0 IPSet.
But, it looks like only outbound connections are dropped, but inbound
connections work. While this is functional, it doesn't prevent anyone
from spoofing a neighbours address, so it's not completely functional.

This is currently due to the connection tracking rules happening too
early. Similarly MAC filtering only happens for IP packets.
If you do not need to disable MAC filtering you can try the
pve-firewall >= 3.0-8 package from pvetest which will setup ebtables for
MAC filtering, that should help. But to make it work completely as most
users expect it we'll have to move the conntrack rules from the forward
chain into the device specific chains.
It's on my todo list along with another round of nftables testing.

@Tom: not sure if you're currently doing anything in the firewall code,
but I thought I'd ping/Cc you to let you know the ebtables patch set
landed in pvetest.

[Mark Schouten]

Post by Wolfgang Bumiller
This is currently due to the connection tracking rules happening too
early. Similarly MAC filtering only happens for IP packets.
If you do not need to disable MAC filtering you can try the
pve-firewall >= 3.0-8 package from pvetest which will setup ebtables for
MAC filtering, that should help. But to make it work completely as most
users expect it we'll have to move the conntrack rules from the forward
chain into the device specific chains.
It's on my todo list along with another round of nftables testing.

It's not really MAC filtering I'm looking for. But wouldn't this be
fixed if the connection inbound would be filtered as well as outbound?
So add the ipfilter-rules to $interface-IN as well?

[Mark Schouten]

Post by Mark Schouten
It's not really MAC filtering I'm looking for. But wouldn't this be
fixed if the connection inbound would be filtered as well as
outbound?
So add the ipfilter-rules to $interface-IN as well?

Like so:
diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 8f545e7..1bf0725 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -2202,6 +2202,12 @@ sub ruleset_create_vm_chain {
        }
        ruleset_addrule($ruleset, $chain, "", "-j MARK --set-mark
$FWACCEPTMARK_OFF"); # clear mark
     }
+    if ($direction eq 'IN') {
+       if ($ipfilter_ipset) {
+           ruleset_addrule($ruleset, $chain, "-m set ! --match-set
$ipfilter_ipset dst", "-j DROP");
+       }
+    }
+
 
     my $accept_action = $direction eq 'OUT' ? '-g PVEFW-SET-ACCEPT-
MARK' : "-j $accept";
     ruleset_chain_add_ndp($ruleset, $chain, $ipversion, $options,
$direction, $accept_action);

[Wolfgang Bumiller]

Post by Mark Schouten

Post by Mark Schouten
It's not really MAC filtering I'm looking for. But wouldn't this be
fixed if the connection inbound would be filtered as well as
outbound?
So add the ipfilter-rules to $interface-IN as well?

For simple connections this works, but then you also break multicast
traffic unless you add all multicast IPs to the ipfilter as well. The
real solution would be to move the conntrack rules from PVEFW-FORWARD
into tap/veth${vmid}i* to below the ipfilter.

Post by Mark Schouten
diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm
index 8f545e7..1bf0725 100644
--- a/src/PVE/Firewall.pm
+++ b/src/PVE/Firewall.pm
@@ -2202,6 +2202,12 @@ sub ruleset_create_vm_chain {
        }
        ruleset_addrule($ruleset, $chain, "", "-j MARK --set-mark
$FWACCEPTMARK_OFF"); # clear mark
     }
+    if ($direction eq 'IN') {
+       if ($ipfilter_ipset) {
+           ruleset_addrule($ruleset, $chain, "-m set ! --match-set
$ipfilter_ipset dst", "-j DROP");
+       }
+    }
+
 
     my $accept_action = $direction eq 'OUT' ? '-g PVEFW-SET-ACCEPT-
MARK' : "-j $accept";
     ruleset_chain_add_ndp($ruleset, $chain, $ipversion, $options,
$direction, $accept_action);

[Mark Schouten]

Post by Wolfgang Bumiller
For simple connections this works, but then you also break multicast
traffic unless you add all multicast IPs to the ipfilter as well. The
real solution would be to move the conntrack rules from PVEFW-FORWARD
into tap/veth${vmid}i* to below the ipfilter.

True. But moving the conntrack rules to every individual chain extends
the ruleset, a lot. Multicast addresses are pretty much limited to
two(?) subnets, which could be added to an already existing ipset,
which the kernel already visits.

I'm no kernel guru, I have the feeling that increasing the ruleset is
more resourcehungry.

Either way, it would be great if this would be fixed!

[Wolfgang Bumiller]

Post by Mark Schouten

Post by Wolfgang Bumiller
For simple connections this works, but then you also break multicast
traffic unless you add all multicast IPs to the ipfilter as well. The
real solution would be to move the conntrack rules from PVEFW-FORWARD
into tap/veth${vmid}i* to below the ipfilter.

True. But moving the conntrack rules to every individual chain extends
the ruleset, a lot. Multicast addresses are pretty much limited to
two(?) subnets, which could be added to an already existing ipset,
which the kernel already visits.
I'm no kernel guru, I have the feeling that increasing the ruleset is
more resourcehungry.

A bit, sure, but it may not be that bad. According to
itpables-extensions(8) the 'established' state has already seen packets
in both directions, so it may be enough to just move the 'related' rule
into the VM chains. Would have to test UDP (since it's connection-less,
but the wording is pretty clear that packets in both directions must
have already happened at least once). That way fully-established
connections would still take the fast path.

Post by Mark Schouten
Either way, it would be great if this would be fixed!

Agreed.

[Mark Schouten]

Post by Mark Schouten
Either way, it would be great if this would be fixed!
Agreed.

Is there any idea of a timeframe when this would be fixed? The current
setup renders the 'ipfilter'-functionality useless, as someone can run
services using whichever IP they want. The description of the ipfilter-
function makes you suspect that that is prevented.

(I don't mind testing/spending time on speeding this up)