[PVE-User] Interfaces startup and ip-up.d scripts...

[Marco Gaiarin]

I've to setup a little PVE server in a private but hostile network, and
i've only an IP available, so i was forced to assign the IP to the
phisical server, running latest proxmox, and i've setup a firewall
using my hand-made scripts.

I've put the script, as usual with debian, in /etc/network/if-up.d/ and
if-down.d/, but i've found that not at every boot they get started.

So, i've enabled networking debug (eg, set VERBOSE=yes in
/etc/default/networking) and found that at every boot scripts get
called with 'lo' interface, but only roughly 1 out of 10 times the
other bridge interfaces get started.

So, i got:

May 15 10:18:25 clerk ifup[2958]: /bin/ip link set dev lo up
May 15 10:18:25 clerk ifup[2958]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

but then:

May 15 10:18:26 clerk ifup[2958]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

without interface name. With some more debug i discovered that is the
'--all' interface.
The strange things is that interfaces vmbr0 and vmbr1 are up, simply
the scripts get not called.

Some boot, instead:

May 15 10:18:27 clerk ifup[4043]: /bin/ip link set dev vmbr0 up
May 15 10:18:27 clerk ifup[4043]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

May 15 10:18:28 clerk ifup[4043]: /bin/ip addr add 10.5.2.230/255.255.0.0 broadcast 10.5.255.255 dev vmbr1 label vmbr1
May 15 10:18:28 clerk ifup[4043]: /bin/ip link set dev vmbr1 up
May 15 10:18:28 clerk ifup[4043]: /bin/ip route add default via 10.5.1.254 dev vmbr1 onlink
May 15 10:18:28 clerk ifup[4043]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

and clearly firewalling script works.


For now, i've put 'systemctl restart networking' in /etc/rc.local.

My /etc/network/interfaces is rather simple:

auto lo
iface lo inet loopback

iface enp2s0f0 inet manual

iface enp2s0f1 inet manual
ethernet-autoneg on
link-speed 100
link-duplex full
hardware-dma-ring-tx 18
offload-tso off
offload-gso off
offload-gro off

auto vmbr0
iface vmbr0 inet static
address 10.99.25.254
netmask 255.255.252.0
bridge_ports enp2s0f0
bridge_stp off
bridge_fd 0
bridge_vlan_aware yes

auto vmbr1
iface vmbr1 inet static
address 10.5.2.230
netmask 255.255.0.0
gateway 10.5.1.254
bridge_ports enp2s0f1
bridge_stp off
bridge_fd 0


There's something i can do to fix this? Thanks.

[dORSY]

I suggest using post-up / pre-down hooks in interfaces. It makes sure that the interfaces are actually up before the commands get executed.

On Tuesday, 15 May 2018, 15:26:49 CEST, Marco Gaiarin <***@sv.lnf.it> wrote:


I've to setup a little PVE server in a private but hostile network, and
i've only an IP available, so i was forced to assign the IP to the
phisical server, running latest proxmox, and i've setup a firewall
using my hand-made scripts.

I've put the script, as usual with debian, in /etc/network/if-up.d/ and
if-down.d/, but i've found that not at every boot they get started.

So, i've enabled networking debug (eg, set VERBOSE=yes in
/etc/default/networking) and found that at every boot scripts get
called with 'lo' interface, but only roughly 1 out of 10 times the
other bridge interfaces get started.

So, i got:

May 15 10:18:25 clerk ifup[2958]: /bin/ip link set dev lo up
May 15 10:18:25 clerk ifup[2958]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

but then:

May 15 10:18:26 clerk ifup[2958]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

without interface name. With some more debug i discovered that is the
'--all' interface.
The strange things is that interfaces vmbr0 and vmbr1 are up, simply
the scripts get not called.

Some boot, instead:

May 15 10:18:27 clerk ifup[4043]: /bin/ip link set dev vmbr0  up
May 15 10:18:27 clerk ifup[4043]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

May 15 10:18:28 clerk ifup[4043]: /bin/ip addr add 10.5.2.230/255.255.0.0 broadcast 10.5.255.255          dev vmbr1 label vmbr1
May 15 10:18:28 clerk ifup[4043]: /bin/ip link set dev vmbr1  up
May 15 10:18:28 clerk ifup[4043]:  /bin/ip route add default via 10.5.1.254  dev vmbr1 onlink
May 15 10:18:28 clerk ifup[4043]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d

and clearly firewalling script works.


For now, i've put 'systemctl restart networking' in /etc/rc.local.

My /etc/network/interfaces is rather simple:

auto lo
iface lo inet loopback

iface enp2s0f0 inet manual

iface enp2s0f1 inet manual
    ethernet-autoneg on
    link-speed 100
    link-duplex full
    hardware-dma-ring-tx 18
    offload-tso off
    offload-gso off
    offload-gro off

auto vmbr0
iface vmbr0 inet static
    address  10.99.25.254
    netmask  255.255.252.0
    bridge_ports enp2s0f0
    bridge_stp off
    bridge_fd 0
    bridge_vlan_aware yes

auto vmbr1
iface vmbr1 inet static
    address  10.5.2.230
    netmask  255.255.0.0
    gateway  10.5.1.254
    bridge_ports enp2s0f1
    bridge_stp off
    bridge_fd 0


There's something i can do to fix this? Thanks.

[Marco Gaiarin]

Mandi! dORSY
In chel di` si favelave...

Post by dORSY
I suggest using post-up / pre-down hooks in interfaces. It makes sure that the interfaces are actually up before the commands get executed.

From 'interfeces' manpage:

Please note that as post-up and pre-down are aliases, no files in the corresponding directories are processed. Please use if-up.d and if-down.d directories instead.

so seems that put script in if-up.d or as post-up stanzas is the same.

Also, seems to me that the trouble came from the fact that network
scripts does not run at all fro vmbr0 and vmbr1 interfaces!

[Josh Knight]

Are you using the script to assign an IP address manually, or are you using
it to set firewall rules? I'm trying to determine what you're trying to
do. Having the IP defined in /etc/network/interfaces should be enough for
it to come up correctly without any custom scripts. Is this not the case?

Josh Knight

Post by Marco Gaiarin
I've to setup a little PVE server in a private but hostile network, and
i've only an IP available, so i was forced to assign the IP to the
phisical server, running latest proxmox, and i've setup a firewall
using my hand-made scripts.
I've put the script, as usual with debian, in /etc/network/if-up.d/ and
if-down.d/, but i've found that not at every boot they get started.
So, i've enabled networking debug (eg, set VERBOSE=yes in
/etc/default/networking) and found that at every boot scripts get
called with 'lo' interface, but only roughly 1 out of 10 times the
other bridge interfaces get started.
May 15 10:18:25 clerk ifup[2958]: /bin/ip link set dev lo up
May 15 10:18:25 clerk ifup[2958]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-up.d
May 15 10:18:26 clerk ifup[2958]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-up.d
without interface name. With some more debug i discovered that is the
'--all' interface.
The strange things is that interfaces vmbr0 and vmbr1 are up, simply
the scripts get not called.
May 15 10:18:27 clerk ifup[4043]: /bin/ip link set dev vmbr0 up
May 15 10:18:27 clerk ifup[4043]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-up.d
May 15 10:18:28 clerk ifup[4043]: /bin/ip addr add 10.5.2.230/255.255.0.0
broadcast 10.5.255.255 dev vmbr1 label vmbr1
May 15 10:18:28 clerk ifup[4043]: /bin/ip link set dev vmbr1 up
May 15 10:18:28 clerk ifup[4043]: /bin/ip route add default via
10.5.1.254 dev vmbr1 onlink
May 15 10:18:28 clerk ifup[4043]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-up.d
and clearly firewalling script works.
For now, i've put 'systemctl restart networking' in /etc/rc.local.
auto lo
iface lo inet loopback
iface enp2s0f0 inet manual
iface enp2s0f1 inet manual
ethernet-autoneg on
link-speed 100
link-duplex full
hardware-dma-ring-tx 18
offload-tso off
offload-gso off
offload-gro off
auto vmbr0
iface vmbr0 inet static
address 10.99.25.254
netmask 255.255.252.0
bridge_ports enp2s0f0
bridge_stp off
bridge_fd 0
bridge_vlan_aware yes
auto vmbr1
iface vmbr1 inet static
address 10.5.2.230
netmask 255.255.0.0
gateway 10.5.1.254
bridge_ports enp2s0f1
bridge_stp off
bridge_fd 0
There's something i can do to fix this? Thanks.
--
240A3D66
Associazione ``La Nostra Famiglia''
http://www.lanostrafamiglia.it/
Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN)
marco.gaiarin(at)lanostrafamiglia.it t [phone] f
[phone]
Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[Marco Gaiarin]

Mandi! Josh Knight
In chel di` si favelave...

Post by Josh Knight
Are you using the script to assign an IP address manually, or are you using
it to set firewall rules?

I'm setting firewall rules; my script are coded carefully, and exit
always with 0 status.
They works in my debisn server/firewall without a trouble (but, indeed, with
standard debian kernel, and still in jessie).

Post by Josh Knight
I'm trying to determine what you're trying to
do. Having the IP defined in /etc/network/interfaces should be enough for
it to come up correctly without any custom scripts. Is this not the case?

Post by Marco Gaiarin
auto vmbr0
iface vmbr0 inet static
address 10.99.25.254
netmask 255.255.252.0
bridge_ports enp2s0f0
bridge_stp off
bridge_fd 0
bridge_vlan_aware yes
auto vmbr1
iface vmbr1 inet static
address 10.5.2.230
netmask 255.255.0.0
gateway 10.5.1.254
bridge_ports enp2s0f1
bridge_stp off
bridge_fd 0

interfaces get correctly brought up (eg, and 'ip address show' list
interfaces correctly), but looking at 'journalctl -b' seems that
if-up.d and if-down.d scripts get never executed.

*ALL* scripts, of course, not only mine...

[dORSY]

You can also use either proxmox's built-in firewall (which is awesome if you get used to it) or netfilter-persistent to load/save iptables rules.
They tend to be more robust than shellscipt hooks.

On Tuesday, 15 May 2018, 19:08:42 CEST, Marco Gaiarin <***@sv.lnf.it> wrote:

Mandi! Josh Knight
  In chel di` si favelave...

Post by Josh Knight
Are you using the script to assign an IP address manually, or are you using
it to set firewall rules?

I'm setting firewall rules; my script are coded carefully, and exit
always with 0 status.
They works in my debisn server/firewall without a trouble (but, indeed, with
standard debian kernel, and still in jessie).

Post by Josh Knight
I'm trying to determine what you're trying to
do.  Having the IP defined in /etc/network/interfaces should be enough for
it to come up correctly without any custom scripts.  Is this not the case?

  auto vmbr0
  iface vmbr0 inet static
        address  10.99.25.254
        netmask  255.255.252.0
        bridge_ports enp2s0f0
        bridge_stp off
        bridge_fd 0
        bridge_vlan_aware yes
  auto vmbr1
  iface vmbr1 inet static
        address  10.5.2.230
        netmask  255.255.0.0
        gateway  10.5.1.254
        bridge_ports enp2s0f1
        bridge_stp off
        bridge_fd 0

interfaces get correctly brought up (eg, and 'ip address show' list
interfaces correctly), but looking at 'journalctl -b' seems that
if-up.d and if-down.d scripts get never executed.

*ALL* scripts, of course, not only mine...
--
dott. Marco Gaiarin                        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG  -  Via della Bontà, 7 - 33078  -  San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it  t [phone]  f [phone]

        Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
    (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[dORSY]

Also I use this like:        post-up /etc/network/custom/vmbr0-post-up
        post-up /sbin/ethtool -G eth0 rx 2048 || true
        post-up /sbin/ethtool -G eth0 tx 2048 || true
        pre-down /etc/network/custom/vmbr0-pre-down
And never had problems with it.

On Tuesday, 15 May 2018, 19:08:42 CEST, Marco Gaiarin <***@sv.lnf.it> wrote:

Mandi! Josh Knight
  In chel di` si favelave...

Post by Josh Knight
Are you using the script to assign an IP address manually, or are you using
it to set firewall rules?

I'm setting firewall rules; my script are coded carefully, and exit
always with 0 status.
They works in my debisn server/firewall without a trouble (but, indeed, with
standard debian kernel, and still in jessie).

Post by Josh Knight
I'm trying to determine what you're trying to
do.  Having the IP defined in /etc/network/interfaces should be enough for
it to come up correctly without any custom scripts.  Is this not the case?

  auto vmbr0
  iface vmbr0 inet static
        address  10.99.25.254
        netmask  255.255.252.0
        bridge_ports enp2s0f0
        bridge_stp off
        bridge_fd 0
        bridge_vlan_aware yes
  auto vmbr1
  iface vmbr1 inet static
        address  10.5.2.230
        netmask  255.255.0.0
        gateway  10.5.1.254
        bridge_ports enp2s0f1
        bridge_stp off
        bridge_fd 0

interfaces get correctly brought up (eg, and 'ip address show' list
interfaces correctly), but looking at 'journalctl -b' seems that
if-up.d and if-down.d scripts get never executed.

*ALL* scripts, of course, not only mine...
--
dott. Marco Gaiarin                        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG  -  Via della Bontà, 7 - 33078  -  San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it  t [phone]  f [phone]

        Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
    (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[dORSY]

"So, now i cannot play with networking. ;-)"
Then don't play with them. Simply use the "old-school" working post-up and pre-down directives for the interfaces. As we all linux admins do for ages. And use built-in firewall tools (iptables/netfilter-persistent or proxmox's fireall or anything coming as a debian package) All in all proxmox5 is debian9 and works exactly as one :).

On Friday, 18 May 2018, 17:23:20 CEST, Marco Gaiarin <***@sv.lnf.it> wrote:

Mandi! Josh Knight
  In chel di` si favelave...

Interesting, I couldn't reproduce the problem on my server.

I'm not a very large user case: i've many PVE system, but they are 4.4
and not ''firewalled'', this is a 5.2 and a case ''per se''...

I set
verbose=yes, I created a test script that simply did echo $IFACE and after
3 reboots it seems to execute each time. After boot I just did  journalctl
-b | grep ifup  and I was able to see the interface names printed.

Boh...

in journalctl -b, are you seeing anything related to run-parts? Or does
ifup not print anything at all?

In a ''falied'' boot i can se the logs for interfaces 'lo' and '--all'
(why '--all'?). Logs report, for every interface:
    /bin/ip link set dev <iface> up
and then the run of the 'run-parts':
    run-parts: executing /etc/network/if-up.d/0sysctl
and the the single runs of the scripts:
    run-parts: executing /etc/network/if-up.d/bridgevlan

In a good boot, i can se the logs for interfaces 'lo', 'vmbr0', 'vmbr1'
and '--all''. Same logs.

Is your firewall script using anything interface specific?  If you put it
in that directory, it will be executed for each interface.

My script are parametrizied, and get runned only on particular
interfaces.

I'm curious if
you add a post-up line to your /etc/network/interfaces file, it would be
called only once when your mgt interface comes up.

I was short on time, and so i was forced to put that server
in production, in a non too easy reachable place.

So, now i cannot play with networking. ;-)
--
dott. Marco Gaiarin                        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG  -  Via della Bontà, 7 - 33078  -  San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it  t [phone]  f [phone]

        Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
    (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user