Discussion:
[PVE-User] PVE kernel (sources, build process, etc.)
Uwe Sauter
2018-08-22 07:48:19 UTC
Permalink
Hi all,

some quick questions:

* As far as I can tell the PVE kernel is a modified version of Ubuntu kernels, correct?
Modifications can be viewed in the pve-kernel.git repository (https://git.proxmox.com/?p=pve-kernel.git;a=tree).

* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?

* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?


Thanks,

Uwe
Thomas Lamprecht
2018-08-22 07:55:10 UTC
Permalink
Hi Uwe,
Post by Uwe Sauter
Hi all,
* As far as I can tell the PVE kernel is a modified version of Ubuntu kernels, correct?
Modifications can be viewed in the pve-kernel.git repository ( https://git.proxmox.com/?p=pve-kernel.git;a=tree ).
Yes, in the respective git branch (master is currently 4.13 and pve-kernel-4.15 is, you guess it, 4.15
patches/ includes on-top bug/security fixes also some out of tree modules get included (ZFS, igb, e1000e)
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.

cheers,
Thomas
Uwe Sauter
2018-08-22 07:58:53 UTC
Permalink
Hi Thomas,
Post by Thomas Lamprecht
Hi Uwe,
Post by Uwe Sauter
Hi all,
* As far as I can tell the PVE kernel is a modified version of Ubuntu kernels, correct?
Modifications can be viewed in the pve-kernel.git repository ( https://git.proxmox.com/?p=pve-kernel.git;a=tree ).
Yes, in the respective git branch (master is currently 4.13 and pve-kernel-4.15 is, you guess it, 4.15
patches/ includes on-top bug/security fixes also some out of tree modules get included (ZFS, igb, e1000e)
I'm mostly interested in the myri10ge driver right now. From what I can tell, you do ship this particular driver without modification?
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…


Thanks,

Uwe
Post by Thomas Lamprecht
cheers,
Thomas
Thomas Lamprecht
2018-08-22 08:24:15 UTC
Permalink
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
Hi all,
* As far as I can tell the PVE kernel is a modified version of Ubuntu kernels, correct?
Modifications can be viewed in the pve-kernel.git repository ( https://git.proxmox.com/?p=pve-kernel.git;a=tree ).
Yes, in the respective git branch (master is currently 4.13 and pve-kernel-4.15 is, you guess it, 4.15
patches/ includes on-top bug/security fixes also some out of tree modules get included (ZFS, igb, e1000e)
I'm mostly interested in the myri10ge driver right now. From what I can tell, you do ship this particular driver without modification?
You're right, no modifications of this module.

# modinfo myri10ge
filename: /lib/modules/4.15.18-2-pve/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko
firmware: myri10ge_rss_eth_z8e.dat
firmware: myri10ge_rss_ethp_z8e.dat
firmware: myri10ge_eth_z8e.dat
firmware: myri10ge_ethp_z8e.dat
license: Dual BSD/GPL
version: 1.5.3-1.534
author: Maintainer: ***@myri.com
description: Myricom 10G driver (10GbE)
srcversion: 46526E4E4E82667CBFF2D7C
alias: pci:v000014C1d00000009sv*sd*bc*sc*i*
alias: pci:v000014C1d00000008sv*sd*bc*sc*i*
depends: dca
retpoline: Y
intree: Y
name: myri10ge
vermagic: 4.15.18-2-pve SMP mod_unload modversions
parm: myri10ge_fw_name:Firmware image name (charp)
parm: myri10ge_fw_names:Firmware image names per board (array of charp)
parm: myri10ge_ecrc_enable:Enable Extended CRC on PCI-E (int)
parm: myri10ge_small_bytes:Threshold of small packets (int)
parm: myri10ge_msi:Enable Message Signalled Interrupts (int)
parm: myri10ge_intr_coal_delay:Interrupt coalescing delay (int)
parm: myri10ge_flow_control:Pause parameter (int)
parm: myri10ge_deassert_wait:Wait when deasserting legacy interrupts (int)
parm: myri10ge_force_firmware:Force firmware to assume aligned completions (int)
parm: myri10ge_initial_mtu:Initial MTU (int)
parm: myri10ge_napi_weight:Set NAPI weight (int)
parm: myri10ge_watchdog_timeout:Set watchdog timeout (int)
parm: myri10ge_max_irq_loops:Set stuck legacy IRQ detection threshold (int)
parm: myri10ge_debug:Debug level (0=none,...,16=all) (int)
parm: myri10ge_fill_thresh:Number of empty rx slots allowed (int)
parm: myri10ge_max_slices:Max tx/rx queues (int)
parm: myri10ge_rss_hash:Type of RSS hashing to do (int)
parm: myri10ge_dca:Enable DCA if possible (int)
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…
What's your exact issue, if I may ask?

cheers,
Thomas
Uwe Sauter
2018-08-22 08:50:19 UTC
Permalink
Post by Thomas Lamprecht
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…
What's your exact issue, if I may ask?
cheers,
Thomas
Short story is that since updating from 4.13 to 4.15 I get slow_requests in Ceph with only low load on Ceph. If I boot back, those
are gone.

Or at least almost as I was able to produce slow_requests with 4.13 but only if deep scrubing was manually initiated on all PGs.


Sage Weil (Ceph dev) suggested that this probably is MTU or bonding related and thus I'm currently testing with different
settings. Another guy on the ceph-devel list suggested different driver versions so I was checking that as well.



Long story is here:

https://pve.proxmox.com/pipermail/pve-user/2018-May/169472.html

https://pve.proxmox.com/pipermail/pve-user/2018-May/169492.html

http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/026627.html

http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-August/028862.html

https://marc.info/?l=ceph-devel&m=153449419830984&w=2
Marcus Haarmann
2018-08-22 09:08:20 UTC
Permalink
Hi,
did you try this:
(taken from the ceph list)
This is PTI, I think.  Try to add "noibrs noibpb nopti nospectre_v2" to
kernel cmdline and reboot.

Did this make a difference ?
We are struggeling with proxmox/ceph too and we have the suspect that it is kernel related or network related,
but could not narrow it down to a specific reason.
But the effects are different... We encountered stuck I/O on rdb devices.
And kernel says it is losing a mon connection and hunting for a new mon all the time (when backup takes
place and heavy I/O is done).

Marcus Haarmann


Von: "uwe sauter de" <***@gmail.com>
An: "Thomas Lamprecht" <***@proxmox.com>, "pve-user" <pve-***@pve.proxmox.com>
Gesendet: Mittwoch, 22. August 2018 10:50:19
Betreff: Re: [PVE-User] PVE kernel (sources, build process, etc.)
Post by Thomas Lamprecht
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…
What's your exact issue, if I may ask?
cheers,
Thomas
Short story is that since updating from 4.13 to 4.15 I get slow_requests in Ceph with only low load on Ceph. If I boot back, those
are gone.

Or at least almost as I was able to produce slow_requests with 4.13 but only if deep scrubing was manually initiated on all PGs.


Sage Weil (Ceph dev) suggested that this probably is MTU or bonding related and thus I'm currently testing with different
settings. Another guy on the ceph-devel list suggested different driver versions so I was checking that as well.



Long story is here:

https://pve.proxmox.com/pipermail/pve-user/2018-May/169472.html

https://pve.proxmox.com/pipermail/pve-user/2018-May/169492.html

http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/026627.html

http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-August/028862.html

https://marc.info/?l=ceph-devel&m=153449419830984&w=2
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Uwe Sauter
2018-08-22 09:45:15 UTC
Permalink
Hi Marcus,

no, I haven't disabled Spectre/Meltdown mitigations (yet).

For 4 of my hosts I have the Intel microcode from 2018-05-08 running which is the most up-to-date version from Ubuntu.

Using the spectre-meltdown-checker.sh script from https://github.com/speed47/spectre-meltdown-checker gives below output which
let's me believe that performace should be goot with mitigations enabled.

#### output for sandybridge based hosts ####
CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB, IBRS_FW)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: YES (for kernel and firmware code)
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: YES
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
STATUS: NOT VULNERABLE (Full retpoline + IBPB are mitigating the vulnerability)
CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports PCID, performance impact of PTI will be reduced)
* Running as a Xen PV DomU: NO
STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 [rogue system register read] aka 'Variant 3a'
* CPU microcode mitigates the vulnerability: YES
STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 [speculative store bypass] aka 'Variant 4'
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
* Kernel supports speculation store bypass: YES (found in /proc/self/status)
STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615/3620/3646 [L1 terminal fault] aka 'Foreshadow & Foreshadow-NG'
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
##### end #####


The other 2 hosts are base on Westmere architecture and won't get microcode updates as fas as I know. Disabling mitigations on
these hosts might be worth a try. Same script from above gives:

#### output of westmere based hosts ####
CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: NO
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: NO
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
STATUS: NOT VULNERABLE (Full retpoline is mitigating the vulnerability)
IBPB is considered as a good addition to retpoline for Variant 2 mitigation, but your CPU microcode doesn't support it

CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports PCID, performance impact of PTI will be reduced)
* Running as a Xen PV DomU: NO
STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 [rogue system register read] aka 'Variant 3a'
* CPU microcode mitigates the vulnerability: NO
STATUS: VULNERABLE (an up-to-date CPU microcode is needed to mitigate this vulnerability)
CVE-2018-3639 [speculative store bypass] aka 'Variant 4'
* Mitigated according to the /sys interface: NO (Vulnerable)
* Kernel supports speculation store bypass: YES (found in /proc/self/status)
STATUS: VULNERABLE (Your CPU doesn't support SSBD)
CVE-2018-3615/3620/3646 [L1 terminal fault] aka 'Foreshadow & Foreshadow-NG'
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT disabled)
STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT disabled)
##### end #####
Hi,
(taken from the ceph list)
This is PTI, I think.  Try to add "noibrs noibpb nopti nospectre_v2" to
kernel cmdline and reboot.
Did this make a difference ?
We are struggeling with proxmox/ceph too and we have the suspect that it is kernel related or network related,
but could not narrow it down to a specific reason.
But the effects are different... We encountered stuck I/O on rdb devices.
And kernel says it is losing a mon connection and hunting for a new mon all the time (when backup takes
place and heavy I/O is done).
Marcus Haarmann
----------------------------------------------------------------------------------------------------------------------------------
*Gesendet: *Mittwoch, 22. August 2018 10:50:19
*Betreff: *Re: [PVE-User] PVE kernel (sources, build process, etc.)
Post by Thomas Lamprecht
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…
What's your exact issue, if I may ask?
cheers,
Thomas
Short story is that since updating from 4.13 to 4.15 I get slow_requests in Ceph with only low load on Ceph. If I boot back, those
are gone.
Or at least almost as I was able to produce slow_requests with 4.13 but only if deep scrubing was manually initiated on all PGs.
Sage Weil (Ceph dev) suggested that this probably is MTU or bonding related and thus I'm currently testing with different
settings. Another guy on the ceph-devel list suggested different driver versions so I was checking that as well.
https://pve.proxmox.com/pipermail/pve-user/2018-May/169472.html
https://pve.proxmox.com/pipermail/pve-user/2018-May/169492.html
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/026627.html
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-August/028862.html
https://marc.info/?l=ceph-devel&m=153449419830984&w=2
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Uwe Sauter
2018-08-22 12:14:48 UTC
Permalink
One thing speaks againts this being PTI is that both types of nodes have secondary OSDs causing slow requests.

Though it still is an option to try before giving up completely.
Post by Uwe Sauter
Hi Marcus,
no, I haven't disabled Spectre/Meltdown mitigations (yet).
For 4 of my hosts I have the Intel microcode from 2018-05-08 running which is the most up-to-date version from Ubuntu.
Using the spectre-meltdown-checker.sh script from https://github.com/speed47/spectre-meltdown-checker gives below output which
let's me believe that performace should be goot with mitigations enabled.
#### output for sandybridge based hosts ####
CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline, IBPB, IBRS_FW)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: YES (for kernel and firmware code)
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: YES
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
STATUS: NOT VULNERABLE (Full retpoline + IBPB are mitigating the vulnerability)
CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports PCID, performance impact of PTI will be reduced)
* Running as a Xen PV DomU: NO
STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 [rogue system register read] aka 'Variant 3a'
* CPU microcode mitigates the vulnerability: YES
STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)
CVE-2018-3639 [speculative store bypass] aka 'Variant 4'
* Mitigated according to the /sys interface: YES (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
* Kernel supports speculation store bypass: YES (found in /proc/self/status)
STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
CVE-2018-3615/3620/3646 [L1 terminal fault] aka 'Foreshadow & Foreshadow-NG'
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable)
##### end #####
The other 2 hosts are base on Westmere architecture and won't get microcode updates as fas as I know. Disabling mitigations on
#### output of westmere based hosts ####
CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec: YES (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
* Kernel has mask_nospec64 (arm64): NO
STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)
CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface: YES (Mitigation: Full generic retpoline)
* Mitigation 1
* Kernel is compiled with IBRS support: YES
* IBRS enabled and active: NO
* Kernel is compiled with IBPB support: YES
* IBPB enabled and active: NO
* Mitigation 2
* Kernel has branch predictor hardening (arm): NO
* Kernel compiled with retpoline option: YES
* Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
STATUS: NOT VULNERABLE (Full retpoline is mitigating the vulnerability)
IBPB is considered as a good addition to retpoline for Variant 2 mitigation, but your CPU microcode doesn't support it
CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface: YES (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI): YES
* PTI enabled and active: YES
* Reduced performance impact of PTI: YES (CPU supports PCID, performance impact of PTI will be reduced)
* Running as a Xen PV DomU: NO
STATUS: NOT VULNERABLE (Mitigation: PTI)
CVE-2018-3640 [rogue system register read] aka 'Variant 3a'
* CPU microcode mitigates the vulnerability: NO
STATUS: VULNERABLE (an up-to-date CPU microcode is needed to mitigate this vulnerability)
CVE-2018-3639 [speculative store bypass] aka 'Variant 4'
* Mitigated according to the /sys interface: NO (Vulnerable)
* Kernel supports speculation store bypass: YES (found in /proc/self/status)
STATUS: VULNERABLE (Your CPU doesn't support SSBD)
CVE-2018-3615/3620/3646 [L1 terminal fault] aka 'Foreshadow & Foreshadow-NG'
* Mitigated according to the /sys interface: YES (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT disabled)
STATUS: NOT VULNERABLE (Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT disabled)
##### end #####
Hi,
(taken from the ceph list)
This is PTI, I think.  Try to add "noibrs noibpb nopti nospectre_v2" to
kernel cmdline and reboot.
Did this make a difference ?
We are struggeling with proxmox/ceph too and we have the suspect that it is kernel related or network related,
but could not narrow it down to a specific reason.
But the effects are different... We encountered stuck I/O on rdb devices.
And kernel says it is losing a mon connection and hunting for a new mon all the time (when backup takes
place and heavy I/O is done).
Marcus Haarmann
----------------------------------------------------------------------------------------------------------------------------------
*Gesendet: *Mittwoch, 22. August 2018 10:50:19
*Betreff: *Re: [PVE-User] PVE kernel (sources, build process, etc.)
Post by Thomas Lamprecht
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…
What's your exact issue, if I may ask?
cheers,
Thomas
Short story is that since updating from 4.13 to 4.15 I get slow_requests in Ceph with only low load on Ceph. If I boot back, those
are gone.
Or at least almost as I was able to produce slow_requests with 4.13 but only if deep scrubing was manually initiated on all PGs.
Sage Weil (Ceph dev) suggested that this probably is MTU or bonding related and thus I'm currently testing with different
settings. Another guy on the ceph-devel list suggested different driver versions so I was checking that as well.
https://pve.proxmox.com/pipermail/pve-user/2018-May/169472.html
https://pve.proxmox.com/pipermail/pve-user/2018-May/169492.html
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/026627.html
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-August/028862.html
https://marc.info/?l=ceph-devel&m=153449419830984&w=2
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Uwe Sauter
2018-09-10 09:35:23 UTC
Permalink
Hi,

I rebooted the two "old" hosts (Westmere EP generation) with the cmdline options. But for the "new" hosts I don't feel comfortable
to apply the options as those also host the VMs.

Now all hosts are on 4.15.18-4-pve and it seems that it is harder to trigger the issue. But once triggered, it is an OSD running
on one of the newer hosts that doesn't commit the subop.


Regards,

Uwe
Post by Marcus Haarmann
Hi,
(taken from the ceph list)
This is PTI, I think.  Try to add "noibrs noibpb nopti nospectre_v2" to
kernel cmdline and reboot.
Did this make a difference ?
We are struggeling with proxmox/ceph too and we have the suspect that it is kernel related or network related,
but could not narrow it down to a specific reason.
But the effects are different... We encountered stuck I/O on rdb devices.
And kernel says it is losing a mon connection and hunting for a new mon all the time (when backup takes
place and heavy I/O is done).
Marcus Haarmann
----------------------------------------------------------------------------------------------------------------------------------
*Gesendet: *Mittwoch, 22. August 2018 10:50:19
*Betreff: *Re: [PVE-User] PVE kernel (sources, build process, etc.)
Post by Thomas Lamprecht
Post by Uwe Sauter
Post by Thomas Lamprecht
Post by Uwe Sauter
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
Yes. (Note that this may not get much updates anymore)
Post by Uwe Sauter
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Yes. We're normally on the latest stable release tagged on the master branch.
I'll checkout both and compare the myri10ge drivers…
What's your exact issue, if I may ask?
cheers,
Thomas
Short story is that since updating from 4.13 to 4.15 I get slow_requests in Ceph with only low load on Ceph. If I boot back, those
are gone.
Or at least almost as I was able to produce slow_requests with 4.13 but only if deep scrubing was manually initiated on all PGs.
Sage Weil (Ceph dev) suggested that this probably is MTU or bonding related and thus I'm currently testing with different
settings. Another guy on the ceph-devel list suggested different driver versions so I was checking that as well.
https://pve.proxmox.com/pipermail/pve-user/2018-May/169472.html
https://pve.proxmox.com/pipermail/pve-user/2018-May/169492.html
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/026627.html
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-August/028862.html
https://marc.info/?l=ceph-devel&m=153449419830984&w=2
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Marcus Haarmann
2018-08-22 07:57:02 UTC
Permalink
Hi,

yes we tried to work with a standard kernel (Debian) and this was a very wrong decision.
All the CT did not work any more....
You need to apply the patches. The process of compiling a new one is documented,
but you will need some time and lots of additional packages to do this.
(and of course some programming knowledge)

Marcus Haarmann


Von: "Uwe Sauter" <***@gmail.com>
An: "pve-user" <pve-***@pve.proxmox.com>
Gesendet: Mittwoch, 22. August 2018 09:48:19
Betreff: [PVE-User] PVE kernel (sources, build process, etc.)

Hi all,

some quick questions:

* As far as I can tell the PVE kernel is a modified version of Ubuntu kernels, correct?
Modifications can be viewed in the pve-kernel.git repository (https://git.proxmox.com/?p=pve-kernel.git;a=tree).

* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?

* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?


Thanks,

Uwe
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Uwe Sauter
2018-08-22 08:02:38 UTC
Permalink
Marcus,

thanks. Hopefully it won't come as far as compiling my own pve-kernel package. I'm currently hunting a networking issue and need
to compare driver versions…


Regards,

Uwe
Post by Marcus Haarmann
Hi,
yes we tried to work with a standard kernel (Debian) and this was a very wrong decision.
All the CT did not work any more....
You need to apply the patches. The process of compiling a new one is documented,
but you will need some time and lots of additional packages to do this.
(and of course some programming knowledge)
Marcus Haarmann
----------------------------------------------------------------------------------------------------------------------------------
*Gesendet: *Mittwoch, 22. August 2018 09:48:19
*Betreff: *[PVE-User] PVE kernel (sources, build process, etc.)
Hi all,
* As far as I can tell the PVE kernel is a modified version of Ubuntu kernels, correct?
  Modifications can be viewed in the pve-kernel.git repository (https://git.proxmox.com/?p=pve-kernel.git;a=tree).
* pve-kernel 4.13 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-artful.git/ ?
* pve-kernel 4.15 is based on http://kernel.ubuntu.com/git/ubuntu/ubuntu-bionic.git/ ?
Thanks,
        Uwe
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Loading...