[PVE-User] how many nodes can there be in a proxmox cluster ?

[Alexandre DERUMIER]

I'm running 20 nodes clusters here, with unicast. (I need to increase corosync timeout a little bit with more than 12 nodes).

I think the hard limit is around 100nodes in last corosync, but recommendations is around 32. with more than that, I think the corosync timeout need to be increase high and you'll have latency.



De: "Ronny Aasen" <ronny+pve-***@aasen.cx>
À: "proxmoxve" <pve-***@pve.proxmox.com>
Envoyé: Mardi 20 Mars 2018 21:30:13
Objet: [PVE-User] how many nodes can there be in a proxmox cluster ?

once around proxmox 2 i read something about a hard limit about 32
nodes, due to corosync limits. but that there should probably not be
that many.

i have not been able to find any more recent information.

is there a hard or soft limit to how large a proxmox cluster you can
have ? what is the largest installations people are running. and does
this affect stabillity of the cluster?


kind regards

Ronny Aasen

_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[Josh Knight]

Hey Alexandre,

When you say 20 nodes with unicast - you're saying they're across subnets?

I ask because we've been looking at doing this, more for a vCenter-like
single console to manage all VMs and hosts instead of HA, but I thought I
read on the proxmox wiki that it wasn't recommended for larger clusters?
Does it work well for you at 20 nodes or are there any config tweaks or
gotchas that you've run into?

Thanks,

Josh


Josh Knight

Post by Alexandre DERUMIER
I'm running 20 nodes clusters here, with unicast. (I need to increase
corosync timeout a little bit with more than 12 nodes).
I think the hard limit is around 100nodes in last corosync, but
recommendations is around 32. with more than that, I think the corosync
timeout need to be increase high and you'll have latency.
Envoyé: Mardi 20 Mars 2018 21:30:13
Objet: [PVE-User] how many nodes can there be in a proxmox cluster ?
once around proxmox 2 i read something about a hard limit about 32
nodes, due to corosync limits. but that there should probably not be
that many.
i have not been able to find any more recent information.
is there a hard or soft limit to how large a proxmox cluster you can
have ? what is the largest installations people are running. and does
this affect stabillity of the cluster?
kind regards
Ronny Aasen
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[Alexandre DERUMIER]

Post by Alexandre DERUMIER

Post by Josh Knight
When you say 20 nodes with unicast - you're saying they're across subnets?

no , currently, same subnet, on same pair of switch (mlag).
It's a quite new mellanox switch s2100, with fast asics. (avg latency 0.015ms)


I have too much problem with multicast in my infra.

no special tuning.


i had a older cluster, where in past I had to tune

token: 4000

in corosync config.


I'm planning to test it across differents datacenter soon (1ms latency), but I don't think it'll possible
with theses latencies

----- Mail original -----
De: "Josh Knight" <***@noobbox.com>
À: "proxmoxve" <pve-***@pve.proxmox.com>
Envoyé: Mercredi 21 Mars 2018 13:28:12
Objet: Re: [PVE-User] how many nodes can there be in a proxmox cluster ?

Hey Alexandre,

When you say 20 nodes with unicast - you're saying they're across subnets?

I ask because we've been looking at doing this, more for a vCenter-like
single console to manage all VMs and hosts instead of HA, but I thought I
read on the proxmox wiki that it wasn't recommended for larger clusters?
Does it work well for you at 20 nodes or are there any config tweaks or
gotchas that you've run into?

Thanks,

Josh


Josh Knight

Post by Alexandre DERUMIER
I'm running 20 nodes clusters here, with unicast. (I need to increase
corosync timeout a little bit with more than 12 nodes).
I think the hard limit is around 100nodes in last corosync, but
recommendations is around 32. with more than that, I think the corosync
timeout need to be increase high and you'll have latency.
Envoyé: Mardi 20 Mars 2018 21:30:13
Objet: [PVE-User] how many nodes can there be in a proxmox cluster ?
once around proxmox 2 i read something about a hard limit about 32
nodes, due to corosync limits. but that there should probably not be
that many.
i have not been able to find any more recent information.
is there a hard or soft limit to how large a proxmox cluster you can
have ? what is the largest installations people are running. and does
this affect stabillity of the cluster?
kind regards
Ronny Aasen
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user