Discussion:
[PVE-User] Cant connect to ceph anymore
p***@elchaka.de
2018-09-13 17:41:13 UTC
Permalink
Hi,

I had a proxmox 4.x Server with installed ceph Jewel (10.x) via ceph-deploy.

The ceph Cluster is standalone Not managed by proxmox.

I could use this ceph Cluster without any issues and the vms working fine.

Now I upgraded my ceph Cluster to 12.2.x and was still able to connect my vms via proxmox (qemu/librbd).

The last Step was to Update the ceph Client on this proxmox node, but now i am Not able to connect to the ceph Storage anymore :*(


Actually i cannot Update the proxmox Node to 5.x because there is a important vm Running with local Storage.

Proxmox Show me a empty list when i want to Browse the ceph Cluster through the gui - but i dont get any error.

Any suggestions?

Thanks in advise
- Mehmet
Marcus Haarmann
2018-09-13 20:58:04 UTC
Permalink
Hi,

so you would drive a 12.2 (luminous) service with a 10.x (jewel) client.
Just a thought:

Check the "ceph features" output on your server if it allows jewel clients.
Sample output should contain a group with a jewel attribute in the client section.

If this is not the case, try to set:
ceph osd set-require-min-compat-client jewel
(which should be default for luminous, but can be removed which leads to incompability)

Check this link for details: https://ceph.com/community/new-luminous-upgrade-complete/

Maybe this helps, good luck
Marcus Haarmann


Von: ***@elchaka.de
An: "pve-user" <pve-***@pve.proxmox.com>
Gesendet: Donnerstag, 13. September 2018 19:41:13
Betreff: [PVE-User] Cant connect to ceph anymore

Hi,

I had a proxmox 4.x Server with installed ceph Jewel (10.x) via ceph-deploy.

The ceph Cluster is standalone Not managed by proxmox.

I could use this ceph Cluster without any issues and the vms working fine.

Now I upgraded my ceph Cluster to 12.2.x and was still able to connect my vms via proxmox (qemu/librbd).

The last Step was to Update the ceph Client on this proxmox node, but now i am Not able to connect to the ceph Storage anymore :*(


Actually i cannot Update the proxmox Node to 5.x because there is a important vm Running with local Storage.

Proxmox Show me a empty list when i want to Browse the ceph Cluster through the gui - but i dont get any error.

Any suggestions?

Thanks in advise
- Mehmet
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
p***@elchaka.de
2018-09-13 21:59:32 UTC
Permalink
Hey Marcus,

Thanks for your Message.
Post by Marcus Haarmann
Hi,
so you would drive a 12.2 (luminous) service with a 10.x (jewel) client.
Sorry, with

"The last Step was to Update the ceph Client on this proxmox node, but now i am Not able to connect to the ceph Storage anymore :*("

I ment ive upgraded this Node to 12.2.x, too. After this i had the descriped issue.
Post by Marcus Haarmann
Check the "ceph features" output on your server if it allows jewel clients.
Sample output should contain a group with a jewel attribute in the client section.
This is clean. All other Node are already luminous (but with PVE 5.x)
Post by Marcus Haarmann
ceph osd set-require-min-compat-client jewel
It is already On Jewel, becouse ive upgraded from Jewel but didnt set this to luminous till all Node are Running luminous fine.

Hope you or someone Else has Another idea why i can not See my Images through the proxmox GUI anymore - and so i can not start my appropriate vms.

- Mehmet
Post by Marcus Haarmann
(which should be default for luminous, but can be removed which leads to incompability)
https://ceph.com/community/new-luminous-upgrade-complete/
Maybe this helps, good luck
Marcus Haarmann
Gesendet: Donnerstag, 13. September 2018 19:41:13
Betreff: [PVE-User] Cant connect to ceph anymore
Hi,
I had a proxmox 4.x Server with installed ceph Jewel (10.x) via
ceph-deploy.
The ceph Cluster is standalone Not managed by proxmox.
I could use this ceph Cluster without any issues and the vms working fine.
Now I upgraded my ceph Cluster to 12.2.x and was still able to connect
my vms via proxmox (qemu/librbd).
The last Step was to Update the ceph Client on this proxmox node, but
now i am Not able to connect to the ceph Storage anymore :*(
Actually i cannot Update the proxmox Node to 5.x because there is a
important vm Running with local Storage.
Proxmox Show me a empty list when i want to Browse the ceph Cluster
through the gui - but i dont get any error.
Any suggestions?
Thanks in advise
- Mehmet
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Alwin Antreich
2018-09-14 05:00:06 UTC
Permalink
Hi,
Post by p***@elchaka.de
Hey Marcus,
Thanks for your Message.
Am 13. September 2018 22:58:04 MESZ schrieb Marcus Haarmann <
Post by Marcus Haarmann
Hi,
so you would drive a 12.2 (luminous) service with a 10.x (jewel) client.
Sorry, with
"The last Step was to Update the ceph Client on this proxmox node, but now
i am Not able to connect to the ceph Storage anymore :*("
I ment ive upgraded this Node to 12.2.x, too. After this i had the descriped issue.
In 12.2.8 the output for'rbd ls' and other commands had a slight change,
units GB to GiB. For your PVE 5.x, if you're on the latest packages it
should work to connect.
Post by p***@elchaka.de
Post by Marcus Haarmann
Check the "ceph features" output on your server if it allows jewel clients.
Sample output should contain a group with a jewel attribute in the client section.
This is clean. All other Node are already luminous (but with PVE 5.x)
Post by Marcus Haarmann
ceph osd set-require-min-compat-client jewel
It is already On Jewel, becouse ive upgraded from Jewel but didnt set this
to luminous till all Node are Running luminous fine.
Hope you or someone Else has Another idea why i can not See my Images
through the proxmox GUI anymore - and so i can not start my appropriate vms.
- Mehmet
Post by Marcus Haarmann
(which should be default for luminous, but can be removed which leads to incompability)
https://ceph.com/community/new-luminous-upgrade-complete/
Maybe this helps, good luck
Marcus Haarmann
Gesendet: Donnerstag, 13. September 2018 19:41:13
Betreff: [PVE-User] Cant connect to ceph anymore
Hi,
I had a proxmox 4.x Server with installed ceph Jewel (10.x) via ceph-deploy.
The ceph Cluster is standalone Not managed by proxmox.
I could use this ceph Cluster without any issues and the vms working fine.
Now I upgraded my ceph Cluster to 12.2.x and was still able to connect
my vms via proxmox (qemu/librbd).
The last Step was to Update the ceph Client on this proxmox node, but
now i am Not able to connect to the ceph Storage anymore :*(
Actually i cannot Update the proxmox Node to 5.x because there is a
important vm Running with local Storage.
You need to migrate the VM to another node (storage), as you need to get
the node to PVE 5.x. IIRC in PVE4.4, from the command line, you could try
to use the 'with_local_storage' option in 'qm migrate'. Or move the disks
to NFS and then to another node.

--
Cheers,
Alwin
Marcus Haarmann
2018-09-14 06:00:17 UTC
Permalink
Hi,

so the client is updated to luminous but kernel not (?)
Maybe you should downgrade for the moment, as jewel clients are supported.

Marcus Haarmann


Von: "proxmox" <***@elchaka.de>
An: "pve-user" <pve-***@pve.proxmox.com>
Gesendet: Donnerstag, 13. September 2018 23:59:32
Betreff: Re: [PVE-User] Cant connect to ceph anymore

Hey Marcus,

Thanks for your Message.
Post by Marcus Haarmann
Hi,
so you would drive a 12.2 (luminous) service with a 10.x (jewel) client.
Sorry, with

"The last Step was to Update the ceph Client on this proxmox node, but now i am Not able to connect to the ceph Storage anymore :*("

I ment ive upgraded this Node to 12.2.x, too. After this i had the descriped issue.
Post by Marcus Haarmann
Check the "ceph features" output on your server if it allows jewel clients.
Sample output should contain a group with a jewel attribute in the client section.
This is clean. All other Node are already luminous (but with PVE 5.x)
Post by Marcus Haarmann
ceph osd set-require-min-compat-client jewel
It is already On Jewel, becouse ive upgraded from Jewel but didnt set this to luminous till all Node are Running luminous fine.

Hope you or someone Else has Another idea why i can not See my Images through the proxmox GUI anymore - and so i can not start my appropriate vms.

- Mehmet
Post by Marcus Haarmann
(which should be default for luminous, but can be removed which leads to incompability)
https://ceph.com/community/new-luminous-upgrade-complete/
Maybe this helps, good luck
Marcus Haarmann
Gesendet: Donnerstag, 13. September 2018 19:41:13
Betreff: [PVE-User] Cant connect to ceph anymore
Hi,
I had a proxmox 4.x Server with installed ceph Jewel (10.x) via
ceph-deploy.
The ceph Cluster is standalone Not managed by proxmox.
I could use this ceph Cluster without any issues and the vms working fine.
Now I upgraded my ceph Cluster to 12.2.x and was still able to connect
my vms via proxmox (qemu/librbd).
The last Step was to Update the ceph Client on this proxmox node, but
now i am Not able to connect to the ceph Storage anymore :*(
Actually i cannot Update the proxmox Node to 5.x because there is a
important vm Running with local Storage.
Proxmox Show me a empty list when i want to Browse the ceph Cluster
through the gui - but i dont get any error.
Any suggestions?
Thanks in advise
- Mehmet
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
_______________________________________________
pve-user mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
p***@elchaka.de
2018-09-15 23:26:36 UTC
Permalink
Hello Marcus,

Thanks again for your attention :)

Downgrade is Really a passable Way - dont know why i didnt come to this conclusion by myself ^^

So i tried to Downgrade via ceph-deploy install --release Jewel <host> but didnt worked so i decide to remove and reinstal ceph again after this.

So i did
Ceph-deploy remove ceph (dont know the exact Commands Jet...)
this worked but this had also removed proxmox components like proxmox-ve and qemu-server etc. Oo :*(

Luckywise this hadnt any effect to our Running VMs.. Puh..

So i tried to reinstall the removed proxmox components but this is actually Not allowed because some libaries are in use ...

I guess i had to Reboot the Server in a Controlled way and then to instal proxmox again hopefully without loosing the actually Running VMs and local Storage.

- Mehmet
Post by Marcus Haarmann
Hi,
so the client is updated to luminous but kernel not (?)
Maybe you should downgrade for the moment, as jewel clients are
supported.
Marcus Haarmann
Gesendet: Donnerstag, 13. September 2018 23:59:32
Betreff: Re: [PVE-User] Cant connect to ceph anymore
Hey Marcus,
Thanks for your Message.
Am 13. September 2018 22:58:04 MESZ schrieb Marcus Haarmann
Post by Marcus Haarmann
Hi,
so you would drive a 12.2 (luminous) service with a 10.x (jewel) client.
Sorry, with
"The last Step was to Update the ceph Client on this proxmox node, but
now i am Not able to connect to the ceph Storage anymore :*("
I ment ive upgraded this Node to 12.2.x, too. After this i had the descriped issue.
Post by Marcus Haarmann
Check the "ceph features" output on your server if it allows jewel clients.
Sample output should contain a group with a jewel attribute in the client section.
This is clean. All other Node are already luminous (but with PVE 5.x)
Post by Marcus Haarmann
ceph osd set-require-min-compat-client jewel
It is already On Jewel, becouse ive upgraded from Jewel but didnt set
this to luminous till all Node are Running luminous fine.
Hope you or someone Else has Another idea why i can not See my Images
through the proxmox GUI anymore - and so i can not start my appropriate
vms.
- Mehmet
Post by Marcus Haarmann
(which should be default for luminous, but can be removed which leads to incompability)
https://ceph.com/community/new-luminous-upgrade-complete/
Maybe this helps, good luck
Marcus Haarmann
Gesendet: Donnerstag, 13. September 2018 19:41:13
Betreff: [PVE-User] Cant connect to ceph anymore
Hi,
I had a proxmox 4.x Server with installed ceph Jewel (10.x) via ceph-deploy.
The ceph Cluster is standalone Not managed by proxmox.
I could use this ceph Cluster without any issues and the vms working fine.
Now I upgraded my ceph Cluster to 12.2.x and was still able to connect
my vms via proxmox (qemu/librbd).
The last Step was to Update the ceph Client on this proxmox node, but
now i am Not able to connect to the ceph Storage anymore :*(
Actually i cannot Update the proxmox Node to 5.x because there is a
important vm Running with local Storage.
Proxmox Show me a empty list when i want to Browse the ceph Cluster
through the gui - but i dont get any error.
Any suggestions?
Thanks in advise
- Mehmet
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
_______________________________________________
pve-user mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
Loading...