[PVE-User] ERROR: unable to connect to VM 105 qmp socket

Discussion:

[PVE-User] ERROR: unable to connect to VM 105 qmp socket - timeout after 31 retries

[SOLTECSIS] Carles Xavier Munyoz Baldó

2015-05-05 16:50:42 UTC

Hello,
I have just licensed and upgraded a promox server to the last version
3.4-3. Before the upgrade I has the version 3.1.

A few hours after upgrade, I have had problems with two different
virtual machines, one with Windows and the other one with Linux. The
virtual machines have been stopped working. I have tried to access the
console of them but it was not possible and in the /var/log/syslogfile I
have seen errors like these:
[...]
May 5 13:23:43 bra1 pvedaemon[40507]: unable to connect to VM 105 qmp
socket - timeout after 31 retries
May 5 13:23:44 bra1 pvedaemon[40724]: unable to connect to VM 105 qmp
socket - timeout after 31 retries
May 5 13:23:46 bra1 pvedaemon[40507]: unable to connect to VM 105 qmp
socket - timeout after 31 retries
May 5 13:23:50 bra1 pvedaemon[40507]: unable to connect to VM 105 qmp
socket - timeout after 31 retries
May 5 13:23:54 bra1 pvedaemon[40507]: unable to connect to VM 105 qmp
socket - timeout after 31 retries
[...]

The only option I had is to stop the virtual machine and launch it again.

These virtual machines where going fine in the Proxmox 3.1 version.
Which is the reason for which I'm having these problems now with the
last Proxmox version?

Best regards.
========================================
SOLTECSIS SOLUCIONES TECNOLOGICAS, S.L.
Carles Xavier Munyoz Baldó
Departamento de I+D+I
Tel./Fax: 966 446 046
***@soltecsis.com
www.soltecsis.com
========================================

---
La información contenida en este e-mail es confidencial,
siendo para uso exclusivo del destinatario arriba mencionado.
Le informamos que está totalmente prohibida cualquier
utilización, divulgación, distribución y/o reproducción de
esta comunicación sin autorización expresa en virtud de la
legislación vigente. Si ha recibido este mensaje por error,
le rogamos nos lo notifique inmediatamente por la misma vía
y proceda a su eliminación.
---

Michael Rasmussen

2015-05-05 17:00:44 UTC

Permalink

On Tue, 05 May 2015 18:50:42 +0200

Post by [SOLTECSIS] Carles Xavier Munyoz BaldÃ³
A few hours after upgrade, I have had problems with two different
virtual machines, one with Windows and the other one with Linux. The
virtual machines have been stopped working. I have tried to access the
console of them but it was not possible and in the /var/log/syslogfile I

Did you stop and start the VM's after updating proxmox?

proxmox 3.1 -> 3.4 includes a major upgrade of qemu why a stop and
start of the VM's is required.
--
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
REST:
P: Linus Torvalds
S: Buried alive in email
-- from /usr/src/linux/MAINTAINERS

[SOLTECSIS] Carles Xavier Munyoz Baldó

2015-05-05 17:14:30 UTC

Permalink

Hello,
Yes, I rebooted the entire host server because in the updates where
included a new kernel.

The problem arises after the upgrade and host server reboot. And it is
not inmediatly after the virtual machines boot, it has happen after
several hours of normal operation and the two virtual machines where not
affected at the same time.

Thank you for your help.

Post by Michael Rasmussen
On Tue, 05 May 2015 18:50:42 +0200

Did you stop and start the VM's after updating proxmox?
proxmox 3.1 -> 3.4 includes a major upgrade of qemu why a stop and
start of the VM's is required.
_______________________________________________
pve-user mailing list
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

========================================
SOLTECSIS SOLUCIONES TECNOLOGICAS, S.L.
Carles Xavier Munyoz Baldó
Departamento de I+D+I
Tel./Fax: 966 446 046
***@soltecsis.com
www.soltecsis.com
========================================

---
La información contenida en este e-mail es confidencial,
siendo para uso exclusivo del destinatario arriba mencionado.
Le informamos que está totalmente prohibida cualquier
utilización, divulgación, distribución y/o reproducción de
esta comunicación sin autorización expresa en virtud de la
legislación vigente. Si ha recibido este mensaje por error,
le rogamos nos lo notifique inmediatamente por la misma vía
y proceda a su eliminación.
---

[SOLTECSIS] Carles Xavier Munyoz Baldó

2015-05-05 17:18:02 UTC

Permalink

More info about the problem ...
It seems that the virtual machines lost access to its disks, because
there are no logs in the /var/log/syslog file since the problem until de
hard reboot.

The two virtual machines use qcow2 virtual disks.
¿Should I do something in the virtual disks after the upgrade?

Post by [SOLTECSIS] Carles Xavier Munyoz BaldÃ³
Hello,
Yes, I rebooted the entire host server because in the updates where
included a new kernel.
The problem arises after the upgrade and host server reboot. And it is
not inmediatly after the virtual machines boot, it has happen after
several hours of normal operation and the two virtual machines where not
affected at the same time.
Thank you for your help.

Post by Michael Rasmussen
On Tue, 05 May 2015 18:50:42 +0200

========================================
SOLTECSIS SOLUCIONES TECNOLOGICAS, S.L.
Carles Xavier Munyoz Baldó
Departamento de I+D+I
Tel./Fax: 966 446 046
www.soltecsis.com
========================================
---
La información contenida en este e-mail es confidencial,
siendo para uso exclusivo del destinatario arriba mencionado.
Le informamos que está totalmente prohibida cualquier
utilización, divulgación, distribución y/o reproducción de
esta comunicación sin autorización expresa en virtud de la
legislación vigente. Si ha recibido este mensaje por error,
le rogamos nos lo notifique inmediatamente por la misma vía
y proceda a su eliminación.
---
_______________________________________________
pve-user mailing list
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

--
Saludos.
========================================
SOLTECSIS SOLUCIONES TECNOLOGICAS, S.L.
Carles Xavier Munyoz Baldó
Departamento de I+D+I
Tel./Fax: 966 446 046
***@soltecsis.com
www.soltecsis.com
========================================

---
La información contenida en este e-mail es confidencial,
siendo para uso exclusivo del destinatario arriba mencionado.
Le informamos que está totalmente prohibida cualquier
utilización, divulgación, distribución y/o reproducción de
esta comunicación sin autorización expresa en virtud de la
legislación vigente. Si ha recibido este mensaje por error,
le rogamos nos lo notifique inmediatamente por la misma vía
y proceda a su eliminación.
---

[SOLTECSIS] Carles Xavier Munyoz Baldó

2015-05-07 08:55:16 UTC

Permalink

Hello,
It seems that converting the qcow2 virtual disk file to the last qcow2
format solves the problem.

In a virtual machine with the problem, before the conversión, the
qemu-img info comands shows this:
[...]
# qemu-img info disk.qcow2.OLD
image: disk.qcow2.OLD
file format: qcow2
virtual size: 32G (34359738368 bytes)
disk size: 8.3G
cluster_size: 65536
Format specific information:
compat: 0.10
[...]

After the virtual disk conversion using the command:
# qemu-img convert -O qcow2 disk.qcow2.OLD disk.qcow2
the info command shows:
[...]
# qemu-img info disk.qcow2
image: disk.qcow2
file format: qcow2
virtual size: 32G (34359738368 bytes)
disk size: 8.1G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
corrupt: false
[...]

We are waiting a prudential time in order to conclude that the problem
is solved, but since the conversion the virtual machines are stable.

The conversion moves the virtual disk qcow2 file from compat: 0.10 to
compat: 1.1. Is it possible that the software under Proxmox 3.4 is
incompatible with qcow2 files compat: 0.10?
As I said in my first e-mail, we were running the virtual machines
without problems until we upgrade Proxmox from 3.1 to 3.4.

Best regards.

Post by [SOLTECSIS] Carles Xavier Munyoz BaldÃ³
More info about the problem ...
It seems that the virtual machines lost access to its disks, because
there are no logs in the /var/log/syslog file since the problem until de
hard reboot.
The two virtual machines use qcow2 virtual disks.
¿Should I do something in the virtual disks after the upgrade?

Post by Michael Rasmussen
On Tue, 05 May 2015 18:50:42 +0200

[SOLTECSIS] Carles Xavier Munyoz Baldó

2015-05-14 08:29:19 UTC

Permalink

Good morning,
We are still having this problem and it is becoming a very serious problem.

We have several installations of Proxmox and we have seen that it is
happening in all installations with the last Proxmox version (3.4-3) and
with qcow2 disks.

We have seen that in all installations with Proxmox 3.4-3 and in virtual
machines with qcow2 disks the problem is happening. It happens very
randomly. When one VM has the problem, after rebooting it killing the
kvm process with kill -9, it may go fine several days but then fails again.

We have seen that in other installations with previos versions of
Proxmox all is going fine.

All of this make us think that there is a bug in the software included
in the last Proxmox version.
Anyone has had problems like this?

We are thinking about downgrade Proxmox version to 3.1, is it possible?
How can we do it?

Best regards.

Post by [SOLTECSIS] Carles Xavier Munyoz BaldÃ³
Hello,
It seems that converting the qcow2 virtual disk file to the last qcow2
format solves the problem.
In a virtual machine with the problem, before the conversión, the
[...]
# qemu-img info disk.qcow2.OLD
image: disk.qcow2.OLD
file format: qcow2
virtual size: 32G (34359738368 bytes)
disk size: 8.3G
cluster_size: 65536
compat: 0.10
[...]
# qemu-img convert -O qcow2 disk.qcow2.OLD disk.qcow2
[...]
# qemu-img info disk.qcow2
image: disk.qcow2
file format: qcow2
virtual size: 32G (34359738368 bytes)
disk size: 8.1G
cluster_size: 65536
compat: 1.1
lazy refcounts: false
corrupt: false
[...]
We are waiting a prudential time in order to conclude that the problem
is solved, but since the conversion the virtual machines are stable.
The conversion moves the virtual disk qcow2 file from compat: 0.10 to
compat: 1.1. Is it possible that the software under Proxmox 3.4 is
incompatible with qcow2 files compat: 0.10?
As I said in my first e-mail, we were running the virtual machines
without problems until we upgrade Proxmox from 3.1 to 3.4.
Best regards.

Post by Michael Rasmussen
On Tue, 05 May 2015 18:50:42 +0200

Gerry

2016-01-08 11:15:26 UTC

Permalink

Post by [SOLTECSIS] Carles Xavier Munyoz BaldÃ³
<..>
All of this make us think that there is a bug in the software included
in the last Proxmox version.
Anyone has had problems like this?

Digging up this thread, because we are experiencing the same issue.

At first, we had this problem with a few Windows machines. They were
showing the same problem, but we thought it could be due to the OS
acting up. It was solved after moving them away from a qcow2 disk to
dedicated storage.

Now, we have started to see the same issue with a Linux machine, so
it can not be linked to the OS.

Did you happen to find a solution for this problem in the end?

Thanks,

Gerry