Below is current test config.

[libvirt]

connection_uri = qemu+tcp://127.0.0.1/system

cpu_mode = host-model

disk_cachemodes = network=writeback

hw_disk_discard = unmap

images_rbd_ceph_conf = /etc/ceph/ceph.conf

images_rbd_pool = vms

images_type = default

live_migration_completion_timeout = 180

live_migration_inbound_addr = 192.168.206.150

live_migration_permit_auto_converge = true

mem_stats_period_seconds = 0

rbd_secret_uuid = 457eb676-33da-42ec-9a8c-9293d545c337

rbd_user = cinder

remove_unused_resized_minimum_age_seconds = 86400

virt_type = qemu

I also did below convert command manually, it seems OK.

controller-0:~$ qemu-img convert -t none -O raw -f qcow2 cirros-0.4.0-x86_64-disk.img test1.img

controller-0:~$ qemu-img convert -t writeback -O raw -f qcow2 cirros-0.4.0-x86_64-disk.img test2.img

controller-0:~$ qemu-img info test1.img

image: test1.img

file format: raw

virtual size: 44M (46137344 bytes)

disk size: 13M

controller-0:~$ qemu-img info test2.img

image: test2.img

file format: raw

virtual size: 44M (46137344 bytes)

disk size: 13M

Need further check why in the code after convert, the image format is still qcow2.

Zhipeng

From: Liu, ZhipengS <zhipengs.liu@intel.com>
Sent: 2019年10月31日 11:12
To: 朱博祥 <zhu.boxiang@99cloud.net>; Xie, Cindy <cindy.xie@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io>
Cc: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>; Hu, Yong <yong.hu@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Subject: Re: [Starlingx-discuss] About EB verification for openstack train upgrade

Hi Boxiang,

Yes, my below patch can be abandoned, just need to remove “force_raw_images: false” in

stx/openstack-armada-app/stx-openstack-helm/stx-openstack-helm/manifests/manifest.yamll file since the default value is true currently.

https://review.opendev.org/#/c/691802/

I will modify my patch.

However, it is not related to current issue since this flag is set to true in the test.

The issue is just as error log mentioned below.

“ Image 70f74018-24f6-4d7d-8f99-2194a4449cfd is unacceptable: Converted to raw, but format is now qcow2 ”

We need check this point further.

Thanks!

Zhipeng

From: 朱博祥 <zhu.boxiang@99cloud.net>
Sent: 2019年10月31日 10:41
To: Liu, ZhipengS <zhipengs.liu@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io>
Cc: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>; Hu, Yong <yong.hu@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Subject: Re:RE: [Starlingx-discuss] About EB verification for openstack train upgrade

Hi Zhipeng,

I have succeeded to create the vm in openstack upstream(master, code is two weeks ago) now. I deployed the environment with kolla-ansible(ceph as the backend storage).

In my nova.conf file, the section of libvirt is as followed:

[libvirt]

connection_uri = qemu+tcp://172.16.140.14/system

images_type = rbd

images_rbd_pool = vms

images_rbd_ceph_conf = /etc/ceph/ceph.conf

rbd_user = nova

disk_cachemodes = network=writeback

hw_disk_discard = unmap

rbd_secret_uuid = 5c79d567-e142-4ab4-87f6-0fdf53060ab9

virt_type = kvm

The default value of 'force_raw_images' in nova is True, so I did not set it as True again in my nova.conf file.

I use the min image cirros with qcow2 format and created the vm without volume which is created by cinder.

It's my image's info:

(nova-compute)[root@zbx-allinone /]# qemu-img info cirros-0.4.0-x86_64-disk.img

image: cirros-0.4.0-x86_64-disk.img

file format: qcow2

virtual size: 44M (46137344 bytes)

disk size: 12M

cluster_size: 65536

Format specific information:

    compat: 1.1

    lazy refcounts: false

    refcount bits: 16

    corrupt: false

So as you said, command qemu-img failed to convert the image from qcow2 to raw. BTW, is it a phenomenon that must recur now in STX?

Are there other error logs in nova-compute or other nova-* services?

--

Thanks,

Boxiang

发件人："Liu, ZhipengS" <zhipengs.liu@intel.com>
发送日期：2019-10-31 10:14:44
收件人："Liu, ZhipengS" <zhipengs.liu@intel.com>,"Xie, Cindy" <cindy.xie@intel.com>,"Lemus Contreras, Cristopher J" <cristopher.j.lemus.contreras@intel.com>,"'朱博祥'" <zhu.boxiang@99cloud.net>,"'starlingx-discuss@lists.starlingx.io'" <starlingx-discuss@lists.starlingx.io>
抄送人："Alonso, Juan Carlos" <juan.carlos.alonso@intel.com>,"Hu, Yong" <yong.hu@intel.com>,"Cabrales, Ada" <ada.cabrales@intel.com>,"Jones, Bruce E" <bruce.e.jones@intel.com>,"Martinez Monroy, Elio" <elio.martinez.monroy@intel.com>
主题：RE: [Starlingx-discuss] About EB verification for openstack train upgrade

Hi Boxiang and all,

In fetch_to_raw(),

After force convert image from qcow2 to raw,  the format is still not raw,  then throw below error.

It seems qemu-image convert image failed.

Zhipeng

From: Liu, ZhipengS <zhipengs.liu@intel.com>
Sent: 2019年10月31日 9:35
To: Xie, Cindy <cindy.xie@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; '朱博祥' <zhu.boxiang@99cloud.net>; 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io>
Cc: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>; Hu, Yong <yong.hu@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Subject: Re: [Starlingx-discuss] About EB verification for openstack train upgrade

From: Liu, ZhipengS
Sent: 2019年10月30日 23:24
To: Xie, Cindy <cindy.xie@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; 朱博祥 <zhu.boxiang@99cloud.net>; starlingx-discuss@lists.starlingx.io
Cc: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>; Hu, Yong <yong.hu@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Subject: RE: [Starlingx-discuss] About EB verification for openstack train upgrade

Hi Boxiang,

It should be an issue in nova train.

Error log is below:

build of instance 363dddce-7af6-4701-8cea-52630b91ebcf aborted: Image 70f74018-24f6-4d7d-8f99-2194a4449cfd is unacceptable: Converted to raw, but format is now qcow2'

You tried to fix this issue before in nova LP:

https://bugs.launchpad.net/nova/+bug/1816686

In your patch: https://review.opendev.org/#/c/640271 (This patch is already in train release now)

You added below commit message in the patch.

“When we import image into rbd, check the format of cache

images. If the format is not raw, remove it first and

fetch it again. It will be raw format now.”

However, it seems not work!  Below code will throw error when image format is not raw and block the VM creation.

In nova/virt/images.py

def fetch_to_raw()  è

if data.file_format != "raw":

                      raise exception.ImageUnacceptable(image_id=image_href,

                        reason=_("Converted to raw, but format is now %s")

Not sure if your patch fixed this issue or is there any regression introduced in nova code later.

Any comment from you? 

I will check nova code further tomorrow and sync with you.

We may need a patch for nova☹

Thanks!

Zhipeng

From: Xie, Cindy <cindy.xie@intel.com>
Sent: 2019年10月30日 22:49
To: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; 朱博祥 <zhu.boxiang@99cloud.net>; starlingx-discuss@lists.starlingx.io
Cc: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>; Hu, Yong <yong.hu@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Subject: RE: [Starlingx-discuss] About EB verification for openstack train upgrade

+ mailing list in case anybody can help on debug the issue.

From: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Sent: Wednesday, October 30, 2019 7:45 PM
To: Liu, ZhipengS <zhipengs.liu@intel.com>; 朱博祥 <zhu.boxiang@99cloud.net>
Cc: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>; Hu, Yong <yong.hu@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Subject: Re: [Starlingx-discuss] About EB verification for openstack train upgrade

Hi Zhipeng,

The results are OK for Simplex, Duplex and Standard Local Storage, all Test Cases PASSED.

For Standard External Storage (2+2+2) there is a high quantity of Test Cases failing because we cannot launch instances. Here’s a sample output: http://paste.openstack.org/show/785641/ with the error that matches what is described on the launchpad.

Sanity summary for this specific configuration: http://paste.openstack.org/show/785655/

Please, let us know if you need additional details.

Thanks & Regards,

Cristopher Lemus