[Starlingx-discuss] [cinder] Analysis of patch c45bb0e for StartlingX upstreaming

[Mihai, Irina]

Hey Liang,

When we saw the IO error, we had created 20 boot from volume instances at once with the following flavor: 8gb rootfs, 8gb ephemeral and 512 meg swap, using a qcow2 Centos7 image (quite large, not sure what you've been using in your tests).

Irina

________________________________
From: Fang, Liang A [liang.a.fang at intel.com]
Sent: Tuesday, November 27, 2018 5:36 AM
To: Mihai, Irina
Cc: 'starlingx-discuss at lists.starlingx.io'
Subject: RE: [Starlingx-discuss] [cinder] Analysis of patch c45bb0e for StartlingX upstreaming

Hi Irina

Thanks for the info. Some status:

-          I can reproduce the issue that solved in _convert_image(). The issue I reproduced may be not the same one as the original issue this patch to fix. Reproduced by: uploading a volume to image, but with insufficient space in image_conversion_dir.

-          But I still cannot reproduce the issue solved in _create_from_image_cache_or_download() even with multiple volume creating requests. Take two requests as example, the second request will be rejected by image_utils.check_available_space() at the beginning of _create_from_image_cache_or_download(). If I add debug code - sleep(60) after the line of image_utils.check_available_space, so that both two requests can pass the check, but the second request will raise exception(ImageTooBig) and be caught in the line 777, 810: “image_utils.TemporaryImages.fetch”. And I take a look of other code following, don’t find any chance of raising “IOError”.

o   verify_glance_image_signature

o   volume_glance_metadata_bulk_create

o   qemu_img_info

o   check_virtual_size

o   _create_from_image_download: This will re-use image file. Even insufficient space happen, the exception be re-raised is exception.InsufficientConversionSpace, not “IOError”.

If you can get the original reproduce steps, that would be great!

Some info related:
Change-Id: Ie5eba32f92eb1740554bfc44baf612e326d6fde5, submitted in this year, with 4 commits, shows dest will be a volume. So unlikely be the same as image_conversion_dir.

Thanks,
Liang
From: Mihai, Irina [mailto:Irina.Mihai at windriver.com]
Sent: Tuesday, November 27, 2018 1:25 AM
To: Fang, Liang A <liang.a.fang at intel.com>
Cc: 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
Subject: RE: [Starlingx-discuss] [cinder] Analysis of patch c45bb0e for StartlingX upstreaming

Hi Liang,

This patch was covering the scenario when we're trying to create multiple volumes from image at the same time.
In this situation, multiple downloads/convertions would be allowed to start since there would still be conversion space left, but eventually the operation would fail because of the disk being full. This is not caught by any of the "ImageTooBig" or "ImageUnacceptable" exceptions.

Could you please help to address one doubt: why image is converted in the same location with image itself? This patch is to create a volume from image, right?
Right, but I'm not aware of the reasoning behind this. However, if the used image is qcow2 then it needs to be converted to raw before being saved to the volume.

Please let me know if I can help in any other way.

Thanks,
Irina

________________________________
From: Fang, Liang A [liang.a.fang at intel.com]
Sent: Sunday, November 25, 2018 11:44 AM
To: Mihai, Irina
Cc: 'starlingx-discuss at lists.starlingx.io'
Subject: [Starlingx-discuss] [cinder] Analysis of patch c45bb0e for StartlingX upstreaming
Hi Irina

Code review: https://review.openstack.org/#/c/618976/

I’m working of upstreaming this patch. Could you please help to address one doubt: why image is converted in the same location with image itself? This patch is to create a volume from image, right?

I take a look of related code, the destination location would be the newly created volume and the source location is configured in cinder.conf, something like:
image_conversion_dir = /mnt/d100m/image_conversion_dir

I did an experiment, in the latest master branch of cinder upstream repo:

-          If image_conversion_dir has no space to store the downloading image, the exception ImageTooBig will be caught in the beginning of _create_from_image_cache_or_download().

-          If has space to store the image, but dest volume size is smaller than image’s virtual size, an exception “ImageUnacceptable” will raised by check_virtual_size.
So it seems no chance to go to the situation this patch aimed to solve.

Could you please help to give more background of this patch? Or if you think this patch is invalid now, please just tell me, I will mark this patch as abandoned.

Thanks
Liang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20181127/d66bbb2c/attachment.html>