[Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching

[Li, Xiaoyan]

Hi Brent and Ovidiu,

As this email has a long history, I re-summarize the raw cache in StarlingX and Cinder upstream image cache.
Please vote whether we can abandon raw cache in StarlingX.

StarlingX
Create an image cache in ceph when Glance creates an image.
And delete the cached image in ceph when deleting the original image in Glance.

Cinder:
When creating a volume from an image in a backend storage at the first time, Cinder creates a volume from this image, and uses it as the image cache.
So next time if users create another volume from this image in the same backend storage, Cinder at first finds out the cached image volume and clones a new volume from it.
Cinder allows capacity configuration for cached images. If the space is used up, Cinder will evict the cached image volumes.

>From my viewpoint,
Cinder image cache can achieve same functionality as Raw cache in StarlingX with more enhancement.  It is for all Cinder supported backend storage, not just for Ceph.

Best wishes
Lisa

From: Li, Xiaoyan
Sent: Monday, November 19, 2018 9:44 AM
To: Poncea, Ovidiu <Ovidiu.Poncea at windriver.com>; Rowsell, Brent <Brent.Rowsell at windriver.com>; 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
Subject: RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching

Hi Ovidiu,

A cached image ( new volume from this image)  is created on a storage backend when Cinder firstly creates a volume in the same backend storage from the image.
All the information are stored in Cinder, including volume id, image id etc. https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/manager.py#L1368
https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L82


A cached image is deleted when the configure space for cache is used up.  So currently Cinder doesn't delete the cached image volumes even if the image is deleted. But this can be an enhancement of current cinder image cache.
https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L117
https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/manager.py#L1351

Best wishes
Lisa

From: Poncea, Ovidiu [mailto:Ovidiu.Poncea at windriver.com]
Sent: Friday, November 16, 2018 4:57 PM
To: Li, Xiaoyan <xiaoyan.li at intel.com<mailto:xiaoyan.li at intel.com>>; Rowsell, Brent <Brent.Rowsell at windriver.com<mailto:Brent.Rowsell at windriver.com>>; 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>
Subject: RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching

Hi Li,

Quick question: Is cache going to be freed when an image is deleted from glance? It would be a waste to cache images that are no longer needed.

Thanks,
Ovidiu
________________________________
From: Li, Xiaoyan [xiaoyan.li at intel.com]
Sent: Tuesday, November 13, 2018 9:19 AM
To: Rowsell, Brent; 'starlingx-discuss at lists.starlingx.io'
Subject: Re: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching
Hi,

About the raw cache function in StarlingX Cinder and Glance, I would like to remove it as Cinder has similar function. Please see following detail.
And if I  would like to remove the function in StarlingX, there are two methods:

1.       Submit a patch to revert the changes in Glance and Cinder.

2.       Ignore these patches during upgrading StarlingX/Cinder to new Cinder release.
Which way do we prefer to?

Best wishes
Lisa

From: Li, Xiaoyan
Sent: Thursday, September 20, 2018 10:17 AM
To: Rowsell, Brent <Brent.Rowsell at windriver.com<mailto:Brent.Rowsell at windriver.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching


Hi, Brent



The following are mechanism of Cinder volume cache.



Creation of cached volume:

It creates a cached volume in the backend storage when creating from an image.

1.       Create_from_image:

https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L890

2.       Return image cache entry: If not existed, it creates a new entry.

https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L746

3.       Create a new image-volume and cache entry for it:

https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L872



Use a cached volume when creating a volume:

https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L723-L735



 Delete the cache volume: When capacity and number of cache entries exceed specified limit, it deletes cache entries (cached volumes).

https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L164

Best wishes
Lisa

From: Rowsell, Brent [mailto:Brent.Rowsell at windriver.com]
Sent: Thursday, September 6, 2018 10:02 AM
To: Li, Xiaoyan <xiaoyan.li at intel.com<mailto:xiaoyan.li at intel.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching

We would need to review this feature to ensure it provides equivalent functionality first.
If it does, great, we can look at reverting and enabling this cinder functionality.

Brent

From: Li, Xiaoyan [mailto:xiaoyan.li at intel.com]
Sent: Wednesday, September 5, 2018 9:59 PM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching

Hi all,

This email is about Raw caching function in StarlingX. This feature is to cache an image in backend storage like Ceph when we first create a volume in this backend storage.

In fact, Cinder upstream has already had a similar function in Pike release. https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html
So I want to revert Raw caching function in StarlingX, and use Cinder generic image cache instead.
The problem is that we need to update Cinder config in StarlingX. Any comments?

Best wishes
Lisa



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20181122/826a11e5/attachment-0001.html>