<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.emailquote, li.emailquote, div.emailquote
{mso-style-name:emailquote;
mso-style-priority:99;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:1.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.msochpdefault, li.msochpdefault, div.msochpdefault
{mso-style-name:msochpdefault;
mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Times New Roman","serif";}
span.emailstyle20
{mso-style-name:emailstyle20;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.emailstyle21
{mso-style-name:emailstyle21;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.emailstyle22
{mso-style-name:emailstyle22;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.emailstyle24
{mso-style-name:emailstyle24;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.emailstyle25
{mso-style-name:emailstyle25;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle26
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle27
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:381708923;
mso-list-type:hybrid;
mso-list-template-ids:-2071167048 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1
{mso-list-id:478156291;
mso-list-type:hybrid;
mso-list-template-ids:-2144317316 -1129923720 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2
{mso-list-id:1009480943;
mso-list-type:hybrid;
mso-list-template-ids:-1338602172 67698709 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l2:level1
{mso-level-number-format:alpha-upper;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l3
{mso-list-id:1327056737;
mso-list-type:hybrid;
mso-list-template-ids:451214914 -1129923720 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l3:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l3:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l3:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l4
{mso-list-id:1336230813;
mso-list-type:hybrid;
mso-list-template-ids:1261102404 67698689 67698703 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l4:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l4:level2
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l4:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l4:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l4:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l4:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l4:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l4:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l5
{mso-list-id:1344674476;
mso-list-type:hybrid;
mso-list-template-ids:-502336994 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l5:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l5:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l5:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l5:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l5:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l5:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l5:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l5:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l5:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Li,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks for providing clarifications! So, for our use cases, main problem is that glance’s raw caching is more controllable that cinder’s. If it’s not enough
we need to improve it, if we can live with it then at a minimum it needs to be enabled though sysinv configuration and then remove the raw-caching from glance.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">See inline comments plus bellow summary and proposal, we need Brent’s input on this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I see two main solutions to the problem:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo6"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">A.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Always enable cache, for any backend, but only cache glance images that have a certain attribute – this needs a cinder upstream change. Cache limit
has to be removed (another cinder upstream change). We may also need a way to kick-start the caching in cinder & clean up cache (periodically and/or user triggered should be enough).<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo6"><![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">B.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Make enabling cache storage backend specific and configurable (through sysinv). Once cinder’s cache is enabled for a backend, cache everything. Size
of the cache should be configurable.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I would go for B. as it, most likely, doesn’t need upstream changes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Summary of problems, TBD if we can live with them:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l4 level1 lfo4"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Images are not cached on creation – if we can’t live with it we may need a trigger to cinder on image creation or a way to manually kick-start the
caching process.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l4 level1 lfo4"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Since first volume creation is slow for larger volumes this
<i>may</i> timeout (keystone token expiration) – we had a customer using 200GB qcow2 windows images that would timeout on conversion. I don’t see a workaround for it, just ask him to manually do the conversion when importing very large images to glance.
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l4 level1 lfo4"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">we can’t provide a 100% guarantee that, once converted, successive creation won’t need to get converted again due to cache exhaustion. Can we live
with it? Users may intermittently see slowdowns and wonder what’s going on.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l4 level1 lfo4"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">cache will waste space, if original images no longer exists there is no automated way to remove them from the cache – admin can clean up the cache
manually if he so desires. We can either:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:1.0in;text-indent:-.25in;mso-list:l4 level2 lfo4">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Live with it – assume that the space allocated to the cache is for the cache only or users can clean up cache by themselves.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:1.0in;text-indent:-.25in;mso-list:l4 level2 lfo4">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Clean up cache through a cron job (although this is a cache, some caches are supposed to clean themselves up if cached data is no longer present).<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:1.0in;text-indent:-.25in;mso-list:l4 level2 lfo4">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Implement another mechanism to clean the cache
<i>when</i> an image is deleted not at a later time (this is way too complex to upstream).<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l4 level1 lfo4"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">What happens with images that users don’t want to cache? Should we add a filter (glance property)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I vote for #2 as it does not seems too hard to implement. A once a day cron task can free up wasted space.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Summary of TODOs (assuming B. is chosen) before removing raw-caching (open for discussions & dependent on resolution to above issues):<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo5"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Enable caching per backend through sysinv system storage-backend-add/modify commands though a capabilities field (this seems the simplest solution)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo5"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Add sysinv configuration option per storage backend to set cache size. [Clean up images in cache when size is decreased]<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo5"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">When first enabling: create shadow tenant (no need to remove it when disabling cache)<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo5"><![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Support disabling cache for a backend (clean up residual images)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ovidiu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Li, Xiaoyan [mailto:xiaoyan.li@intel.com]
<br>
<b>Sent:</b> Tuesday, November 27, 2018 4:30 AM<br>
<b>To:</b> Poncea, Ovidiu; Rowsell, Brent; 'starlingx-discuss@lists.starlingx.io'<br>
<b>Cc:</b> Miller, Frank<br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Ovidiu,
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As far as I’m concerned, Cinder image cache is an cache mechanism. So overall, users don’t need to clean it manually.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Currently when capacity for cache is full, it removes the cached image volumes with LRU policy. More detailed please see the following comments.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Best wishes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Lisa<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Poncea, Ovidiu [mailto:Ovidiu.Poncea@windriver.com]
<br>
<b>Sent:</b> Monday, November 26, 2018 11:15 PM<br>
<b>To:</b> Li, Xiaoyan <xiaoyan.li@intel.com>; Rowsell, Brent <Brent.Rowsell@windriver.com>; 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io><br>
<b>Cc:</b> Miller, Frank <Frank.Miller@windriver.com><br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">Hi Lisa,<br>
<br>
Yeah, even if we refactor raw caching, it's most likely going to be rejected by upstream due to replicating existing functionality in cinder. Yet, imho, we should have an working replacement before retiring raw caching and we should have some agreed mitigations
in place for cinder's disadvantages (if we can't live with them, Brent please help here). See my questions bellow & inline. Also, please correct text bellow if I made wrong assumptions as you know cinder's caching better than me.<br>
<br>
Short comparison of the two:<br>
<br>
<b>Raw caching</b><br>
<br>
Uses --raw-cache cli option in Glance to trigger a background process that converts the image. Once cached, new volumes get created on Ceph instantly by levereging Ceph's copy-on-write. Cache is allocated from the "images" RBD pool.<br>
<br>
<i>Advantages:</i><br>
- user can select the images it wants to cache<br>
- user can monitor the progress and can check used space for each image (cli + dashboard).<br>
- on image delete the cache is also cleared if there is no volume using it. Else it is cleared with the last volume keeping the cache data in-use.<br>
- no wasted space<br>
- complete control by user<br>
<br>
<i>Disadvantages:</i><br>
- There is almost no way this is going to be accepted upstream. Maybe, yet with small hopes, if we refactor everything as a 3rd party glance feature, but we may need to push some hooks upstream to make it work.<br>
- Ceph only<br>
<br>
<b>Cinder's caching</b><br>
<br>
Uses a "shadow" tenant to store shadow volumes. Cache is created with the first volume from that images. Next volume will be created instantly by leveraging copy-on-write if backend provides support for it (e.g. on Ceph). Space for cache is allocated on one
of the cinder backends, has a configurable threshold.<br>
<br>
<i>Advantages:</i><br>
- already upstream<br>
- works with all backends<br>
- all cached images are displayed for the "admin" if he changes to the shadow tenant and lists volumes.<br>
- admin (not user, only admin) can free cache by deleting volumes of the shadow tenant (need confirmation)<br>
<i><br>
Disadvantages:</i><br>
1. it's either globally enabled or disabled => needs sysinv configuration option<br>
2. it caches every image. No way to select what image to cache nor with what backend (question bellow) => space waste<br>
3. cached images are not removed. It needs to hit a space provision to do that, and it will remove the oldest image, although that image cache may be important.<br>
4. less control: Images are cached on first use and are removed when provisioned space hits threshold. This means that user does not have control over what images are converted and what images are in cache. So, sometimes volume creation works fast, other times
it's slow. This can be a problem especially on parallel volume creation through helm charts as, if the image did not have a cache, then stack creation may timeout. Another problem may be if cache is small and images get rotated in the cache => we need alarms
when threshold is hit.<br>
5. needs the shadow tenant created before use => puppet / helm chart chart update (for --kubernates)<br>
<br>
<b><i>Mitigations of disadvantages above - possible solutions and alternatives:</i></b><br>
#1: Customers may not want to enable it, we should allow customers to choose when to enable it (it can be added as a custom capabilities parameter to "system storage-backend-add/system storage-backend-modify")</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] Currently image cache can be enabled/disabled per backend storage. https://docs.openstack.org/cinder/latest/admin/blockstorage-image-volume-cache.html<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I think it is enough.
<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Ovi] Nice, we need a configuration option per backend in sysinv to enable it. (most likely in the capabilities fields of storage-backends table. See
‘system storage-backend-*’ commands).<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
#2: No workaround comes to my mind - we can probably live with it</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
#3: A simple solution would be to implement a cron job to clean cache periodically or a more elaborate solution would be remove the cache with the last volume that used that image (need a cinder upstream feature for it)</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] From doc, currently Cinder removes cached images from least used to recently used. Every time Cinder uses a cached image volume, it updates
its last_used field. This is the normal policy for data eviction. <o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As it is cache and should be transparent for users, why do we need users to evict data?<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Ovi] If we conclude than this is enough from data usage perspectives then we are ok with it.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
#4: Two options comes to mind:<br>
1. To get some control we should not limit the cache size, given that we do propper cleanup in #3.</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] Even we do cleanup, the limit can’t be removed.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#00B050">[Ovi] We may need to enhance this.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
2. If we limit the cache, we have to make the limit configurable and raise an alarm once cache gets near full so that admin takes preventive measures and either increases provisioned space or</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
#5: This is mandatory, otherwise cinder's caching won't work at all.</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] It has to set cinder_internal_tenant_project_id and cinder_internal_tenant_user_id before enabling cache images. As this user can manage
these cached image volumes. Why can’t it work with Kubernetes?<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#00B050">[Ovi] I did not say it won’t work with kubernates
</span></i></b><b><i><span style="font-size:11.0pt;font-family:Wingdings;color:#00B050">J</span></i></b><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#00B050"> What I said is that we need to provision the shadow tenant automatically
when the feature is enabled.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
<br>
<b><i>Questions, (maybe if you get time to play with cinder's caching to get a better understanding):</i></b><br>
1. How does cinder's caching behaves when multiple volumes are created in parallel from a newly created image? Will it wait for the cache to be created before creating the volumes or just start all volume creations in parallel?</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] Inside a volume service it is sequential to run volume creation tasks. But as we have HA. For image cache, it creates an entry in cinder
db at first and then creates volumes. The primary key is not image_id+backend_storage.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">It is possible that several entries or volumes will be created in same backend storage.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#00B050">[Ovi] So, only the first volume creation is going to be slow? If that’s the case then parallel volume creation will work ok as only first volume creation
will be slow.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><br>
2. What is the cinder backend that store the cache? If it is the one used by the volume, will this lead to multiple cached volumes of the same image? Can we chose the backend?<o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] We can set whether enabled cache per backend. If users create a volume in backend ceph from an image, an cached image volume will be created
in Ceph if it is enabled. Next time if users create a volume in IBM storage from the same image, it will create another image cached volume in IBM storage if it is enabled.
<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#00B050">[Ovi] Then we need to enable it and configure cache size per backend, I guess.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">3. How is cache space provisioned? Do we need to restart cinder-volume for changes to take effect?<o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] These config needs to be done in config file. So it needs to restart cinder volume services once config are changed.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Ovi] So after we make the changes, we re-apply the manifests and restart the services (reload the helm charts for k8s deployments)<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">4. Is admin able to clean up individual cached images in the shadow tenant? Maybe also user?</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[Li, Xiaoyan] Admin and shadow tenants can both do cleanup.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">Ovidiu<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="color:black">
<hr size="2" width="100%" align="center">
</span></div>
<div id="divRpF227160">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"> Li, Xiaoyan [xiaoyan.li@intel.com]<br>
<b>Sent:</b> Thursday, November 22, 2018 2:41 AM<br>
<b>To:</b> Poncea, Ovidiu; Rowsell, Brent; 'starlingx-discuss@lists.starlingx.io'<br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Brent and Ovidiu,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As this email has a long history, I re-summarize the raw cache in StarlingX and Cinder upstream image cache.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Please vote whether we can abandon raw cache in StarlingX.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">StarlingX</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Create an image cache in ceph when Glance creates an image.
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">And delete the cached image in ceph when deleting the original image in Glance.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Cinder:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">When creating a volume from an image in a backend storage at the first time, Cinder creates a volume from this image, and uses it as the image cache.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">So next time if users create another volume from this image in the same backend storage, Cinder at first finds out the cached image volume and clones a new
volume from it.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Cinder allows capacity configuration for cached images. If the space is used up, Cinder will evict the cached image volumes.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">From my viewpoint,
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Cinder image cache can achieve same functionality as Raw cache in StarlingX with more enhancement. It is for all Cinder supported backend storage, not just
for Ceph.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Best wishes</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Lisa</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> Li, Xiaoyan
<br>
<b>Sent:</b> Monday, November 19, 2018 9:44 AM<br>
<b>To:</b> Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com">Ovidiu.Poncea@windriver.com</a>>; Rowsell, Brent <<a href="mailto:Brent.Rowsell@windriver.com">Brent.Rowsell@windriver.com</a>>; 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a>><br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Ovidiu,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">A cached image ( new volume from this image) is created on a storage backend when Cinder firstly creates a volume in the same backend storage from the image.
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">All the information are stored in Cinder, including volume id, image id etc.
<a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/manager.py#L1368" target="_blank">
https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/manager.py#L1368</a>
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L82" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L82</a>
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">A cached image is deleted when the configure space for cache is used up. So currently Cinder doesn’t delete the cached image volumes even if the image is deleted.
But this can be an enhancement of current cinder image cache.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L117" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L117</a>
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/manager.py#L1351" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/manager.py#L1351</a>
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Best wishes</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Lisa</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> Poncea, Ovidiu [<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">mailto:Ovidiu.Poncea@windriver.com</a>]
<br>
<b>Sent:</b> Friday, November 16, 2018 4:57 PM<br>
<b>To:</b> Li, Xiaoyan <<a href="mailto:xiaoyan.li@intel.com" target="_blank">xiaoyan.li@intel.com</a>>; Rowsell, Brent <<a href="mailto:Brent.Rowsell@windriver.com" target="_blank">Brent.Rowsell@windriver.com</a>>; 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>><br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">Hi Li,</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">Quick question: Is cache going to be freed when an image is deleted from glance? It would be a waste to cache images that are no longer needed.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">Thanks,</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">Ovidiu</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="color:black">
<hr size="2" width="100%" align="center">
</span></div>
<div id="divRpF502789">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"> Li, Xiaoyan [xiaoyan.li@intel.com]<br>
<b>Sent:</b> Tuesday, November 13, 2018 9:19 AM<br>
<b>To:</b> Rowsell, Brent; 'starlingx-discuss@lists.starlingx.io'<br>
<b>Subject:</b> Re: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">About the raw cache function in StarlingX Cinder and Glance, I would like to remove it as Cinder has similar function. Please see following detail.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">And if I would like to remove the function in StarlingX, there are two methods:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">1.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Submit a patch to revert the changes in Glance and Cinder.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ignore these patches during upgrading StarlingX/Cinder to new Cinder release.
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Which way do we prefer to?</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Best wishes</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Lisa</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> Li, Xiaoyan
<br>
<b>Sent:</b> Thursday, September 20, 2018 10:17 AM<br>
<b>To:</b> Rowsell, Brent <<a href="mailto:Brent.Rowsell@windriver.com" target="_blank">Brent.Rowsell@windriver.com</a>>;
<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Hi, Brent</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">The following are mechanism of Cinder volume cache.</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Creation of cached volume:
</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">It creates a cached volume in the backend storage when creating from an image.</span><span style="color:black"><o:p></o:p></span></p>
<p style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">1.</span><span style="font-size:7.0pt;color:black">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Create_from_image:
</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L890" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L890</a></span><span style="color:black"><o:p></o:p></span></p>
<p style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">2.</span><span style="font-size:7.0pt;color:black">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Return image cache entry: If not existed, it creates a new entry.</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L746" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L746</a></span><span style="color:black"><o:p></o:p></span></p>
<p style="margin-left:.5in;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">3.</span><span style="font-size:7.0pt;color:black">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Create a new image-volume and cache entry for it:</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L872" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L872</a></span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Use a cached volume when creating a volume:</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L723-L735" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/volume/flows/manager/create_volume.py#L723-L735</a>
</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> Delete the cache volume: When capacity and number of cache entries exceed specified limit, it deletes cache entries (cached volumes).
</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"><a href="https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L164" target="_blank">https://github.com/openstack/cinder/blob/stable/pike/cinder/image/cache.py#L164</a>
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Best wishes</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Lisa</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> Rowsell, Brent [<a href="mailto:Brent.Rowsell@windriver.com" target="_blank">mailto:Brent.Rowsell@windriver.com</a>]
<br>
<b>Sent:</b> Thursday, September 6, 2018 10:02 AM<br>
<b>To:</b> Li, Xiaoyan <<a href="mailto:xiaoyan.li@intel.com" target="_blank">xiaoyan.li@intel.com</a>>;
<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> RE: [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We would need to review this feature to ensure it provides equivalent functionality first.
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If it does, great, we can look at reverting and enabling this cinder functionality.
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Brent
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> Li, Xiaoyan [<a href="mailto:xiaoyan.li@intel.com" target="_blank">mailto:xiaoyan.li@intel.com</a>]
<br>
<b>Sent:</b> Wednesday, September 5, 2018 9:59 PM<br>
<b>To:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">
starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> [Starlingx-discuss] [StarlingX] Use Cinder generic image cache to replace raw caching</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Hi all,</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">This email is about Raw caching function in StarlingX. This feature is to cache an image in backend storage like Ceph when we first create a volume in this backend
storage.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">In fact, Cinder upstream has already had a similar function in Pike release.
<a href="https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html" target="_blank">
<span style="color:#0563C1">https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html</span></a>
</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">So I want to revert Raw caching function in StarlingX, and use Cinder generic image cache instead.
</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">The problem is that we need to update Cinder config in StarlingX. Any comments?
</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Best wishes</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Lisa</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>