<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal00, li.msonormal00, div.msonormal00
{mso-style-name:msonormal0;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal000, li.msonormal000, div.msonormal000
{mso-style-name:msonormal00;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0000, li.msonormal0000, div.msonormal0000
{mso-style-name:msonormal000;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msochpdefault, li.msochpdefault, div.msochpdefault
{mso-style-name:msochpdefault;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
span.emailstyle19
{mso-style-name:emailstyle19;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle20
{mso-style-name:emailstyle20;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle21
{mso-style-name:emailstyle21;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle22
{mso-style-name:emailstyle22;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle23
{mso-style-name:emailstyle23;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle25
{mso-style-name:emailstyle25;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle27
{mso-style-name:emailstyle27;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle29
{mso-style-name:emailstyle29;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.emailstyle31
{mso-style-name:emailstyle31;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle33
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi Ovidiu & Bob<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Another question. I found starlingx save ceph cluster’s client.admin key in kube secret “ceph-pool-kube-rbd”. I wonder when starlingx create this secret, read ceph cluster keyring and save to secret. In bootstrap?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Martin, Chen<o:p></o:p></p>
<p class="MsoNormal">IOTG, Software Engineer<o:p></o:p></p>
<p class="MsoNormal">021-61164330<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Poncea, Ovidiu <Ovidiu.Poncea@windriver.com> <br>
<b>Sent:</b> Monday, December 30, 2019 11:40 PM<br>
<b>To:</b> Chen, Haochuan Z <haochuan.z.chen@intel.com>; Church, Robert <Robert.Church@windriver.com><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io><br>
<b>Subject:</b> RE: question about ceph or storage<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">I don't know if it's out of date :) it may just need to be retested if it wasn't for current release.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black"><o:p> </o:p></span></p>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black">
<hr size="3" width="100%" align="center">
</span></div>
<div id="divRpF399249">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black"> Chen, Haochuan Z [haochuan.z.chen@intel.com]<br>
<b>Sent:</b> Monday, December 30, 2019 3:51 PM<br>
<b>To:</b> Poncea, Ovidiu; Church, Robert<br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io'<br>
<b>Subject:</b> RE: question about ceph or storage</span><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">Thanks Ovidiu! If “-c” is out of date, what about remove it later.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com">Ovidiu.Poncea@windriver.com</a>>
<br>
<b>Sent:</b> Monday, December 30, 2019 4:43 PM<br>
<b>To:</b> Chen, Haochuan Z <<a href="mailto:haochuan.z.chen@intel.com">haochuan.z.chen@intel.com</a>>; Church, Robert <<a href="mailto:Robert.Church@windriver.com">Robert.Church@windriver.com</a>><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a>><br>
<b>Subject:</b> RE: question about ceph or storage<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">Hi Chen,</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">It was used in the previous release for connecting a deployment (especially an openstack one) to an external Ceph cluster. I don't know what is the feature testing
status for current release.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">Ovidiu</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black">
<hr size="3" width="100%" align="center">
</span></div>
<div id="divRpF345142">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black"> Chen, Haochuan Z [haochuan.z.chen@intel.com]<br>
<b>Sent:</b> Monday, December 23, 2019 9:18 AM<br>
<b>To:</b> Poncea, Ovidiu; Church, Robert<br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io'<br>
<b>Subject:</b> RE: question about ceph or storage</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">Sorry to disturb!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Merry Christmas to all StarlingX member!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Chen, Haochuan Z
<br>
<b>Sent:</b> Monday, December 23, 2019 3:17 PM<br>
<b>To:</b> Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>>; Church, Robert <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>><br>
<b>Subject:</b> RE: question about ceph or storage<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Hi Ovidiu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">For “system storage-backend-add -c <ceph_conf>”, user could deploy another ceph cluster, and add as storage backend, correctly.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">And this is this command’s intention, correct?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>>
<br>
<b>Sent:</b> Wednesday, December 18, 2019 7:47 PM<br>
<b>To:</b> Chen, Haochuan Z <<a href="mailto:haochuan.z.chen@intel.com" target="_blank">haochuan.z.chen@intel.com</a>>; Church, Robert <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>><br>
<b>Subject:</b> RE: question about ceph or storage<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">Hi Chen, see inline.</span><span style="color:black"><o:p></o:p></span></p>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black">
<hr size="3" width="100%" align="center">
</span></div>
<div id="divRpF719062">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black"> Chen, Haochuan Z [haochuan.z.chen@intel.com]<br>
<b>Sent:</b> Tuesday, December 17, 2019 10:46 AM<br>
<b>To:</b> Church, Robert; Poncea, Ovidiu<br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io'<br>
<b>Subject:</b> question about ceph or storage</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">Hi Bob & Ovidiu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Some question about ceph or storage.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">1, What’s storage tier and storage profile? What’s the
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red">[Ovi] Storage tiering is equivalent with this:
<a href="https://ceph.io/planet/deploying-ceph-with-storage-tiering/" target="_blank">
https://ceph.io/planet/deploying-ceph-with-storage-tiering/</a> .</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red">[Ovi] Profiles are managed by system storprofile-* and system host-apply-profile and are used to copy configuration from one node to another, identical node on initial provisioning. These profiles are only in system
inventory, there is no Ceph equivalent.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">2, why for duplex it request such puppet class dependency in ceph.pp? Is this request make all drbd config before class ceph?
</span><span style="color:red">[Ovi] ceph-mon in AIO-DX is DRBD managed and it has a single, floating, monitor. On DX, when you swact, the monitor is stopped on the active controller and started on the standby controller.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> Drbd::Resource <| |> -> Class['::ceph']<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> And flag file “.node_ceph_configured”, to inform drbd make init setup before ceph config?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">3, To launch ceph-mon, create a logical volume “ceph-mon-lv” and mount to /var/lib/ceph/mon, not directly mkdir “/var/lib/ceph/mon” for ceph-mon</span><span style="color:red"> [Ovi] Ceph monitors have their own
logical volume. They are managed through "system ceph-mon*" commands.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Thanks!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="color:black">From:</span></b><span style="color:black"> Chen, Haochuan Z
<br>
<b>Sent:</b> Wednesday, December 11, 2019 4:46 PM<br>
<b>To:</b> Church, Robert <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>>; Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>>;
Qi, Mingyuan <<a href="mailto:mingyuan.qi@intel.com" target="_blank">mingyuan.qi@intel.com</a>><br>
<b>Subject:</b> RE: ceph ops enabling in sysinv-conductor<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Hi Bob<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Some question, what’s storage tier and storage profile? As you said, we no longer manage pool and pg num, is this also unnecessary and we should remove it?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">BR!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Church, Robert <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>>
<br>
<b>Sent:</b> Wednesday, December 4, 2019 12:09 AM<br>
<b>To:</b> Chen, Haochuan Z <<a href="mailto:haochuan.z.chen@intel.com" target="_blank">haochuan.z.chen@intel.com</a>><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>>; Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>><br>
<b>Subject:</b> Re: ceph ops enabling in sysinv-conductor<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">See inline…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"Chen, Haochuan Z" <<a href="mailto:haochuan.z.chen@intel.com" target="_blank">haochuan.z.chen@intel.com</a>><br>
<b>Date: </b>Tuesday, December 3, 2019 at 2:18 AM<br>
<b>To: </b>Robert Church <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>><br>
<b>Cc: </b>"'starlingx-discuss@lists.starlingx.io'" <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>>, Ovidiu Poncea <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>><br>
<b>Subject: </b>RE: ceph ops enabling in sysinv-conductor</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black">Hi Bob<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">1, we could update default pg num in sysinv/helm/rbd_provisioner.py. But when platform-integ-apps applied, maybe user has not add osded, so the default pg num is still for user reference.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">And for user override, I think it should add in ceph-pool-audit, stx-platform-helm/helm-charts/ceph-pools-audit/templates/job-ceph-pools-audit.yaml. but pg num update will make rebalance which cause management
network jam.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">[RTC] For a more robust solution, consider the following:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:20.25pt;text-indent:-.25in"><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif;color:black">
</span><span style="color:black">We can update _met_app_apply_prerequisites () in sysinv/conductor.py for platform-integ-apps to require OSDs to be provisioned prior to applying the application. This will ensure an accurate OSD view when the application is
initially applied.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:20.25pt;text-indent:-.25in"><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif;color:black">
</span><span style="color:black">The provisioner system overrides (in sysinv/helm/rbd_provisioner.py) should have the ability to calculate and set an optimal PG number to avoid generating a warning.
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:20.25pt;text-indent:-.25in"><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif;color:black">
</span><span style="color:black">The rbd provisioner chart should also support setting a new chunk size (if greater than the existing size) to update the PG num. This will support user PG updates from the helm-overrides API.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:20.25pt;text-indent:-.25in"><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif;color:black">
</span><span style="color:black">I think it’s potentially a good idea for ceph-pool-audit to support adjusting the PG numbers as well but this is tricky as I don’t think we can reduce PG_num in Mimic without creating a new pool and copying the contents (pg_autoscaling
is added in Nautilis). This audit code would have to be very specific in adjusting the PG num as installing applications that add additional pools will change the PG_num distribution.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">2, the above case is only for system application. Any more alarm for PG number, request user should manage.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">BR!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Church, Robert <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>>
<br>
<b>Sent:</b> Tuesday, December 3, 2019 10:34 AM<br>
<b>To:</b> Chen, Haochuan Z <<a href="mailto:haochuan.z.chen@intel.com" target="_blank">haochuan.z.chen@intel.com</a>><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>>; Poncea, Ovidiu <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>><br>
<b>Subject:</b> Re: ceph ops enabling in sysinv-conductor<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Hi Martin,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">In STX 1.0, we ran bare-metal openstack services for which we created and managed specific ceph storage pools for glance, cinder, nova, and swift. In this environment, we managed their creation and parameters
(PGs and quotas) as the cluster scaled up.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">With the move to containerizing the openstack services in STX 2.0, pool creation is now driven mostly by helm charts packaged into an application (radosgw pools are the exception). Since we don’t know what additional
application(s) will be deployed and what pools may be created, we are currently longer managing pools as in STX 1.0.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">These related functions in sysinv/conductor/ceph.py are remaining from STX 1.0 and need to be removed and/or repurposed to meet any new requirements.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I took a look at the LP logs and it looks like we only have a single application applied: platform-integ-apps. This currently results in a single pool for 6 OSDs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> cluster:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> id: 6231df84-33be-4aa4-82ea-7408e0f2421c<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> health: HEALTH_WARN<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <span style="background:yellow">
too few PGs per OSD (21 < min 30)</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> services:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> mon: 3 daemons, quorum controller-0,controller-1,storage-0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> mgr: controller-0(active), standbys: controller-1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <span style="background:yellow">
osd: 6 osds: 6 up, 6 in</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> data:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> pools: 1 pools, 64 pgs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> objects: 0 objects, 0 B<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> usage: 645 MiB used, 5.4 TiB / 5.4 TiB avail<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> pgs: 64 active+clean<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Since every installation will install platform-integ-apps, I think we should do the following:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:20.25pt;text-indent:-.25in"><span style="color:black">1.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif;color:black">
</span><span style="color:black">Update the chunk_size calculation in sysinv/helm/rbd_provisioner.py to be dynamically calculated based on the number OSDs provisioned in the cluster. As this may be the only pool created, it should meet the minimum size characteristics
to avoid a ceph warning.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:20.25pt;text-indent:-.25in"><span style="color:black">2.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif;color:black">
</span><span style="color:black">Update the rbd-provisioner helm chart to support explicitly setting the pg_num based on the chunk size. Do this to allow setting a user override for the chunk size so that we can re-apply platform-integ-apps and explicitly set
new values. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Bob<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"Chen, Haochuan Z" <<a href="mailto:haochuan.z.chen@intel.com" target="_blank">haochuan.z.chen@intel.com</a>><br>
<b>Date: </b>Monday, December 2, 2019 at 1:15 AM<br>
<b>To: </b>Robert Church <<a href="mailto:Robert.Church@windriver.com" target="_blank">Robert.Church@windriver.com</a>><br>
<b>Cc: </b>"'starlingx-discuss@lists.starlingx.io'" <<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank">starlingx-discuss@lists.starlingx.io</a>>, Ovidiu Poncea <<a href="mailto:Ovidiu.Poncea@windriver.com" target="_blank">Ovidiu.Poncea@windriver.com</a>><br>
<b>Subject: </b>ceph ops enabling in sysinv-conductor</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black">Hi Bob<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I find some many function in sysinv-conductor/ceph.py, which could manage ceph cluster, such as create/delete/configure pool, audit pg etc.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">But why these function is not enabled? Or plan to request user to manage ceph cluster, such as create pool and configure pg num?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Now I checked this issue, pg too few, as user maybe deploy few osd, which make alarm. So for such issue, request user to decide correct pg num or user could ignore?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><a href="https://bugs.launchpad.net/starlingx/+bug/1844164" target="_blank">https://bugs.launchpad.net/starlingx/+bug/1844164</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">BR!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Martin, Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">SSP, Software Engineer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">021-61164330<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>