<div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div style="margin: 0;">The machine belongs to my school. How rich a school it is! :-)</div><div style="margin: 0;"><br></div><div style="margin: 0;">I have tried that method. </div><div style="margin: 0;"><br></div><div style="margin: 0;">Without any change, the result of "cat /sys/block/sda/device/../*/scsi_disk/*/max_write_same_blocks" is already 0.</div><div style="margin: 0;"><br></div><div style="margin: 0;">BTW, the puppy log has some errors, but I think the most important one is listed below:</div><div style="margin: 0;">===============================================</div><div style="margin: 0;">2020-04-18T03:07:22.686 Error: 2020-04-18 03:07:22 +0000 drbdadm -- --assume-peer-has-space resize drbd-dockerdistribution returned 10 instead of one of [0]</div><div style="margin: 0;">2020-04-18T03:07:22.691 /usr/share/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'</div><div style="margin: 0;">2020-04-18T03:07:22.696 /usr/share/ruby/vendor_ruby/puppet/type/exec.rb:160:in `sync'</div><div style="margin: 0;">2020-04-18T03:07:22.702 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync'</div><div style="margin: 0;">2020-04-18T03:07:22.709 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed'</div><div style="margin: 0;">2020-04-18T03:07:22.715 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes'</div><div style="margin: 0;">2020-04-18T03:07:22.721 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each'</div><div style="margin: 0;">2020-04-18T03:07:22.727 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes'</div><div style="margin: 0;">2020-04-18T03:07:22.732 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate'</div><div style="margin: 0;">2020-04-18T03:07:22.739 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:230:in `apply'</div><div style="margin: 0;">2020-04-18T03:07:22.744 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:246:in `eval_resource'</div><div style="margin: 0;">2020-04-18T03:07:22.750 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'</div><div style="margin: 0;">2020-04-18T03:07:22.756 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'</div><div style="margin: 0;">2020-04-18T03:07:22.761 /usr/share/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark'</div><div style="margin: 0;">2020-04-18T03:07:22.768 /usr/share/ruby/benchmark.rb:296:in `realtime'</div><div style="margin: 0;">2020-04-18T03:07:22.774 /usr/share/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark'</div><div style="margin: 0;">2020-04-18T03:07:22.781 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'</div><div style="margin: 0;">2020-04-18T03:07:22.787 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'</div><div style="margin: 0;">2020-04-18T03:07:22.792 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'</div><div style="margin: 0;">2020-04-18T03:07:22.799 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'</div><div style="margin: 0;">2020-04-18T03:07:22.806 /usr/share/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'</div><div style="margin: 0;">2020-04-18T03:07:22.812 /usr/share/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination'</div><div style="margin: 0;">2020-04-18T03:07:22.818 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'</div><div style="margin: 0;">2020-04-18T03:07:22.824 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'</div><div style="margin: 0;">2020-04-18T03:07:22.830 /usr/share/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark'</div><div style="margin: 0;">2020-04-18T03:07:22.837 /usr/share/ruby/benchmark.rb:296:in `realtime'</div><div style="margin: 0;">2020-04-18T03:07:22.843 /usr/share/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'</div><div style="margin: 0;">2020-04-18T03:07:22.850 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'</div><div style="margin: 0;">2020-04-18T03:07:22.855 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'</div><div style="margin: 0;">2020-04-18T03:07:22.861 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'</div><div style="margin: 0;">2020-04-18T03:07:22.867 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'</div><div style="margin: 0;">2020-04-18T03:07:22.874 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'</div><div style="margin: 0;">2020-04-18T03:07:22.880 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'</div><div style="margin: 0;">2020-04-18T03:07:22.887 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog'</div><div style="margin: 0;">2020-04-18T03:07:22.893 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'</div><div style="margin: 0;">2020-04-18T03:07:22.900 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'</div><div style="margin: 0;">2020-04-18T03:07:22.906 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'</div><div style="margin: 0;">2020-04-18T03:07:22.912 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'</div><div style="margin: 0;">2020-04-18T03:07:22.918 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'</div><div style="margin: 0;">2020-04-18T03:07:22.923 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `block in run'</div><div style="margin: 0;">2020-04-18T03:07:22.929 /usr/share/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail'</div><div style="margin: 0;">2020-04-18T03:07:22.935 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `run'</div><div style="margin: 0;">2020-04-18T03:07:22.941 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'</div><div style="margin: 0;">2020-04-18T03:07:22.946 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'</div><div style="margin: 0;">2020-04-18T03:07:22.952 /usr/bin/puppet:5:in `<main>'</div><div style="margin: 0;">2020-04-18T03:07:22.957 Error: 2020-04-18 03:07:22 +0000 /Stage[main]/Platform::Drbd::Dockerdistribution::Bootstrap/Platform::Drbd::Filesystem[drbd-dockerdistribution]/Exec[drbd resize drbd-dockerdistribution]/returns: change from notrun to 0 failed: drbdadm -- --assume-peer-has-space resize drbd-dockerdistribution returned 10 instead of one of [0]</div><div>======================================================</div><div>It seems that drbdadm doesn't work. And I am not sure what is Error 10 is. I wonder if you have some hints,</div><div style="margin: 0;"><br></div><p style="margin: 0;"><br></p><p style="margin: 0;"><br></p><p style="margin: 0;"><br></p><p style="margin: 0;"><br></p><div style="position:relative;zoom:1"></div><div id="divNeteaseMailCard"></div><p style="margin: 0;"><br></p><p>在 2020-04-19 16:01:31，"Sun, Austin" <austin.sun@intel.com> 写道：</p><blockquote id="isReplyContent" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">



<style></style>


<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Sda is so big. rich man :)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Get some search result from web [1]
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Would you like check the result via the command
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">cat /sys/block/sda/device/../*/scsi_disk/*/max_write_same_blocks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">it is 0 in my setup .   if it is 1 , you might change to 0 and try .
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">disable write same like this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">echo 0 | tee /sys/block/*/device/../*/scsi_disk/*/max_write_same_blocks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">(for the relevant backend devices)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">If you use LVM, you may need to vgchange -an ; vgchange -ay after that,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">(at least for the relevant VGs), if they have already been activated.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal">[1] <a href="https://lists.linbit.com/pipermail/drbd-user/2017-February/023024.html">
https://lists.linbit.com/pipermail/drbd-user/2017-February/023024.html</a><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Thanks.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">BR<br>
Austin Sun. <o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">
</span><span lang="ZH-CN" style="font-size:11.0pt">周擎阳</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> <18245043314@163.com>
<br>
<b>Sent:</b> Sunday, April 19, 2020 3:38 PM<br>
<b>To:</b> Sun, Austin <austin.sun@intel.com><br>
<b>Cc:</b> starlingx-discuss@lists.starlingx.io<br>
<b>Subject:</b> Re:RE: Re:RE: [Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">4.9T is sda size.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">the "lsblk" on my startlingx looks like this:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><img border="0" width="1151" height="711" style="width:11.9916in;height:7.4083in" id="_x0000_i1025" src="cid:cf96346$1$171917e77b6$Coremail$18245043314$163.com"><o:p></o:p></span></p>
</div>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p> </o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p> </o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p> </o:p></span></p>
<p><span lang="ZH-CN" style="font-size:10.5pt;color:black">在</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> 2020-04-19 15:14:33</span><span lang="ZH-CN" style="font-size:10.5pt;color:black">，</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">"Sun,
 Austin" <<a href="mailto:austin.sun@intel.com">austin.sun@intel.com</a>> </span>
<span lang="ZH-CN" style="font-size:10.5pt;color:black">写道：</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in" id="isReplyContent">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">is this 4.9T sda size or all disks size ?
</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">how much sda is ?
</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Thanks.</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">BR<br>
Austin Sun. </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">
</span><span lang="ZH-CN" style="font-size:11.0pt;color:black">周擎阳</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> <<a href="mailto:18245043314@163.com">18245043314@163.com</a>>
<br>
<b>Sent:</b> Sunday, April 19, 2020 2:36 PM<br>
<b>To:</b> Sun, Austin <<a href="mailto:austin.sun@intel.com">austin.sun@intel.com</a>><br>
<b>Cc:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> Re:RE: [Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">Dear Sir:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">1.)  I am installing in</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> bare
 metal environment, funny thing is that the process works when I install it in virtual environment in the same machine. The memory is 250G and the disk is 4.9T. </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2.)  the localhost.yml is just the one in the guide. Here is the detail:</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">===============================================================</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">system_mode: simplex</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">dns_servers:</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">  - 8.8.8.8</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">  - 8.8.4.4</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">external_oam_subnet: 172.16.0.0/16</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">external_oam_gateway_address: 172.16.0.1</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">external_oam_floating_address: 172.16.10.205</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">admin_username: admin</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">admin_password: ASdf1234!@#$</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">ansible_become_pass: ASdf1234!@#$</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"># Add these lines to configure Docker to use a proxy server</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"># docker_http_proxy:
<a href="http://my.proxy.com:1080">http://my.proxy.com:1080</a></span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"># docker_https_proxy:
<a href="https://my.proxy.com:1443">https://my.proxy.com:1443</a></span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"># docker_no_proxy:</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">#   - 1.2.3.4</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">===============================================================</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">3.) For the log file is too long, I simply attach it to this email. For convenience I simply put the
 error part of the log file below:</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">===============================================================</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.010 Error: 2020-04-18 03:10:32 +0000 Failed to apply catalog: Execution of '/usr/bin/openstack
 --os-interface internal role list --quiet --format csv' returned 1: Unable to establish connection to
<a href="http://127.0.0.1:5000/v3/roles">http://127.0.0.1:5000/v3/roles</a>?: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /v3/roles (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f416632e190>:
 Failed to establish a new connection: [Errno 111] Connection refused',)) (tried 44, for a total of 170 seconds)</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.017 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:126:in
 `rescue in block (2 levels) in request'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.021 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:88:in
 `block (2 levels) in request'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.026 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:87:in
 `loop'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.032 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:87:in
 `block in request'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.039 /usr/share/ruby/vendor_ruby/puppet/util.rb:128:in `withenv'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.043 /usr/share/openstack-puppet/modules/openstacklib/lib/puppet/provider/openstack.rb:82:in
 `request'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.049 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone.rb:325:in
 `request_by_service_token'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.055 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone.rb:258:in
 `rescue in request'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.061 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone.rb:254:in
 `request'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.067 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone_role/openstack.rb:51:in
 `instances'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.073 /usr/share/openstack-puppet/modules/keystone/lib/puppet/provider/keystone_role/openstack.rb:64:in
 `prefetch'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.079 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:323:in `prefetch'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.085 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:223:in `prefetch_if_necessary'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.091 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:107:in `block in evaluate'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.096 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:116:in `call'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.102 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:116:in `traverse'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.108 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.114 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.121 /usr/share/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.126 /usr/share/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.132 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.138 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.143 /usr/share/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.148 /usr/share/ruby/benchmark.rb:296:in `realtime'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.154 /usr/share/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.160 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.166 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.171 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.176 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.182 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.188 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.193 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.199 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.204 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.210 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.216 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.222 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.229 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `block in run'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.234 /usr/share/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.240 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `run'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.246 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.251 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2020-04-18T03:10:32.256 /usr/bin/puppet:5:in `<main>'</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">==========================================================================================</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">thanks,</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">yours,</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">zhou qingyang.</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
</div>
</div>
</div>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">At 2020-04-19 13:36:53, "Sun, Austin" <<a href="mailto:austin.sun@intel.com">austin.sun@intel.com</a>> wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt" id="isReplyContent">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Hi qingyang:
</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">     Would you like to share some more info?</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">1) are u install in bare metal or virtual environment ?  how much are memory and disk ?</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2) could you provide localhost.yml you are using ?
</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">3) provide /var/log/puppet/latest/puppet.log  ?
</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Thanks.</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">BR<br>
Austin Sun. </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">
</span><span lang="ZH-CN" style="font-size:11.0pt;color:black">周擎阳</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> <<a href="mailto:18245043314@163.com">18245043314@163.com</a>>
<br>
<b>Sent:</b> Saturday, April 18, 2020 6:52 PM<br>
<b>To:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> [Starlingx-discuss] error when installing starlingx3.0 all-in-one in bare metal</span><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">Dear sir:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">   I am trying to install starlingx following the all-in-one link:<a href="https://docs.starlingx.io/deploy_install_guides/r3_release/bare_metal/aio_simplex.html">https://docs.starlingx.io/deploy_install_guides/r3_release/bare_metal/aio_simplex.html</a>.
 However I stuck in the "ansible" step. From the dmesg, I can see the errors below:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79591.931815] block drbd0: Resize while not connected was forced by the user!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.058071] block drbd8: local WRITE IO error sector 1296+4088 on dm-8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.058083] block drbd8: disk( UpToDate -> Failed ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.058113] block drbd8: Local IO failed in __req_mod. Detaching...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.058146] block drbd8: IO ERROR: neither local nor remote data, sector 0+0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.058178] block drbd8: helper command: /sbin/drbdadm pri-on-incon-degr minor-8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061512] block drbd8: IO ERROR: neither local nor remote data, sector 5384+4096<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061603] drbd8: WRITE SAME failed. Manually zeroing.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061664] block drbd8: IO ERROR: neither local nor remote data, sector 5384+256<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061764] block drbd8: IO ERROR: neither local nor remote data, sector 5640+256<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061859] block drbd8: IO ERROR: neither local nor remote data, sector 5896+256<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061880] block drbd8: helper command: /sbin/drbdadm pri-on-incon-degr minor-8 exit code 0 (0x0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061972] block drbd8: 1024 MB (262127 bits) marked out-of-sync by on disk bit-map.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.061988] block drbd8: disk( Failed -> Diskless ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.835252] block drbd0: local WRITE IO error sector 2328+4080 on dm-3<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.835262] block drbd0: disk( UpToDate -> Failed ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.835293] block drbd0: Local IO failed in __req_mod. Detaching...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.835369] drbd0: WRITE SAME failed. Manually zeroing.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.835388] block drbd0: helper command: /sbin/drbdadm pri-on-incon-degr minor-0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.838928] block drbd0: helper command: /sbin/drbdadm pri-on-incon-degr minor-0 exit code 0 (0x0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.838970] block drbd0: 2048 MB (524263 bits) marked out-of-sync by on disk bit-map.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79592.838980] block drbd0: disk( Failed -> Diskless ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038394] EXT4-fs error (device drbd0): ext4_wait_block_bitmap:516: comm initdb: Cannot read block
 bitmap - block_group = 1, block_bitmap = 258<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038541] Buffer I/O error on dev drbd0, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038686] EXT4-fs warning (device drbd0): ext4_end_bio:302: I/O error -5 writing to inode 29 (offset
 0 size 20480 starting block 65536)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038689] Buffer I/O error on device drbd0, logical block 65536<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038743] Buffer I/O error on device drbd0, logical block 65537<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038805] Buffer I/O error on device drbd0, logical block 65538<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038858] Buffer I/O error on device drbd0, logical block 65539<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79593.038911] Buffer I/O error on device drbd0, logical block 65540<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79595.501408] block drbd2: Resize while not connected was forced by the user!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79595.701486] EXT4-fs (drbd5): mounted filesystem with ordered data mode. Opts: (null)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79595.731488] EXT4-fs (drbd5): re-mounted. Opts: (null)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79595.755858] block drbd5: Resize while not connected was forced by the user!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.382427] EXT4-fs (drbd1): mounted filesystem with ordered data mode. Opts: (null)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.427131] EXT4-fs (drbd1): re-mounted. Opts: (null)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.456439] block drbd1: Resize while not connected was forced by the user!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.763258] block drbd1: local WRITE IO error sector 2320+4088 on dm-9<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.763270] block drbd1: disk( UpToDate -> Failed ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.763304] block drbd1: Local IO failed in __req_mod. Detaching...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.763412] block drbd1: helper command: /sbin/drbdadm pri-on-incon-degr minor-1<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.767304] block drbd1: helper command: /sbin/drbdadm pri-on-incon-degr minor-1 exit code 0 (0x0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.767348] block drbd1: 2048 MB (524263 bits) marked out-of-sync by on disk bit-map.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.767358] block drbd1: disk( Failed -> Diskless ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79596.788531] drbd1: WRITE SAME failed. Manually zeroing.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.168463] block drbd7: role( Secondary -> Primary ) disk( Inconsistent -> UpToDate ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.179035] block drbd7: Forced to consider local data as UpToDate!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.179053] block drbd7: new current UUID 9759105C28A7AAAD:0000000000000004:0000000000000000:0000000000000000<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.616990] block drbd0: 52 messages suppressed in /builddir/build/BUILD/drbd-8.4.11-1/obj/default/drbd_req.c:1446.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.616999] block drbd0: IO ERROR: neither local nor remote data, sector 1572864+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.617096] Buffer I/O error on dev drbd0, logical block 196608, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.617187] JBD2: Error -5 detected when updating journal superblock for drbd0-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.617274] Aborting journal on device drbd0-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.617328] Buffer I/O error on dev drbd0, logical block 196608, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.617417] JBD2: Error -5 detected when updating journal superblock for drbd0-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.680678] Buffer I/O error on dev drbd8, logical block 65536, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.680772] JBD2: Error -5 detected when updating journal superblock for drbd8-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.680859] Aborting journal on device drbd8-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.680914] Buffer I/O error on dev drbd8, logical block 65536, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.681003] JBD2: Error -5 detected when updating journal superblock for drbd8-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.760212] EXT4-fs (drbd0): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.760337] Buffer I/O error on dev drbd0, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.760450] EXT4-fs error (device drbd0): ext4_journal_check_start:56: Detected aborted journal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.760551] EXT4-fs (drbd0): Remounting filesystem read-only<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.760620] EXT4-fs (drbd0): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79597.760682] Buffer I/O error on dev drbd0, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.177668] block drbd5: local WRITE IO error sector 1296+4088 on dm-2<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.177680] block drbd5: disk( UpToDate -> Failed ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.177711] block drbd5: Local IO failed in __req_mod. Detaching...<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.177786] drbd5: WRITE SAME failed. Manually zeroing.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.177813] block drbd5: helper command: /sbin/drbdadm pri-on-incon-degr minor-5<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.181353] block drbd5: helper command: /sbin/drbdadm pri-on-incon-degr minor-5 exit code 0 (0x0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.181396] block drbd5: 1024 MB (262127 bits) marked out-of-sync by on disk bit-map.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79600.181406] block drbd5: disk( Failed -> Diskless ) <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.244660] EXT4-fs error (device drbd1): ext4_wait_block_bitmap:516: comm async_310: Cannot read block
 bitmap - block_group = 1, block_bitmap = 258<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.244825] Buffer I/O error on dev drbd1, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.244989] EXT4-fs warning (device drbd1): ext4_end_bio:302: I/O error -5 writing to inode 17 (offset
 0 size 4096 starting block 65536)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.244994] Buffer I/O error on device drbd1, logical block 65536<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.607395] Buffer I/O error on dev drbd1, logical block 196608, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.607495] JBD2: Error -5 detected when updating journal superblock for drbd1-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.607582] Aborting journal on device drbd1-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.607638] Buffer I/O error on dev drbd1, logical block 196608, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79601.607727] JBD2: Error -5 detected when updating journal superblock for drbd1-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.351833] EXT4-fs (drbd1): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.351905] block drbd1: 28 messages suppressed in /builddir/build/BUILD/drbd-8.4.11-1/obj/default/drbd_req.c:1446.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.351909] block drbd1: IO ERROR: neither local nor remote data, sector 0+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.351970] Buffer I/O error on dev drbd1, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.352063] EXT4-fs error (device drbd1): ext4_journal_check_start:56: Detected aborted journal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.352159] EXT4-fs (drbd1): Remounting filesystem read-only<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.352216] EXT4-fs (drbd1): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79602.352274] Buffer I/O error on dev drbd1, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79605.662600] Buffer I/O error on dev drbd5, logical block 65536, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79605.662694] JBD2: Error -5 detected when updating journal superblock for drbd5-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79605.662781] Aborting journal on device drbd5-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79605.662836] Buffer I/O error on dev drbd5, logical block 65536, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[79605.662924] JBD2: Error -5 detected when updating journal superblock for drbd5-8.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.347102] block drbd8: 3 messages suppressed in /builddir/build/BUILD/drbd-8.4.11-1/obj/default/drbd_req.c:1446.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.347111] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.347330] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block
 (ino 2, block 0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.347372] block drbd8: IO ERROR: neither local nor remote data, sector 0+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.347586] Buffer I/O error on dev drbd8, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.347798] EXT4-fs error (device drbd8): ext4_journal_check_start:56: Detected aborted journal<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.348118] EXT4-fs (drbd8): Remounting filesystem read-only<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.348130] EXT4-fs (drbd8): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.348135] block drbd8: IO ERROR: neither local nor remote data, sector 0+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80656.348140] Buffer I/O error on dev drbd8, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80661.007923] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80661.008142] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block
 (ino 2, block 0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80663.424845] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80663.425063] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block
 (ino 2, block 0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80670.001525] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80670.001751] EXT4-fs warning (device drbd8): __ext4_read_dirblock:902: error reading directory block
 (ino 2, block 0)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.182038] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.182271] EXT4-fs error (device drbd8): ext4_find_entry:1312: inode #2: comm sh: reading directory
 lblock 0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.182529] EXT4-fs (drbd8): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.182723] block drbd8: IO ERROR: neither local nor remote data, sector 0+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.182956] Buffer I/O error on dev drbd8, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.186801] block drbd8: IO ERROR: neither local nor remote data, sector 1096+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.187036] EXT4-fs error (device drbd8): ext4_find_entry:1312: inode #2: comm sh: reading directory
 lblock 0<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.187271] EXT4-fs (drbd8): previous I/O error to superblock detected<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.187457] block drbd8: IO ERROR: neither local nor remote data, sector 0+8<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">[80817.187648] Buffer I/O error on dev drbd8, logical block 0, lost sync page write<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">I guess there are something wrong with drbd, but I am not sure about the problem. Do you have any ideas
 or do you need more information to solve it?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">Thanks.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">yours,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">zhou qingyang.<o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><br>
 <o:p></o:p></span></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"> <o:p></o:p></span></p>
<p><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black"><br>
 <o:p></o:p></span></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<p><br>
 <o:p></o:p></p>
</div>


</blockquote></div><br><br><span title="neteasefooter"><p><br/> </p></span>