<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:449250257;
mso-list-type:hybrid;
mso-list-template-ids:-2083734358 1548107662 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:SimSun;
mso-bidi-font-family:"Courier New";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1
{mso-list-id:850679240;
mso-list-type:hybrid;
mso-list-template-ids:-78579596 -948135588 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:SimSun;
mso-bidi-font-family:"Courier New";}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2
{mso-list-id:1695957053;
mso-list-type:hybrid;
mso-list-template-ids:-967651464 36330168 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l2:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:54.0pt;
text-indent:-18.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:SimSun;
mso-bidi-font-family:"Courier New";}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:90.0pt;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:126.0pt;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:162.0pt;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:198.0pt;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:234.0pt;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:270.0pt;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:306.0pt;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:342.0pt;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l3
{mso-list-id:1697076423;
mso-list-template-ids:767977270;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Shuicheng,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Ansible bootstrap supports to deploy with authenticated registry.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">To deploy with authenticated registry, you will need to specify username and password in localhost.yml,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Ie…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#1F497D">docker_registries:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#1F497D"> defaults:<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:24.0pt"><span style="font-size:10.0pt;font-family:"Courier New";color:#1F497D">url: registry.com<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:24.0pt"><span style="font-size:10.0pt;font-family:"Courier New";color:#1F497D">username: registryUser<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:24.0pt"><span style="font-family:"Courier New";color:#1F497D">password: registryPass<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">You can also take a look playbooks/host_vars/bootstrap/default.yml for details.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">To test authenticated registry:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> - Bootstrap system with authenticated registry, verify system images are pulled from private authenticated registry and pushed to local registry.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> - After bootstrap is done, verify “auth-secret” is stored for each public registry in service-paramater.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> - Verify platform-integ-apps applied and images are pulled from private authenticated registry and pushed to local registry.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> - Deploy stx-monitor/stx-openstack app, verify images are pulled from private authenticated registry and pushed to local registry.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Angie<o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Lin, Shuicheng [mailto:shuicheng.lin@intel.com]
<br>
<b>Sent:</b> October-26-19 4:02 AM<br>
<b>To:</b> Miller, Frank; starlingx-discuss@lists.starlingx.io; Rowsell, Brent<br>
<b>Subject:</b> Re: [Starlingx-discuss] Minutes: StarlingX Containerization Meeting<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi Frank & Brent,<o:p></o:p></p>
<p class="MsoNormal">“<o:p></o:p></p>
<p class="MsoNormal"> - Have we tested with external registry and where username/pwd is required? Answer: Not yet. Action: Request is for Shuicheng to test (maybe work with Yong)<o:p></o:p></p>
<p class="MsoNormal">“<o:p></o:p></p>
<p class="MsoNormal">For external registry with authentication, the status is below:<o:p></o:p></p>
<ol style="margin-top:0cm" start="1" type="1">
<li class="MsoNormal" style="mso-list:l3 level1 lfo1">Pull image by crictl is supported. Cmd like: crictl pull --creds USERNAME:PASSWORD image_url<o:p></o:p></li><li class="MsoNormal" style="mso-list:l3 level1 lfo1">Pull image by Kubernetes is supported. It is following standard Kubernetes operation.
<o:p></o:p></li></ol>
<p class="MsoListParagraph">A Secret need be created to store the username/password, and pod yaml file will use it.<o:p></o:p></p>
<p class="MsoListParagraph"><a href="https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/">https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/</a><o:p></o:p></p>
<p class="MsoNormal">Since localhost.yml file for ansible playbook doesn’t support authentication info, it means StarlingX cannot be deployed by registry with authentication.<o:p></o:p></p>
<p class="MsoNormal">Registry with authentication should be used for 3<sup>rd</sup> application only. Is it right?<o:p></o:p></p>
<p class="MsoNormal">Please share me your test cases for registry with authentication if you still have concern on it.<o:p></o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Best Regards<o:p></o:p></p>
<p class="MsoNormal">Shuicheng<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><a name="_____replyseparator"></a><b>From:</b> Miller, Frank <Frank.Miller@windriver.com>
<br>
<b>Sent:</b> Wednesday, October 23, 2019 8:11 AM<br>
<b>To:</b> starlingx-discuss@lists.starlingx.io<br>
<b>Subject:</b> [Starlingx-discuss] Minutes: StarlingX Containerization Meeting<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks to all who participated in today’s meeting. Good progress across the stx.3.0 features.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Minutes for Oct 22:<o:p></o:p></p>
<p class="MsoNormal">1. Updates on stx.3.0 container features for MS3 milestone:<o:p></o:p></p>
<p class="MsoNormal">2004008: [Feature] Create HELM chart for Fault project [Ran An] - one horizon related commit needs a new approach which cannot be done in stx.3.0 timeframe<o:p></o:p></p>
<p class="MsoNormal">Decision for stx.3.0 is to keep FM container code but disable it so that FM runs natively for stx.3.0 - then address horizon issue in stx.4.0 and re-enable FM containers.<o:p></o:p></p>
<p class="MsoNormal">Ran has proposed doing this by bringing back the “fault_management_pod_disabled” flag that was introduced in stx.2.0 and removed in stx.3.0<o:p></o:p></p>
<p class="MsoNormal">Ran's forecast for completing this and posting gerrit review is end of week, ie Oct 25<o:p></o:p></p>
<p class="MsoNormal">2005937: Intel GPU K8s device plugin support in StarlingX [An Ran] - Ran confirmed this feature is now merged<o:p></o:p></p>
<p class="MsoNormal">Below 2 patches are for k8s device plugin enabling mechanism in ansible and sysinv, add intel gpu device plugin as well --- merged<o:p></o:p></p>
<p class="MsoNormal">- <a href="https://review.opendev.org/#/c/666511/">https://review.opendev.org/#/c/666511/</a><o:p></o:p></p>
<p class="MsoNormal">- <a href="https://review.opendev.org/#/c/666510/">https://review.opendev.org/#/c/666510/</a><o:p></o:p></p>
<p class="MsoNormal">This patch is for align all device plugins in one method. Will be revisit when default device plugins require to be align. --- abandoned<o:p></o:p></p>
<p class="MsoNormal">- <a href="https://review.opendev.org/#/c/685592/">https://review.opendev.org/#/c/685592/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2005514: QAT K8S device plugin support in StarlingX<o:p></o:p></p>
<p class="MsoNormal">Mingyuan: status of 3 patches which depended on Ran's 666510/666511<o:p></o:p></p>
<p class="MsoNormal">- <a href="https://review.opendev.org/684890">https://review.opendev.org/684890</a> --> merged<o:p></o:p></p>
<p class="MsoNormal">- <a href="https://review.opendev.org/688344">https://review.opendev.org/688344</a> --> Cores have +2'd -- just need WFL +1<o:p></o:p></p>
<p class="MsoNormal">- <a href="https://review.opendev.org/688363">https://review.opendev.org/688363</a> --> Cores have +2'd -- just need WFL +1<o:p></o:p></p>
<p class="MsoNormal">2005860: Upversion container components [Al Bailey] --> merged<o:p></o:p></p>
<p class="MsoNormal">2004761: Integrate Backup & Restore with Containers (for platform) [Ovidiu Poncea] --> merged
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2. Complete review of KATA containers feature with container Cores.
<o:p></o:p></p>
<p class="MsoNormal"> - Shuicheng's update:<o:p></o:p></p>
<p class="MsoNormal"> Here is the latest test status for kata integration in StarlingX.<o:p></o:p></p>
<p class="MsoNormal">(1) Pass Sanity test for AIO-SX/AIO-DX/Multi/Multi+Storage.<o:p></o:p></p>
<p class="MsoNormal">(2) Tested private registry (insecure registry) enabled/disabled.<o:p></o:p></p>
<p class="MsoNormal">(3) Tested docker proxy enabled/disabled.<o:p></o:p></p>
<p class="MsoNormal">(4) Tested swact/lock & unlock/remove & apply application.<o:p></o:p></p>
<p class="MsoNormal">(5) Tested local registry (registry.local:9001) with image pull/push, and access both from kubernetes and crictl (CLI).
<o:p></o:p></p>
<p class="MsoNormal"> For regression test, I checked container related test cases in below doc, and executed most of them. No issue is found yet.<o:p></o:p></p>
<p class="MsoNormal"> <a href="https://docs.google.com/spreadsheets/d/1dwcBwY4Yq1Lo9Der4RylzQ6KYp0BsMHohhEmhwpauDo/edit?usp=sharing">
https://docs.google.com/spreadsheets/d/1dwcBwY4Yq1Lo9Der4RylzQ6KYp0BsMHohhEmhwpauDo/edit?usp=sharing</a><o:p></o:p></p>
<p class="MsoNormal"> Currently, I meet a deploy issue with IPv6 environment, and the issue could be reproduced with master daily build also.<o:p></o:p></p>
<p class="MsoNormal"> I am checking whether it is caused by my environment or a real issue of master now.<o:p></o:p></p>
<p class="MsoNormal"> Here is the patch list waiting review: <a href="https://review.opendev.org/#/q/topic:kata+(status:open)">
https://review.opendev.org/#/q/topic:kata+(status:open)</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Some questions/comments from TL/Brent:<o:p></o:p></p>
<p class="MsoNormal"> - Have we tested with external registry and where username/pwd is required? Answer: Not yet. Action: Request is for Shuicheng to test (maybe work with Yong)<o:p></o:p></p>
<p class="MsoNormal"> - What is the plan to remove dockerd? <o:p></o:p></p>
<p class="MsoNormal"> - Plan is to remove in a separate commit after the main code merges. Shuicheng plan is to wait till stx.4.0 to remove - is this acceptable?
<o:p></o:p></p>
<p class="MsoNormal"> - Shuicheng plan is to have the dockerd removal commit ready by end of November. Plan then to merge this into stx.4.0 once it is open.<o:p></o:p></p>
<p class="MsoNormal"> - We agreed this would be acceptable.<o:p></o:p></p>
<p class="MsoNormal"> - Looks like there is one upstream patch we are carrying (in containerd and runc?). What is the plan to upstream these patches?
<o:p></o:p></p>
<p class="MsoNormal"> - Action: Ensure there is a task to upversion docker distribution registry to remove the need to carry this patch.
<o:p></o:p></p>
<p class="MsoNormal"> - This too will be done early in stx.4.0. Shuicheng to update us in 1 week on when this task can be completed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Decision to allow into stx.3.0? Three requirements:<o:p></o:p></p>
<p class="MsoNormal"> - Shuicheng to complete testing for IPv6 and external registry with username/pwd authentication<o:p></o:p></p>
<p class="MsoNormal"> - Cores to complete their reviews<o:p></o:p></p>
<p class="MsoNormal"> - Frank to confirm with Ada that testing of this feature can be done in time for stx.3.0<o:p></o:p></p>
<p class="MsoNormal"> - Backup plan: If instability is seen in stx.3.0 or significant testing by Ada results in failures then the feature would need to be backed up and fixed/delivered in stx.4.0<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#333333">Frank Miller</span></b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#333333">, Manager, Engineering,
</span><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#CC0000">Wind River</span></b><span style="color:#333333"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#333333">direct 613.963.1389 zoom
<a href="mailto:frank.miller@windriver.com">frank.miller@windriver.com</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#333333">350 Terry Fox Drive, Suite 200, Kanata, ON K2K 2W5<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>