[Starlingx-discuss] 答复: StarlingX QAT validation progress

张志国 zhangzhg at neusoft.com
Tue May 28 05:42:04 UTC 2019


Hi Shuicheng,

 

This is really a problem. First you need to prepare an image with qat driver
and qzip and compile it. Then upload the image to starlingx and create a qat
instance for testing.

 

However, we have also tried cirros to passthrough the qat device, but we
cannot verify the qat function on the cirros.

 

At the same time, we used our own created qat image (including qat driver
and qzip) to test without any problems in the All in one simplex deployment
mode. So I am not sure if there is a problem on the 2+2 multi nodes
deployment mode, I want to confirm that the configuration file is correct.

 

Best Regards

Zhiguo

 

 

·¢¼þÈË: Lin, Shuicheng [mailto:shuicheng.lin at intel.com] 
·¢ËÍʱ¼ä: 2019Äê5ÔÂ28ÈÕ 12:31
ÊÕ¼þÈË: ÕÅÖ¾¹ú <zhangzhg at neusoft.com>; Perez, Ricardo O
<ricardo.o.perez at intel.com>; Zhao, ShuaiX <shuaix.zhao at intel.com>;
starlingx-discuss at lists.starlingx.io
³­ËÍ: Cabrales, Ada <ada.cabrales at intel.com>; Xie, Cindy
<cindy.xie at intel.com>; Wang, Hai Tao <hai.tao.wang at intel.com>;
fuyong at neusoft.com; lilong-neu at neusoft.com; 'zhaos' <zhaos at neusoft.com>;
sy at neusoft.com; zuoyl at neusoft.com
Ö÷Ìâ: RE: StarlingX QAT validation progress

 

Hi Zhiguo/Ricardo,

I find Ricardo is using CirrOS for the qat vm. Is it correct? Shouldn¡¯t we
use CentOS as the guest vm image?

 

Best Regards

Shuicheng

 

From: ÕÅÖ¾¹ú [ <mailto:zhangzhg at neusoft.com> mailto:zhangzhg at neusoft.com] 
Sent: Tuesday, May 28, 2019 11:20 AM
To: Perez, Ricardo O < <mailto:ricardo.o.perez at intel.com>
ricardo.o.perez at intel.com>; Zhao, ShuaiX < <mailto:shuaix.zhao at intel.com>
shuaix.zhao at intel.com>;  <mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
Cc: Cabrales, Ada < <mailto:ada.cabrales at intel.com> ada.cabrales at intel.com>;
Xie, Cindy < <mailto:cindy.xie at intel.com> cindy.xie at intel.com>; Wang, Hai
Tao < <mailto:hai.tao.wang at intel.com> hai.tao.wang at intel.com>; Lin,
Shuicheng < <mailto:shuicheng.lin at intel.com> shuicheng.lin at intel.com>;
<mailto:fuyong at neusoft.com> fuyong at neusoft.com;  <mailto:lilong-neu at neusoft.
com> lilong-neu at neusoft.com; 'zhaos' < <mailto:zhaos at neusoft.com>
zhaos at neusoft.com>;  <mailto:sy at neusoft.com> sy at neusoft.com;
<mailto:zuoyl at neusoft.com> zuoyl at neusoft.com
Subject: ´ð¸´: StarlingX QAT validation progress

 

Hi   Ricardo:

 

Could you please send me the /opt/platform/helm/19.01/openstack-nova.yaml
file on all compute nodes? I want to confirm the format and content of these
configuration files.

 

Thanks

-Zhiguo.

 

·¢¼þÈË: Perez, Ricardo O [ <mailto:ricardo.o.perez at intel.com>
mailto:ricardo.o.perez at intel.com] 
·¢ËÍʱ¼ä: 2019Äê5ÔÂ28ÈÕ 2:12
ÊÕ¼þÈË:  <mailto:zhangzhg at neusoft.com> zhangzhg at neusoft.com; Zhao, ShuaiX <
<mailto:shuaix.zhao at intel.com> shuaix.zhao at intel.com>;
<mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
³­ËÍ: Cabrales, Ada < <mailto:ada.cabrales at intel.com>
ada.cabrales at intel.com>; Xie, Cindy < <mailto:cindy.xie at intel.com>
cindy.xie at intel.com>; Wang, Hai Tao < <mailto:hai.tao.wang at intel.com>
hai.tao.wang at intel.com>; Lin, Shuicheng < <mailto:shuicheng.lin at intel.com>
shuicheng.lin at intel.com>;  <mailto:fuyong at neusoft.com> fuyong at neusoft.com;
<mailto:lilong-neu at neusoft.com> lilong-neu at neusoft.com; 'zhaos' <
<mailto:zhaos at neusoft.com> zhaos at neusoft.com>;  <mailto:sy at neusoft.com>
sy at neusoft.com;  <mailto:zuoyl at neusoft.com> zuoyl at neusoft.com
Ö÷Ìâ: RE: StarlingX QAT validation progress

 

Hi again :)

 

I have tried the change that you proposed and re-applied the helm-override. 

 

However I¡¯m still getting an error status while launching a VM with the
property:pci_passtrough. Even do, if I remove such property from the flavor
the VM it¡¯s created with no issue. Please take a look to the below output: 

 

controller-0:~# export OS_CLOUD=openstack_helm

controller-0:~# openstack server create --image cirros --flavor m1.small
--network public-net0 qat

+-------------------------------------+-------------------------------------
------------+

| Field                               | Value
|

+-------------------------------------+-------------------------------------
------------+

| OS-DCF:diskConfig                   | MANUAL
|

| OS-EXT-AZ:availability_zone         |
|

| OS-EXT-SRV-ATTR:host                | None
|

| OS-EXT-SRV-ATTR:hypervisor_hostname | None
|

| OS-EXT-SRV-ATTR:instance_name       |
|

| OS-EXT-STS:power_state              | NOSTATE
|

| OS-EXT-STS:task_state               | scheduling
|

| OS-EXT-STS:vm_state                 | building
|

| OS-SRV-USG:launched_at              | None
|

| OS-SRV-USG:terminated_at            | None
|

| accessIPv4                          |
|

| accessIPv6                          |
|

| addresses                           |
|

| adminPass                           | iRpqEwjaQd4k
|

| config_drive                        |
|

| created                             | 2019-05-27T12:57:42Z
|

| flavor                              | m1.small
(31081c0f-2c10-40b2-b3f9-049ca660e12d) |

| hostId                              |
|

| id                                  | 17eb84cf-0666-48c7-b049-cd9caae2358f
|

| image                               | cirros
(3ecd6177-ae54-42a2-8aa3-a442e96f7642)   |

| key_name                            | None
|

| name                                | qat
|

| progress                            | 0
|

| project_id                          | 411d5846ca314075af13c7c496e44880
|

| properties                          |
|

| security_groups                     | name='default'
|

| status                              | BUILD
|

| updated                             | 2019-05-27T12:57:42Z
|

| user_id                             | 597d02e09d4d4a3b8172aeda82ab364d
|

| volumes_attached                    |
|

+-------------------------------------+-------------------------------------
------------+

controller-0:~# openstack flavor set m1.small --no-property

controller-0:~# openstack server create --image cirros --flavor m1.small
--network public-net0 qat2

+-------------------------------------+-------------------------------------
------------+

| Field                               | Value
|

+-------------------------------------+-------------------------------------
------------+

| OS-DCF:diskConfig                   | MANUAL
|

| OS-EXT-AZ:availability_zone         |
|

| OS-EXT-SRV-ATTR:host                | None
|

| OS-EXT-SRV-ATTR:hypervisor_hostname | None
|

| OS-EXT-SRV-ATTR:instance_name       |
|

| OS-EXT-STS:power_state              | NOSTATE
|

| OS-EXT-STS:task_state               | scheduling
|

| OS-EXT-STS:vm_state                 | building
|

| OS-SRV-USG:launched_at              | None
|

| OS-SRV-USG:terminated_at            | None
|

| accessIPv4                          |
|

| accessIPv6                          |
|

| addresses                           |
|

| adminPass                           | 8xLec7V69uvp
|

| config_drive                        |
|

| created                             | 2019-05-27T12:58:35Z
|

| flavor                              | m1.small
(31081c0f-2c10-40b2-b3f9-049ca660e12d) |

| hostId                              |
|

| id                                  | b6d0b128-31fe-4733-be17-182a0ebaec20
|

| image                               | cirros
(3ecd6177-ae54-42a2-8aa3-a442e96f7642)   |

| key_name                            | None
|

| name                                | qat2
|

| progress                            | 0
|

| project_id                          | 411d5846ca314075af13c7c496e44880
|

| properties                          |
|

| security_groups                     | name='default'
|

| status                              | BUILD
|

| updated                             | 2019-05-27T12:58:35Z
|

| user_id                             | 597d02e09d4d4a3b8172aeda82ab364d
|

| volumes_attached                    |
|

+-------------------------------------+-------------------------------------
------------+

controller-0:~# openstack server list

+--------------------------------------+------+--------+--------------------
--------+--------+----------+

| ID                                   | Name | Status | Networks
| Image  | Flavor   |

+--------------------------------------+------+--------+--------------------
--------+--------+----------+

| b6d0b128-31fe-4733-be17-182a0ebaec20 | qat2 | ACTIVE |
public-net0=192.168.101.20 | cirros | m1.small |

| 17eb84cf-0666-48c7-b049-cd9caae2358f | qat  | ERROR  |
| cirros | m1.small |

+--------------------------------------+------+--------+--------------------
--------+--------+----------+

 

 

While I¡¯m still hitting the same error:

 

controller-0:~# openstack server show qat

| fault                               | {u'message': u'No valid host was
found. There are not enough hosts available.', u'code': 500, u'details': u'
File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.
py", line 1326, in schedule_and_build_instances\n    instance_uuids,
return_alternates=True)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py",
line 780, in _schedule_instances\n    return_alternates=return_alternates)\n
File
"/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.
py", line 42, in select_destinations\n    instance_uuids, return_objects,
return_alternates)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py",
line 160, in select_destinations\n    return cctxt.call(ctxt,
\'select_destinations\', **msg_args)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py
", line 178, in call\n    retry=self.retry)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py"
, line 128, in _send\n    retry=retry)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp
driver.py", line 645, in send\n    call_monitor_timeout, retry=retry)\n
File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp
driver.py", line 636, in _send\n    raise result\n', u'created':
u'2019-05-27T12:57:43Z'} |

| flavor                              | m1.small
(31081c0f-2c10-40b2-b3f9-049ca660e12d)                       

 

 

Any other idea about what would be missing ? 

 

Thanks

-Ricardo

 

 

From: ÕÅÖ¾¹ú [ <mailto:zhangzhg at neusoft.com> mailto:zhangzhg at neusoft.com] 
Sent: Sunday, May 26, 2019 11:11 PM
To: Perez, Ricardo O < <mailto:ricardo.o.perez at intel.com>
ricardo.o.perez at intel.com>; Zhao, ShuaiX < <mailto:shuaix.zhao at intel.com>
shuaix.zhao at intel.com>;  <mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
Cc: Cabrales, Ada < <mailto:ada.cabrales at intel.com> ada.cabrales at intel.com>;
Xie, Cindy < <mailto:cindy.xie at intel.com> cindy.xie at intel.com>; Wang, Hai
Tao < <mailto:hai.tao.wang at intel.com> hai.tao.wang at intel.com>; Lin,
Shuicheng < <mailto:shuicheng.lin at intel.com> shuicheng.lin at intel.com>;
<mailto:fuyong at neusoft.com> fuyong at neusoft.com;  <mailto:lilong-neu at neusoft.
com> lilong-neu at neusoft.com; 'zhaos' < <mailto:zhaos at neusoft.com>
zhaos at neusoft.com>;  <mailto:sy at neusoft.com> sy at neusoft.com;
<mailto:zuoyl at neusoft.com> zuoyl at neusoft.com
Subject: ´ð¸´: StarlingX QAT validation progress

 

Hi   Ricardo:

 

The cause of the problem may be as follows:

The QAT driver disables the PF function when IOMMU is turned on. Only the VF
function can be used. StarlingX has enabled the IOMMU function.

Please try to change to the following format.

 

 

conf:

  nova:

    pci:

      alias:

        type: multistring

        values:

        - '{"vendor_id": "8086", "product_id": "37c9", "name":
"qat-c62x-vf", "device_type": "type-VF"}'

  overrides:

    nova_compute:

      hosts:

        - conf:

            nova:

              DEFAULT:

                my_ip: 192.168.206.3

                shared_pcpu_map: '""'

                vcpu_pin_set: '"2-35,38-71"'

              libvirt:

                images_type: default

                live_migration_inbound_addr: 192.168.206.3

              pci:

                passthrough_whitelist:

                  type: multistring

                  values:

                    - '{"vendor_id": "8086", "product_id":"37c9", "address":
"0000:1a:01.0"}'

                    - '{"vendor_id": "8086", "product_id":"37c9", "address":
"0000:1c:01.0"}'

                    - '{"vendor_id": "8086", "product_id":"37c9", "address":
"0000:1e:01.0"}'

              vnc:

                vncserver_listen: 0.0.0.0

                vncserver_proxyclient_address: 192.168.206.3

          name: controller-0

 

 

Best Regards.

 

·¢¼þÈË: Perez, Ricardo O [ <mailto:ricardo.o.perez at intel.com>
mailto:ricardo.o.perez at intel.com] 
·¢ËÍʱ¼ä: 2019Äê5ÔÂ27ÈÕ 11:28
ÊÕ¼þÈË: Zhao, ShuaiX < <mailto:shuaix.zhao at intel.com>
shuaix.zhao at intel.com>;  <mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
³­ËÍ: Cabrales, Ada < <mailto:ada.cabrales at intel.com>
ada.cabrales at intel.com>; Xie, Cindy < <mailto:cindy.xie at intel.com>
cindy.xie at intel.com>; Wang, Hai Tao < <mailto:hai.tao.wang at intel.com>
hai.tao.wang at intel.com>; Lin, Shuicheng < <mailto:shuicheng.lin at intel.com>
shuicheng.lin at intel.com>; 'fuyong at neusoft.com' < <mailto:fuyong at neusoft.com>
fuyong at neusoft.com>; 'zhangzhg at neusoft.com' < <mailto:zhangzhg at neusoft.com>
zhangzhg at neusoft.com>; 'lilong-neu at neusoft.com' <
<mailto:lilong-neu at neusoft.com> lilong-neu at neusoft.com>; 'zhaos' <
<mailto:zhaos at neusoft.com> zhaos at neusoft.com>; 'sy at neusoft.com' <
<mailto:sy at neusoft.com> sy at neusoft.com>; 'zuoyl at neusoft.com' <
<mailto:zuoyl at neusoft.com> zuoyl at neusoft.com>
Ö÷Ìâ: RE: StarlingX QAT validation progress

 

Hello team, 

 

Thanks for sharing the information about Passthrough. :)

 

I have some questions regarding this. 

 

Currently I have the following QAT device in our infrastructure servers:

 

[wrsroot at controller-0 ~(keystone_admin)]$ system host-device-list compute-0

+------------+--------------+----------+-----------+-----------+------------
----------+-------------------------+---------------------------------

| name       | address      | class id | vendor id | device id | class name
| vendor name             | device name

+------------+--------------+----------+-----------+-----------+------------
----------+-------------------------+---------------------------------

| pci_0000_0 | 0000:02:00.0 | 030000   | 1a03      | 2000      | VGA
compatible       | ASPEED Technology, Inc. | ASPEED Graphics Family

| 2_00_0     |              |          |           |           | controller
|                         |

|            |              |          |           |           |
|                         |

| pci_0000_1 | 0000:1a:00.0 | 0b4000   | 8086      | 37c8      |
Co-processor         | Intel Corporation       | C62x Chipset QuickAssist
Technol

| a_00_0     |              |          |           |           |
|                         |

|            |              |          |           |           |
|                         |

| pci_0000_1 | 0000:1c:00.0 | 0b4000   | 8086      | 37c8      |
Co-processor         | Intel Corporation       | C62x Chipset QuickAssist
Technol

| c_00_0     |              |          |           |           |
|                         |

|            |              |          |           |           |
|                         |

| pci_0000_1 | 0000:1e:00.0 | 0b4000   | 8086      | 37c8      |
Co-processor         | Intel Corporation       | C62x Chipset QuickAssist
Technol

| e_00_0     |              |          |           |           |
|                         |

|            |              |          |           |           |
|                         |

+------------+--------------+----------+-----------+-----------+------------
----------+-------------------------+---------------------------------

[wrsroot at controller-0 ~(keystone_admin)]$ system host-device-list compute-1

+------------+--------------+----------+-----------+-----------+------------
----------+-------------------------+---------------------------------

| name       | address      | class id | vendor id | device id | class name
| vendor name             | device name

+------------+--------------+----------+-----------+-----------+------------
----------+-------------------------+---------------------------------

| pci_0000_0 | 0000:02:00.0 | 030000   | 1a03      | 2000      | VGA
compatible       | ASPEED Technology, Inc. | ASPEED Graphics Family

| 2_00_0     |              |          |           |           | controller
|                         |

|            |              |          |           |           |
|                         |

| pci_0000_1 | 0000:1a:00.0 | 0b4000   | 8086      | 37c8      |
Co-processor         | Intel Corporation       | C62x Chipset QuickAssist
Technol

| a_00_0     |              |          |           |           |
|                         |

|            |              |          |           |           |
|                         |

| pci_0000_1 | 0000:1c:00.0 | 0b4000   | 8086      | 37c8      |
Co-processor         | Intel Corporation       | C62x Chipset QuickAssist
Technol

| c_00_0     |              |          |           |           |
|                         |

|            |              |          |           |           |
|                         |

| pci_0000_1 | 0000:1e:00.0 | 0b4000   | 8086      | 37c8      |
Co-processor         | Intel Corporation       | C62x Chipset QuickAssist
Technol

| e_00_0     |              |          |           |           |
|                         |

|            |              |          |           |           |
|                         |

+------------+--------------+----------+-----------+-----------+------------
----------+-------------------------+---------------------------------

 

And these are the device listed:

 

compute-1:~$ sudo service qat_service status

Password:

Checking status of all devices.

There is 51 QAT acceleration device(s) in the system:

qat_dev0 - type: c6xx,  inst_id: 0,  node_id: 0,  bsf: 0000:1a:00.0,
#accel: 5 #engines: 10 state: up

qat_dev1 - type: c6xx,  inst_id: 1,  node_id: 0,  bsf: 0000:1c:00.0,
#accel: 5 #engines: 10 state: up

qat_dev2 - type: c6xx,  inst_id: 2,  node_id: 0,  bsf: 0000:1e:00.0,
#accel: 5 #engines: 10 state: up

qat_dev3 - type: c6xxvf,  inst_id: 0,  node_id: 0,  bsf: 0000:1a:01.0,
#accel: 1 #engines: 1 state: up

qat_dev4 - type: c6xxvf,  inst_id: 1,  node_id: 0,  bsf: 0000:1a:01.1,
#accel: 1 #engines: 1 state: up

qat_dev5 - type: c6xxvf,  inst_id: 2,  node_id: 0,  bsf: 0000:1a:01.2,
#accel: 1 #engines: 1 state: up

qat_dev6 - type: c6xxvf,  inst_id: 3,  node_id: 0,  bsf: 0000:1a:01.3,
#accel: 1 #engines: 1 state: up

qat_dev7 - type: c6xxvf,  inst_id: 4,  node_id: 0,  bsf: 0000:1a:01.4,
#accel: 1 #engines: 1 state: up

qat_dev8 - type: c6xxvf,  inst_id: 5,  node_id: 0,  bsf: 0000:1a:01.5,
#accel: 1 #engines: 1 state: up

qat_dev9 - type: c6xxvf,  inst_id: 6,  node_id: 0,  bsf: 0000:1a:01.6,
#accel: 1 #engines: 1 state: up

qat_dev10 - type: c6xxvf,  inst_id: 7,  node_id: 0,  bsf: 0000:1a:01.7,
#accel: 1 #engines: 1 state: up

qat_dev11 - type: c6xxvf,  inst_id: 8,  node_id: 0,  bsf: 0000:1a:02.0,
#accel: 1 #engines: 1 state: up

qat_dev12 - type: c6xxvf,  inst_id: 9,  node_id: 0,  bsf: 0000:1a:02.1,
#accel: 1 #engines: 1 state: up

qat_dev13 - type: c6xxvf,  inst_id: 10,  node_id: 0,  bsf: 0000:1a:02.2,
#accel: 1 #engines: 1 state: up

qat_dev14 - type: c6xxvf,  inst_id: 11,  node_id: 0,  bsf: 0000:1a:02.3,
#accel: 1 #engines: 1 state: up

qat_dev15 - type: c6xxvf,  inst_id: 12,  node_id: 0,  bsf: 0000:1a:02.4,
#accel: 1 #engines: 1 state: up

qat_dev16 - type: c6xxvf,  inst_id: 13,  node_id: 0,  bsf: 0000:1a:02.5,
#accel: 1 #engines: 1 state: up

qat_dev17 - type: c6xxvf,  inst_id: 14,  node_id: 0,  bsf: 0000:1a:02.6,
#accel: 1 #engines: 1 state: up

qat_dev18 - type: c6xxvf,  inst_id: 15,  node_id: 0,  bsf: 0000:1a:02.7,
#accel: 1 #engines: 1 state: up

qat_dev19 - type: c6xxvf,  inst_id: 16,  node_id: 0,  bsf: 0000:1c:01.0,
#accel: 1 #engines: 1 state: up

qat_dev20 - type: c6xxvf,  inst_id: 17,  node_id: 0,  bsf: 0000:1c:01.1,
#accel: 1 #engines: 1 state: up

qat_dev21 - type: c6xxvf,  inst_id: 18,  node_id: 0,  bsf: 0000:1c:01.2,
#accel: 1 #engines: 1 state: up

qat_dev22 - type: c6xxvf,  inst_id: 19,  node_id: 0,  bsf: 0000:1c:01.3,
#accel: 1 #engines: 1 state: up

qat_dev23 - type: c6xxvf,  inst_id: 20,  node_id: 0,  bsf: 0000:1c:01.4,
#accel: 1 #engines: 1 state: up

qat_dev24 - type: c6xxvf,  inst_id: 21,  node_id: 0,  bsf: 0000:1c:01.5,
#accel: 1 #engines: 1 state: up

qat_dev25 - type: c6xxvf,  inst_id: 22,  node_id: 0,  bsf: 0000:1c:01.6,
#accel: 1 #engines: 1 state: up

qat_dev26 - type: c6xxvf,  inst_id: 23,  node_id: 0,  bsf: 0000:1c:01.7,
#accel: 1 #engines: 1 state: up

qat_dev27 - type: c6xxvf,  inst_id: 24,  node_id: 0,  bsf: 0000:1c:02.0,
#accel: 1 #engines: 1 state: up

qat_dev28 - type: c6xxvf,  inst_id: 25,  node_id: 0,  bsf: 0000:1c:02.1,
#accel: 1 #engines: 1 state: up

qat_dev29 - type: c6xxvf,  inst_id: 26,  node_id: 0,  bsf: 0000:1c:02.2,
#accel: 1 #engines: 1 state: up

qat_dev30 - type: c6xxvf,  inst_id: 27,  node_id: 0,  bsf: 0000:1c:02.3,
#accel: 1 #engines: 1 state: up

qat_dev31 - type: c6xxvf,  inst_id: 28,  node_id: 0,  bsf: 0000:1c:02.4,
#accel: 1 #engines: 1 state: up

qat_dev32 - type: c6xxvf,  inst_id: 29,  node_id: 0,  bsf: 0000:1c:02.5,
#accel: 1 #engines: 1 state: up

qat_dev33 - type: c6xxvf,  inst_id: 30,  node_id: 0,  bsf: 0000:1c:02.6,
#accel: 1 #engines: 1 state: up

qat_dev34 - type: c6xxvf,  inst_id: 31,  node_id: 0,  bsf: 0000:1c:02.7,
#accel: 1 #engines: 1 state: up

qat_dev35 - type: c6xxvf,  inst_id: 32,  node_id: 0,  bsf: 0000:1e:01.0,
#accel: 1 #engines: 1 state: up

qat_dev36 - type: c6xxvf,  inst_id: 33,  node_id: 0,  bsf: 0000:1e:01.1,
#accel: 1 #engines: 1 state: up

qat_dev37 - type: c6xxvf,  inst_id: 34,  node_id: 0,  bsf: 0000:1e:01.2,
#accel: 1 #engines: 1 state: up

qat_dev38 - type: c6xxvf,  inst_id: 35,  node_id: 0,  bsf: 0000:1e:01.3,
#accel: 1 #engines: 1 state: up

qat_dev39 - type: c6xxvf,  inst_id: 36,  node_id: 0,  bsf: 0000:1e:01.4,
#accel: 1 #engines: 1 state: up

qat_dev40 - type: c6xxvf,  inst_id: 37,  node_id: 0,  bsf: 0000:1e:01.5,
#accel: 1 #engines: 1 state: up

qat_dev41 - type: c6xxvf,  inst_id: 38,  node_id: 0,  bsf: 0000:1e:01.6,
#accel: 1 #engines: 1 state: up

qat_dev42 - type: c6xxvf,  inst_id: 39,  node_id: 0,  bsf: 0000:1e:01.7,
#accel: 1 #engines: 1 state: up

qat_dev43 - type: c6xxvf,  inst_id: 40,  node_id: 0,  bsf: 0000:1e:02.0,
#accel: 1 #engines: 1 state: up

qat_dev44 - type: c6xxvf,  inst_id: 41,  node_id: 0,  bsf: 0000:1e:02.1,
#accel: 1 #engines: 1 state: up

qat_dev45 - type: c6xxvf,  inst_id: 42,  node_id: 0,  bsf: 0000:1e:02.2,
#accel: 1 #engines: 1 state: up

qat_dev46 - type: c6xxvf,  inst_id: 43,  node_id: 0,  bsf: 0000:1e:02.3,
#accel: 1 #engines: 1 state: up

qat_dev47 - type: c6xxvf,  inst_id: 44,  node_id: 0,  bsf: 0000:1e:02.4,
#accel: 1 #engines: 1 state: up

qat_dev48 - type: c6xxvf,  inst_id: 45,  node_id: 0,  bsf: 0000:1e:02.5,
#accel: 1 #engines: 1 state: up

qat_dev49 - type: c6xxvf,  inst_id: 46,  node_id: 0,  bsf: 0000:1e:02.6,
#accel: 1 #engines: 1 state: up

qat_dev50 - type: c6xxvf,  inst_id: 47,  node_id: 0,  bsf: 0000:1e:02.7,
#accel: 1 #engines: 1 state: up

 

However , following the steps to add the property to the flavor, and then,
launching the VM, seems like there is no success on that (on launching the
VM with the flavor that has been modified with ¡°pci_passthrouh¡± property).
See below:

 

controller-0:~# openstack flavor set m1.small --property
"pci_passthrough:alias"="qat-c62x-vf:1"

 

controller-0:~# openstack server create --image cirros --flavor m1.small
--network public-net0 qat-vm

+-------------------------------------+-------------------------------------
------------+

| Field                               | Value
|

+-------------------------------------+-------------------------------------
------------+

| OS-DCF:diskConfig                   | MANUAL
|

| OS-EXT-AZ:availability_zone         |
|

| OS-EXT-SRV-ATTR:host                | None
|

| OS-EXT-SRV-ATTR:hypervisor_hostname | None
|

| OS-EXT-SRV-ATTR:instance_name       |
|

| OS-EXT-STS:power_state              | NOSTATE
|

| OS-EXT-STS:task_state               | scheduling
|

| OS-EXT-STS:vm_state                 | building
|

| OS-SRV-USG:launched_at              | None
|

| OS-SRV-USG:terminated_at            | None
|

| accessIPv4                          |
|

| accessIPv6                          |
|

| addresses                           |
|

| adminPass                           | HoRe6J97skj2
|

| config_drive                        |
|

| created                             | 2019-05-23T17:37:20Z
|

| flavor                              | m1.small
(31081c0f-2c10-40b2-b3f9-049ca660e12d) |

| hostId                              |
|

| id                                  | 7fd458f6-0584-4533-a30e-7d6e758b681b
|

| image                               | cirros
(3ecd6177-ae54-42a2-8aa3-a442e96f7642)   |

| key_name                            | None
|

| name                                | qat-vm
|

| progress                            | 0
|

| project_id                          | 411d5846ca314075af13c7c496e44880
|

| properties                          |
|

| security_groups                     | name='default'
|

| status                              | BUILD
|

| updated                             | 2019-05-23T17:37:20Z
|

| user_id                             | 597d02e09d4d4a3b8172aeda82ab364d
|

| volumes_attached                    |
|

+-------------------------------------+-------------------------------------
------------+

controller-0:~# openstack server list

+--------------------------------------+--------+--------+----------+-------
-+----------+

| ID                                   | Name   | Status | Networks | Image
| Flavor   |

+--------------------------------------+--------+--------+----------+-------
-+----------+

| 7fd458f6-0584-4533-a30e-7d6e758b681b | qat-vm | ERROR  |          | cirros
| m1.small |

+--------------------------------------+--------+--------+----------+-------
-+----------+

 

 

| fault                               | {u'message': u'No valid host was
found. There are not enough hosts available.', u'code': 500, u'details':
-packages/nova/conductor/manager.py", line 1326, in
schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n
File "/var/lib/optor/manager.py", line 780, in _schedule_instances\n
return_alternates=return_alternates)\n  File
"/var/lib/openstack/lib/python2.7/site-packageselect_destinations\n
instance_uuids, return_objects, return_alternates)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler    return
cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/clientn
File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py"
, line 128, in _send\n    retry=retry)\n  File
"/var/lib/opens/_drivers/amqpdriver.py", line 645, in send\n
call_monitor_timeout, retry=retry)\n  File
"/var/lib/openstack/lib/python2.7/site-packages/oslo_m_send\n    raise
result\n', u'created': u'2019-05-23T17:37:20Z'} |

| flavor                              | m1.small
(31081c0f-2c10-40b2-b3f9-049ca660e12d)  

 

The interesting thing is that if I launch a VM removing or with a flavor
that doesn¡¯t contains the property ¡°pci_passthrough¡± is launched
successfully:

 

controller-0:~# openstack server create --image cirros --flavor m1.tiny
--network public-net0 qat-vm2

+-------------------------------------+-------------------------------------
-----------+

| Field                               | Value
|

+-------------------------------------+-------------------------------------
-----------+

| OS-DCF:diskConfig                   | MANUAL
|

| OS-EXT-AZ:availability_zone         |
|

| OS-EXT-SRV-ATTR:host                | None
|

| OS-EXT-SRV-ATTR:hypervisor_hostname | None
|

| OS-EXT-SRV-ATTR:instance_name       |
|

| OS-EXT-STS:power_state              | NOSTATE
|

| OS-EXT-STS:task_state               | scheduling
|

| OS-EXT-STS:vm_state                 | building
|

| OS-SRV-USG:launched_at              | None
|

| OS-SRV-USG:terminated_at            | None
|

| accessIPv4                          |
|

| accessIPv6                          |
|

| addresses                           |
|

| adminPass                           | L4B92gGvRgjG
|

| config_drive                        |
|

| created                             | 2019-05-23T17:47:26Z
|

| flavor                              | m1.tiny
(34caaa34-0475-4f5a-99ce-26a69adda91d) |

| hostId                              |
|

| id                                  | 218282da-d168-45d8-940a-62ca83c0c170
|

| image                               | cirros
(3ecd6177-ae54-42a2-8aa3-a442e96f7642)  |

| key_name                            | None
|

| name                                | qat-vm2
|

| progress                            | 0
|

| project_id                          | 411d5846ca314075af13c7c496e44880
|

| properties                          |
|

| security_groups                     | name='default'
|

| status                              | BUILD
|

| updated                             | 2019-05-23T17:47:26Z
|

| user_id                             | 597d02e09d4d4a3b8172aeda82ab364d
|

| volumes_attached                    |
|

+-------------------------------------+-------------------------------------
-----------+

controller-0:~# openstack server show qat-vm2

+-------------------------------------+-------------------------------------
---------------------+

| Field                               | Value
|

+-------------------------------------+-------------------------------------
---------------------+

| OS-DCF:diskConfig                   | MANUAL
|

| OS-EXT-AZ:availability_zone         | nova
|

| OS-EXT-SRV-ATTR:host                | compute-1
|

| OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1
|

| OS-EXT-SRV-ATTR:instance_name       | instance-00000004
|

| OS-EXT-STS:power_state              | Running
|

| OS-EXT-STS:task_state               | None
|

| OS-EXT-STS:vm_state                 | active
|

| OS-SRV-USG:launched_at              | 2019-05-23T17:47:31.000000
|

| OS-SRV-USG:terminated_at            | None
|

| accessIPv4                          |
|

| accessIPv6                          |
|

| addresses                           | public-net0=192.168.101.194
|

| config_drive                        |
|

| created                             | 2019-05-23T17:47:26Z
|

| flavor                              | m1.tiny
(34caaa34-0475-4f5a-99ce-26a69adda91d)           |

| hostId                              |
9380803bfe0947a9094e2b7deb8044e5b40cb3761cd4ef63f0c21534 |

| id                                  | 218282da-d168-45d8-940a-62ca83c0c170
|

| image                               | cirros
(3ecd6177-ae54-42a2-8aa3-a442e96f7642)            |

| key_name                            | None
|

| name                                | qat-vm2
|

| progress                            | 0
|

| project_id                          | 411d5846ca314075af13c7c496e44880
|

| properties                          |
|

| security_groups                     | name='default'
|

| status                              | ACTIVE
|

| updated                             | 2019-05-23T17:47:31Z
|

| user_id                             | 597d02e09d4d4a3b8172aeda82ab364d
|

| volumes_attached                    |
|

+-------------------------------------+-------------------------------------
---------------------+

controller-0:~# openstack server list

+--------------------------------------+---------+--------+-----------------
------------+--------+----------+

| ID                                   | Name    | Status | Networks
| Image  | Flavor   |

+--------------------------------------+---------+--------+-----------------
------------+--------+----------+

| 218282da-d168-45d8-940a-62ca83c0c170 | qat-vm2 | ACTIVE | public-net0=192.
168.101.194 | cirros | m1.tiny  |

| 7fd458f6-0584-4533-a30e-7d6e758b681b | qat-vm  | ERROR  |
| cirros | m1.small |

+--------------------------------------+---------+--------+-----------------
------------+--------+----------+


 

 

This is the YAML file, could you please guys take a look and see if
something is missing ?

 

conf:

  nova:

    pci:

      alias:

        type: multistring

        values:

        - '{"vendor_id": "8086", "product_id": "37c8", "name":
"qat-c62x-pf", "device_type": "type-PF"}'

  overrides:

    nova_compute:

      hosts:

        - conf:

            nova:

              DEFAULT:

                my_ip: 192.168.206.3

                shared_pcpu_map: '""'

                vcpu_pin_set: '"2-35,38-71"'

              libvirt:

                images_type: default

                live_migration_inbound_addr: 192.168.206.3

              pci:

                passthrough_whitelist:

                  type: multistring

                  values:

                    - '{"vendor_id": "8086", "product_id":"37c8", "address":
"0000:1a:00.0"}'

                    - '{"vendor_id": "8086", "product_id":"37c8", "address":
"0000:1c:00.0"}'

                    - '{"vendor_id": "8086", "product_id":"37c8", "address":
"0000:1e:00.0"}'

              vnc:

                vncserver_listen: 0.0.0.0

                vncserver_proxyclient_address: 192.168.206.3

          name: controller-0

 

I¡¯m using a 2+2 configuration with the following ISO version:

 

controller-0:~$ cat /etc/build.info

###

### StarlingX

###     Built from master

###

 

OS="centos"

SW_VERSION="19.01"

BUILD_TARGET="Host Installer"

BUILD_TYPE="Formal"

BUILD_ID="20190522T013000Z"

 

JOB="STX_build_master_master"

BUILD_BY="starlingx.build at cengn.ca <mailto:starlingx.build at cengn.ca> "

BUILD_NUMBER="112"

BUILD_HOST="starlingx_mirror"

BUILD_DATE="2019-05-22 01:30:00 +0000"

 

Thanks in advance

-Ricardo

 

From: Zhao, ShuaiX 
Sent: Thursday, May 23, 2019 7:41 AM
To: Perez, Ricardo O < <mailto:ricardo.o.perez at intel.com>
ricardo.o.perez at intel.com>;  <mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
Cc: Cabrales, Ada < <mailto:ada.cabrales at intel.com> ada.cabrales at intel.com>;
Xie, Cindy < <mailto:cindy.xie at intel.com> cindy.xie at intel.com>; Wang, Hai
Tao < <mailto:hai.tao.wang at intel.com> hai.tao.wang at intel.com>; Lin,
Shuicheng < <mailto:shuicheng.lin at intel.com> shuicheng.lin at intel.com>;
'fuyong at neusoft.com' < <mailto:fuyong at neusoft.com> fuyong at neusoft.com>;
'zhangzhg at neusoft.com' < <mailto:zhangzhg at neusoft.com>
zhangzhg at neusoft.com>; 'lilong-neu at neusoft.com' <
<mailto:lilong-neu at neusoft.com> lilong-neu at neusoft.com>; 'zhaos' <
<mailto:zhaos at neusoft.com> zhaos at neusoft.com>; Zhao, ShuaiX <
<mailto:shuaix.zhao at intel.com> shuaix.zhao at intel.com>; 'sy at neusoft.com' <
<mailto:sy at neusoft.com> sy at neusoft.com>; 'zuoyl at neusoft.com' <
<mailto:zuoyl at neusoft.com> zuoyl at neusoft.com>
Subject: RE: StarlingX QAT validation progress

 

Hi StarlingX Team :

               

               Regarding upgrade qat driver to latest version for CentOS
7.6. After the successful QAT transplant, 

we found that the current STX Temporarily unable to achieve perfect device
passthrough operation.

Through continuous trials, we have found a way to achieve device passthrough
method. 

we hope to be able to help with the current device Passthrough test task..

Currently updated in Ricardo's documentation. Welcome your review.

 

Thx. ¨C Neusoft Dalian ODC Intel Team

 

From: Xie, Cindy 
Sent: Thursday, May 23, 2019 7:24 AM
To: Perez, Ricardo O < <mailto:ricardo.o.perez at intel.com>
ricardo.o.perez at intel.com>;  <mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
Cc: Cabrales, Ada < <mailto:ada.cabrales at intel.com> ada.cabrales at intel.com>;
Zhao, ShuaiX < <mailto:shuaix.zhao at intel.com> shuaix.zhao at intel.com>
Subject: RE: StarlingX QAT validation progress

 

Thanks Ricardo. Haitao and Shuai will follow up with you for the progress
we¡¯ve made from China side.

 

Thx. - cindy

 

From: Perez, Ricardo O 
Sent: Thursday, May 23, 2019 7:07 AM
To:  <mailto:starlingx-discuss at lists.starlingx.io>
starlingx-discuss at lists.starlingx.io
Cc: Xie, Cindy < <mailto:cindy.xie at intel.com> cindy.xie at intel.com>;
Cabrales, Ada < <mailto:ada.cabrales at intel.com> ada.cabrales at intel.com>
Subject: StarlingX QAT validation progress

 

Hi super StarlingX team,

 

In the following document, you can find the progress about QAT + StarlingX
validation in a 2+2 system.

 

Overall, some blockers that we have been facing to get more progress is the
stability of the latest ISOs. However, we have at least executed 4
successfully test cases:

 


QAT_StarlingX_001

Verify that QAT PCIe module doesn't generate any StarlingX installation
issue


QAT_StarlingX_002

Verify that QAT module can be listed via lsmod


QAT_StarlingX_003

Verify that QAT driver / service is up and running 


QAT_StarlingX_004

Verify newly added system inventory commands to support crypto device

 

For all the details about the remaining test cases, expected coverage, and
results, please take a look here: 

 

https://docs.google.com/spreadsheets/d/1TEjHB5JCYKFUwf2TM4D23OIA-9huYhGZrokZ
ABJP2Lg/edit#gid=118299536

 

Also feel free to add any comment / input related to such tests that you
might consider important to improve them.

 

Thanks in advance

-Ricardo (a.k.a. Richo)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190528/150a9546/attachment-0002.html>
-------------- next part --------------


---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing,  storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190528/150a9546/attachment-0003.html>


More information about the Starlingx-discuss mailing list