Re: [Starlingx-discuss] <StarlingX-Discuss> StarlingX 2018.10 without DPDK query
Hi Brent, Thanks for looking into the issue. Yes these logs are with RAID 0. But same was the behaviour with RAID 1 Thanks Anirudh Gupta From: Rowsell, Brent Sent: Sunday, 23 June, 10:36 PM Subject: RE: [Starlingx-discuss] <StarlingX-Discuss> StarlingX 2018.10without DPDK query To: Anirudh Gupta, Zhao, Forrest, Xie, Cindy, Jones, Bruce E, Winnicki, Chris, Ildiko Vancsa, Miller, Frank, Arce Moreno, Abraham, Hernandez Gonzalez, Fernando, yong.hu@intel.com, Khalil, Ghada Cc: starlingx-discuss@lists.starlingx.io, starlingx-announce@lists.starlingx.io In the kern logs I see I/O errors. This suggests either a h/w issue (disk or controller) or a device driver issue. The logs provided are with RAID0 ? Brent From: Anirudh Gupta [mailto:Anirudh.Gupta@hsc.com] Sent: Thursday, June 20, 2019 9:00 PM To: Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Winnicki, Chris <Chris.Winnicki@windriver.com>; Ildiko Vancsa <ildiko.vancsa@gmail.com>; Miller, Frank <Frank.Miller@windriver.com>; Arce Moreno, Abraham <abraham.arce.moreno@intel.com>; Hernandez Gonzalez, Fernando <fernando.hernandez.gonzalez@intel.com>; yong.hu@intel.com; Khalil, Ghada <Ghada.Khalil@windriver.com> Cc: starlingx-discuss@lists.starlingx.io; starlingx-announce@lists.starlingx.io Subject: Re: [Starlingx-discuss] <StarlingX-Discuss> StarlingX 2018.10 without DPDK query Hi Team, Can someone please provide any pointer to resolve the issue. Regards Anirudh Gupta From: Anirudh Gupta Sent: Thursday, June 20, 2019 6:55:15 PM To: Zhao, Forrest; Xie, Cindy; Jones, Bruce E; Winnicki, Chris; Ildiko Vancsa; Miller, Frank; Arce Moreno, Abraham; Hernandez Gonzalez, Fernando; yong.hu@intel.com<mailto:yong.hu@intel.com>; Khalil, Ghada Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: RE: <StarlingX-Discuss> StarlingX 2018.10 without DPDK query Hi Cindy, Yong, Ghada & Team, I need to create AIO 2018.10 Simplex Bare Metal Setup. As discussed in the mail chain, I have arranged a server with DPDK NIC and tried installing the ISO downloaded from the link http://mirror.starlingx.cengn.ca/mirror/starlingx/centos/2018.10/20181110/outputs/iso/<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmirror.starlingx.cengn.ca%2Fmirror%2Fstarlingx%2Fcentos%2F2018.10%2F20181110%2Foutputs%2Fiso%2F&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608130186&sdata=OIvjoGcJchI2%2BEk63sxEggK4IcsJ3TyZRKMZLbbMIck%3D&reserved=0> The server configuration is as below: CPU core – 16 Hard disk – 2 Hardisk each of 930GB RAM – 64 GB DPDK supported NIC – Yes controller-0:~$ lspci | grep -i ethernet 03:00.0 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T 03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T 06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 06:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) When I ran the config_controller command, it throws me an error in 02/08 Step : Applying Bootstrap Manifest” EXT4-fs error (device drb0): ext4_journal_check_start:56: Detected aborted journal EXT4-fs error(dbr0): Remounting filesystem read-only The completed screenshot of the error is attached in the mail. I also tried following the below mailing list http://lists.starlingx.io/pipermail/starlingx-discuss/2019-February/003140.html<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.starlingx.io%2Fpipermail%2Fstarlingx-discuss%2F2019-February%2F003140.html&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608130186&sdata=E5UAG6%2FYEZILYyAVzkxHtRyGqU8A2bTYk04RMplYdr8%3D&reserved=0> There are 2 Hard-disk attached in the server each of 930 GB I had tried configured my server in RAID 0 and RAID 1, but getting the same error of running the config_controller command in the step 2. I am also attaching the complete logs of path /var/log/ for your reference. Please help me in resolving the issue. Regards Anirudh Gupta (Senior Engineer) From: Khalil, Ghada <Ghada.Khalil@windriver.com<mailto:Ghada.Khalil@windriver.com>> Sent: 19 June 2019 06:07 To: Anirudh Gupta <Anirudh.Gupta@hsc.com<mailto:Anirudh.Gupta@hsc.com>>; Zhao, Forrest <forrest.zhao@intel.com<mailto:forrest.zhao@intel.com>>; Xie, Cindy <cindy.xie@intel.com<mailto:cindy.xie@intel.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Winnicki, Chris <Chris.Winnicki@windriver.com<mailto:Chris.Winnicki@windriver.com>>; Ildiko Vancsa <ildiko.vancsa@gmail.com<mailto:ildiko.vancsa@gmail.com>>; Miller, Frank <Frank.Miller@windriver.com<mailto:Frank.Miller@windriver.com>>; Arce Moreno, Abraham <abraham.arce.moreno@intel.com<mailto:abraham.arce.moreno@intel.com>>; Hazzim Anaya Casas? <hazzim.i.anaya.casas@intel.com<mailto:hazzim.i.anaya.casas@intel.com>>; Hernandez Gonzalez, Fernando <fernando.hernandez.gonzalez@intel.com<mailto:fernando.hernandez.gonzalez@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: RE: <StarlingX-Discuss> StarlingX 2018.10 without DPDK query I’m not aware of a way to allow the use of ovs (w/o dpdk) in 2018.10 To use OVS, you will need to use a recent load built from master and follow the updated deployment instructions: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmirror.starlingx.cengn.ca%2Fmirror%2Fstarlingx%2Fmaster%2Fcentos%2F&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608140181&sdata=gncumgfgoYSMYYm0K3OZY5wfoKMxce3FXdcSNb1eB98%3D&reserved=0> The last green sanity was using ISO 20190613. You can use the symlink: latest_green_build or monitor the sanity emails sent regularly to the mailing list. Regards, Ghada From: Anirudh Gupta [mailto:Anirudh.Gupta@hsc.com] Sent: Tuesday, June 18, 2019 2:14 PM To: Zhao, Forrest; Xie, Cindy; Jones, Bruce E; Winnicki, Chris; Ildiko Vancsa; Khalil, Ghada; Miller, Frank; Arce Moreno, Abraham; Hazzim Anaya Casas?; Hernandez Gonzalez, Fernando Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: Re: <StarlingX-Discuss> StarlingX 2018.10 without DPDK query Hi Forrest & Team I agree that there is no such instruction mentioned in the document for 2018.10 AIO Simplex Mode for OVS. It's just that my system doesn't have DPDK support and I need to deploy 2018.10 release until Release 2.0 is released in August 2019 as per the plan. So Can I setup STARLINGX 2018.10 AIO SIMPLEX in my bare metal without DPDK support? Is there any flag that could be disabled or any workaround that can work for me? Looking Forward for your response. Regards Anirudh Gupta From: Zhao, Forrest Sent: Tuesday, 18 June, 7:28 PM Subject: RE: <StarlingX-Discuss> StarlingX 2018.10 without DPDK query To: Xie, Cindy, Anirudh Gupta, Jones, Bruce E, Winnicki, Chris, Ildiko Vancsa, Ghada Khalil?, Jones, Bruce E, Frank Miller, Arce Moreno, Abraham, Hazzim Anaya Casas?, Hernandez Gonzalez, Fernando Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>, starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> If I remember correctly, 2018.10 release only support OVS-DPDK as virtual switch. Ghada may be able to double confirm that. Also I can’t find any instruction in 2018.10 AIO simplex deployment guide https://docs.starlingx.io/deployment_guides/current/simplex.html<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.starlingx.io%2Fdeployment_guides%2Fcurrent%2Fsimplex.html&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608140181&sdata=oORRjugvMta0gNvEGBSBxIT%2BaFco%2BeieDjlnMdAQ%2BFs%3D&reserved=0> to set the virtual switch to OVS. From: Xie, Cindy Sent: Tuesday, June 18, 2019 8:35 PM To: Anirudh Gupta <Anirudh.Gupta@hsc.com<mailto:Anirudh.Gupta@hsc.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Winnicki, Chris <Chris.Winnicki@windriver.com<mailto:Chris.Winnicki@windriver.com>>; Ildiko Vancsa <ildiko.vancsa@gmail.com<mailto:ildiko.vancsa@gmail.com>>; Ghada Khalil? <Ghada.Khalil@windriver.com<mailto:Ghada.Khalil@windriver.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Frank Miller <frank.miller@windriver.com<mailto:frank.miller@windriver.com>>; Arce Moreno, Abraham <abraham.arce.moreno@intel.com<mailto:abraham.arce.moreno@intel.com>>; Hazzim Anaya Casas? <hazzim.i.anaya.casas@intel.com<mailto:hazzim.i.anaya.casas@intel.com>>; Hernandez Gonzalez, Fernando <fernando.hernandez.gonzalez@intel.com<mailto:fernando.hernandez.gonzalez@intel.com>>; Zhao, Forrest <forrest.zhao@intel.com<mailto:forrest.zhao@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: RE: <StarlingX-Discuss> StarlingX 2018.10 without DPDK query + Forrest, do you have good answer to Gupta about using OVS without DPDK for 2018.10 release? I understand that we have the containerized OVS option without DPDK with vswitch type to “none” in stx.2.0 today. Not sure about the behavior in 2018.10 release. From: Anirudh Gupta [mailto:Anirudh.Gupta@hsc.com] Sent: Tuesday, June 18, 2019 7:22 PM To: Xie, Cindy <cindy.xie@intel.com<mailto:cindy.xie@intel.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Winnicki, Chris <Chris.Winnicki@windriver.com<mailto:Chris.Winnicki@windriver.com>>; Ildiko Vancsa <ildiko.vancsa@gmail.com<mailto:ildiko.vancsa@gmail.com>>; Ghada Khalil? <Ghada.Khalil@windriver.com<mailto:Ghada.Khalil@windriver.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Frank Miller <frank.miller@windriver.com<mailto:frank.miller@windriver.com>>; Arce Moreno, Abraham <abraham.arce.moreno@intel.com<mailto:abraham.arce.moreno@intel.com>>; Hazzim Anaya Casas? <hazzim.i.anaya.casas@intel.com<mailto:hazzim.i.anaya.casas@intel.com>>; Hernandez Gonzalez, Fernando <fernando.hernandez.gonzalez@intel.com<mailto:fernando.hernandez.gonzalez@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: <StarlingX-Discuss> StarlingX 2018.10 without DPDK query Note :- Subject Changed Hi Cindy, Thanks for the update. I want to setup StarlingX simplex 2018.10 setup, but I don't have DPDK support on my machine as a result of which the compute is in degraded state. I can see error in ovs-vswitch logs. Error Message: error: "Error attaching device '0000:03:00.0' to DPDK" Can you please suggest an alternative to this? I have tried setting the vswitch type to be “none” and “ovs”, using the below command system modify –vswitch_type=ovs system modify –vswitch_type=none But after unlocking the host each time, when the system boots up, all my services (openvswitch, neutron-openvswitch-agent and nova-compute) remains in dead-state. I need to manually start all the services, but even after starting the services, my compute remains in “degraded” state. Can’t I create a StarlingX Simplex 2018.10 Setup without DPDK support? Regards Anirudh Gupta (Senior Engineer) From: Xie, Cindy <cindy.xie@intel.com<mailto:cindy.xie@intel.com>> Sent: 18 June 2019 15:30 To: Anirudh Gupta <Anirudh.Gupta@hsc.com<mailto:Anirudh.Gupta@hsc.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Winnicki, Chris <Chris.Winnicki@windriver.com<mailto:Chris.Winnicki@windriver.com>>; Ildiko Vancsa <ildiko.vancsa@gmail.com<mailto:ildiko.vancsa@gmail.com>>; Ghada Khalil? <Ghada.Khalil@windriver.com<mailto:Ghada.Khalil@windriver.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Frank Miller <frank.miller@windriver.com<mailto:frank.miller@windriver.com>>; Arce Moreno, Abraham <abraham.arce.moreno@intel.com<mailto:abraham.arce.moreno@intel.com>>; Hazzim Anaya Casas? <hazzim.i.anaya.casas@intel.com<mailto:hazzim.i.anaya.casas@intel.com>>; Hernandez Gonzalez, Fernando <fernando.hernandez.gonzalez@intel.com<mailto:fernando.hernandez.gonzalez@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: RE: <StarlingX-Discuss> StarlingX 2019.05 Release Queries Gupta, - We are still on track to release our release 2 (stx.2.0) on Aug’19 - StarlingX will be based on K8s from stx.2.0 release, and we will have another ISO released. If you want to try the ISO, you can get our daily build and get an idea of footprint. - Right now, we do not support the version upgrade from 2018.10 release to new release. You need to re-deploy your cluster. Thx. - cindy From: Anirudh Gupta [mailto:Anirudh.Gupta@hsc.com] Sent: Tuesday, June 18, 2019 5:11 PM To: Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Winnicki, Chris <Chris.Winnicki@windriver.com<mailto:Chris.Winnicki@windriver.com>>; Ildiko Vancsa <ildiko.vancsa@gmail.com<mailto:ildiko.vancsa@gmail.com>>; Ghada Khalil <Ghada.Khalil@windriver.com<mailto:Ghada.Khalil@windriver.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Frank Miller <frank.miller@windriver.com<mailto:frank.miller@windriver.com>>; Xie, Cindy <cindy.xie@intel.com<mailto:cindy.xie@intel.com>>; Arce Moreno, Abraham <abraham.arce.moreno@intel.com<mailto:abraham.arce.moreno@intel.com>>; Hazzim Anaya Casas <hazzim.i.anaya.casas@intel.com<mailto:hazzim.i.anaya.casas@intel.com>>; Hernandez Gonzalez, Fernando <fernando.hernandez.gonzalez@intel.com<mailto:fernando.hernandez.gonzalez@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; starlingx-announce@lists.starlingx.io<mailto:starlingx-announce@lists.starlingx.io> Subject: <StarlingX-Discuss> StarlingX 2019.05 Release Queries Hi All, It would be great if anyone can please address my below queries. I have prepared All in One Simplex/Duplex Setup on release 2018.10 and would like to further continue my work on the upcoming 2019.05 release. Going forward, I do have some queries: As per the recent update, the release 2019.05 is delayed and now is expected to be out in the month of August 2019. https://wiki.openstack.org/wiki/StarlingX/Release_Plan<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.openstack.org%2Fwiki%2FStarlingX%2FRelease_Plan&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608150174&sdata=5XTbQLqOiHqvdhLNFrMc1xc4z%2BsNhHddSmC4mi7m9s4%3D&reserved=0> Is there any further update on the release date? As per the release notes of 2018.10, I am using Pre-built StarlingX Image https://docs.starlingx.io/releasenotes/index.html#release-notes<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.starlingx.io%2Freleasenotes%2Findex.html%23release-notes&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608150174&sdata=gRzLq5CjYN02YXs1ImyFc4MznMF%2B4LGQ1hI2E1OwERo%3D&reserved=0> http://mirror.starlingx.cengn.ca/mirror/starlingx/centos/2018.10/20181110/outputs/iso/<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmirror.starlingx.cengn.ca%2Fmirror%2Fstarlingx%2Fcentos%2F2018.10%2F20181110%2Foutputs%2Fiso%2F&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608160169&sdata=stT%2F5gUD7Q3WvjWBiCIcrQ058%2FRLDsHY%2B%2FgGedFt3%2BY%3D&reserved=0> Will the release 2019.05 be based on Kubernetes? If Yes, What will be the changes in the footprint? And will there be another ISO with Kubernetes support available for 2019.05 Release? As per the below links of 2018.10 and 2019.05, the Hardware requirements is same. https://docs.starlingx.io/deployment_guides/current/duplex.html<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.starlingx.io%2Fdeployment_guides%2Fcurrent%2Fduplex.html&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608160169&sdata=5n%2Fgiv4Q%2FC9qSdgmf89Gk5OjCCvkdvn0MxhSTRzT9Nw%3D&reserved=0> https://docs.starlingx.io/deployment_guides/latest/aio_duplex/index.html<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.starlingx.io%2Fdeployment_guides%2Flatest%2Faio_duplex%2Findex.html&data=02%7C01%7C%7Caea5aceb03ed467a6bec08d6f7fd0e94%7Ca65543b9ae9349b580f00b85821adc50%7C1%7C0%7C636969063608170168&sdata=XQPCTD8394g8DyEL1Fjx8vScJTETrTDXDJNzLgc3B3k%3D&reserved=0> So will there be any change in the Hardware Requirement for both the releases or they’ll remain unchanged? What efforts would be required if I need to upgrade my system from the current 2018.10 to the new 2019.05 release? Regards Anirudh Gupta (Senior Engineer) DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus. DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus. DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus. DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus. DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.
participants (1)
-
Anirudh Gupta