On Mon, Mar 11, 2019 at 10:21 PM Victor Rodriguez <vm.rod25@gmail.com> wrote:
On Mon, Mar 11, 2019 at 5:19 PM Curtis <serverascode@gmail.com> wrote:
>
>
>
> On Sun, Mar 10, 2019 at 11:54 AM Victor Rodriguez <vm.rod25@gmail.com> wrote:
>>
>> Hi team
>>
>> After some research on the topic and the great help from Numan's team,
>> I came into this great presentation about how to use OPNFV for
>> performance metrics (video available at [0]):
>>
>> https://www.openstack.org/assets/presentation-media/OpenStack-Summit-Vancouver-OPNFV-Test-tools-2.pdf
>>
>> It was presented on May 2018 OpenStack Summit, Here is described
>> Yardstick Test Case for performance measurement, They do have a very
>> good wiki on how to do :
>>
>> Installation and run:
>> http://artifacts.opnfv.org/yardstick/colorado/2.0/docs/userguide/index.html#document-07-installation
>>
>> Description of every test case :
>> https://docs.opnfv.org/en/stable-fraser/submodules/yardstick/docs/testing/user/userguide/15-list-of-tcs.html
>>
>> After a quick view, Yardstick has the capability to measure
>> performance in 3 sections:
>>
>> * Compute
>> * Storage
>> * Network
>>
>> With multiple tests cases for each one ( slide 13 of presentation )
>>
>> This is a simple update to keep the community informed about the
>> progress on this topic, if someone has any feedback, please let me
>> know.
>
>
> I haven't used Yardstick, but it sounds like a step in the right direction. Perhaps we can find things to push back upstream to OPNFV.
>
> What I'm hoping is that we can use the Packet.com resources that should be available soon to run automated performance tests and then publish the results on a continual basis. I'm definitely interested in helping out. :)
>
> Thanks,
> Curtis
>

Thanks a lot Curtis, CI/CD definitely is the next step ones I could
enable this from the developer perspective. My goal is to provide a
simple tutorial/script to run a test on a pre-configured and working
STX cloud system that developers could use to measure their changes.
Once I have this and make a demo I will ping you to try in your dev
machine and then TSC can decide to deploy this on Packet. The good
thing about Yardstick is that it has a connection to grafana so we
don't have to worry about a dashboard : ) .

The concern that I have (that I would like input from the TSC) is how
to define what configuration we want to use on Packet as baseline
configuration, this is important because a change in the configuration
might represent a huge change in performance results.

Yes it would be pretty easy to create a configuration that doesn't perform well. :) There are many, many variables, many of which would be out of our control or are a black box.

For sure part of this would be to determine what configuration, or configurations, would be best to test, and those decisions described. I would expect we would create a baseline configuration and then expand from there.

Also, are we ok
to compare the STX results against regular OpenStack
Performanceomrance numbers is always hard to handle since they are
very senitive. I would like to bring this topic to the TSC or comunity
meeting so we can tak e a decision as comunity. My work might finish
when we could have a way to get the numbers, publication, and track of
them might be a decision out of my scope.


I think the focus would be on simply having data to compare for STX, not to compare STX against other distros or projects. I'm not sure what "regular openstack" is to be honest. :)

My example is Kata Containers who do performance testing to ensure that they are not losing 1% a month. I'd just like to see some trending data.

Thanks,
Curtis

 
I will keep you posted

Regards


>
>>
>>
>> Numan/ Peng thanks a lot for the document, after reading I realize it
>> is focused on Tempest ( which is used by OPNFV but as part of
>> Functest, which is the test framework provided to answer the question:
>> Does my infrastructure simply work as expected?.
>>
>> Regards
>>
>> BTW, sorry for the long delay of an update for this topic
>>
>> Victor Rodriguez
>>
>> [0] https://www.youtube.com/watch?v=RxMbF8ILl-M
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss@lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>
>
>
> --
> Blog: serverascode.com


--
Blog: serverascode.com