[Starlingx-discuss] STX performance tests update

[Victor Rodriguez]

On Tue, Mar 12, 2019 at 7:15 AM Curtis <serverascode at gmail.com> wrote:
>
>
>
> On Mon, Mar 11, 2019 at 10:21 PM Victor Rodriguez <vm.rod25 at gmail.com> wrote:
>>
>> On Mon, Mar 11, 2019 at 5:19 PM Curtis <serverascode at gmail.com> wrote:
>> >
>> >
>> >
>> > On Sun, Mar 10, 2019 at 11:54 AM Victor Rodriguez <vm.rod25 at gmail.com> wrote:
>> >>
>> >> Hi team
>> >>
>> >> After some research on the topic and the great help from Numan's team,
>> >> I came into this great presentation about how to use OPNFV for
>> >> performance metrics (video available at [0]):
>> >>
>> >> https://www.openstack.org/assets/presentation-media/OpenStack-Summit-Vancouver-OPNFV-Test-tools-2.pdf
>> >>
>> >> It was presented on May 2018 OpenStack Summit, Here is described
>> >> Yardstick Test Case for performance measurement, They do have a very
>> >> good wiki on how to do :
>> >>
>> >> Installation and run:
>> >> http://artifacts.opnfv.org/yardstick/colorado/2.0/docs/userguide/index.html#document-07-installation
>> >>
>> >> Description of every test case :
>> >> https://docs.opnfv.org/en/stable-fraser/submodules/yardstick/docs/testing/user/userguide/15-list-of-tcs.html
>> >>
>> >> After a quick view, Yardstick has the capability to measure
>> >> performance in 3 sections:
>> >>
>> >> * Compute
>> >> * Storage
>> >> * Network
>> >>
>> >> With multiple tests cases for each one ( slide 13 of presentation )
>> >>
>> >> This is a simple update to keep the community informed about the
>> >> progress on this topic, if someone has any feedback, please let me
>> >> know.
>> >
>> >
>> > I haven't used Yardstick, but it sounds like a step in the right direction. Perhaps we can find things to push back upstream to OPNFV.
>> >
>> > What I'm hoping is that we can use the Packet.com resources that should be available soon to run automated performance tests and then publish the results on a continual basis. I'm definitely interested in helping out. :)
>> >
>> > Thanks,
>> > Curtis
>> >
>>
>> Thanks a lot Curtis, CI/CD definitely is the next step ones I could
>> enable this from the developer perspective. My goal is to provide a
>> simple tutorial/script to run a test on a pre-configured and working
>> STX cloud system that developers could use to measure their changes.
>> Once I have this and make a demo I will ping you to try in your dev
>> machine and then TSC can decide to deploy this on Packet. The good
>> thing about Yardstick is that it has a connection to grafana so we
>> don't have to worry about a dashboard : ) .
>>
>> The concern that I have (that I would like input from the TSC) is how
>> to define what configuration we want to use on Packet as baseline
>> configuration, this is important because a change in the configuration
>> might represent a huge change in performance results.
>
>
> Yes it would be pretty easy to create a configuration that doesn't perform well. :) There are many, many variables, many of which would be out of our control or are a black box.
>
> For sure part of this would be to determine what configuration, or configurations, would be best to test, and those decisions described. I would expect we would create a baseline configuration and then expand from there.
>
Having a baseline configuration will be a great milestone

>> Also, are we ok
>> to compare the STX results against regular OpenStack
>> Performanceomrance numbers is always hard to handle since they are
>> very senitive. I would like to bring this topic to the TSC or comunity
>> meeting so we can tak e a decision as comunity. My work might finish
>> when we could have a way to get the numbers, publication, and track of
>> them might be a decision out of my scope.
>>
>
> I think the focus would be on simply having data to compare for STX, not to compare STX against other distros or projects. I'm not sure what "regular openstack" is to be honest. :)
>
> My example is Kata Containers who do performance testing to ensure that they are not losing 1% a month. I'd just like to see some trending data.
>

I see your point now , thanks . Yes , the goal of having a dashboard
that track performance of STX daily will be very usefull for the
project .  Without comparing to other projects we will be able to
detect regresions on perfomrance ( something we currentlyd ont have )

Regards

> Thanks,
> Curtis
>
>
>>
>> I will keep you posted
>>
>> Regards
>>
>>
>> >
>> >>
>> >>
>> >> Numan/ Peng thanks a lot for the document, after reading I realize it
>> >> is focused on Tempest ( which is used by OPNFV but as part of
>> >> Functest, which is the test framework provided to answer the question:
>> >> Does my infrastructure simply work as expected?.
>> >>
>> >> Regards
>> >>
>> >> BTW, sorry for the long delay of an update for this topic
>> >>
>> >> Victor Rodriguez
>> >>
>> >> [0] https://www.youtube.com/watch?v=RxMbF8ILl-M
>> >>
>> >> _______________________________________________
>> >> Starlingx-discuss mailing list
>> >> Starlingx-discuss at lists.starlingx.io
>> >> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>> >
>> >
>> >
>> > --
>> > Blog: serverascode.com
>
>
>
> --
> Blog: serverascode.com