On Tue, Mar 12, 2019 at 7:15 AM Curtis <serverascode@gmail.com> wrote:
On Mon, Mar 11, 2019 at 10:21 PM Victor Rodriguez <vm.rod25@gmail.com> wrote:
On Mon, Mar 11, 2019 at 5:19 PM Curtis <serverascode@gmail.com> wrote:
On Sun, Mar 10, 2019 at 11:54 AM Victor Rodriguez <vm.rod25@gmail.com> wrote:
Hi team
After some research on the topic and the great help from Numan's team, I came into this great presentation about how to use OPNFV for performance metrics (video available at [0]):
https://www.openstack.org/assets/presentation-media/OpenStack-Summit-Vancouv...
It was presented on May 2018 OpenStack Summit, Here is described Yardstick Test Case for performance measurement, They do have a very good wiki on how to do :
Installation and run: http://artifacts.opnfv.org/yardstick/colorado/2.0/docs/userguide/index.html#...
Description of every test case : https://docs.opnfv.org/en/stable-fraser/submodules/yardstick/docs/testing/us...
After a quick view, Yardstick has the capability to measure performance in 3 sections:
* Compute * Storage * Network
With multiple tests cases for each one ( slide 13 of presentation )
This is a simple update to keep the community informed about the progress on this topic, if someone has any feedback, please let me know.
I haven't used Yardstick, but it sounds like a step in the right direction. Perhaps we can find things to push back upstream to OPNFV.
What I'm hoping is that we can use the Packet.com resources that should be available soon to run automated performance tests and then publish the results on a continual basis. I'm definitely interested in helping out. :)
Thanks, Curtis
Thanks a lot Curtis, CI/CD definitely is the next step ones I could enable this from the developer perspective. My goal is to provide a simple tutorial/script to run a test on a pre-configured and working STX cloud system that developers could use to measure their changes. Once I have this and make a demo I will ping you to try in your dev machine and then TSC can decide to deploy this on Packet. The good thing about Yardstick is that it has a connection to grafana so we don't have to worry about a dashboard : ) .
The concern that I have (that I would like input from the TSC) is how to define what configuration we want to use on Packet as baseline configuration, this is important because a change in the configuration might represent a huge change in performance results.
Yes it would be pretty easy to create a configuration that doesn't perform well. :) There are many, many variables, many of which would be out of our control or are a black box.
For sure part of this would be to determine what configuration, or configurations, would be best to test, and those decisions described. I would expect we would create a baseline configuration and then expand from there.
Having a baseline configuration will be a great milestone
Also, are we ok to compare the STX results against regular OpenStack Performanceomrance numbers is always hard to handle since they are very senitive. I would like to bring this topic to the TSC or comunity meeting so we can tak e a decision as comunity. My work might finish when we could have a way to get the numbers, publication, and track of them might be a decision out of my scope.
I think the focus would be on simply having data to compare for STX, not to compare STX against other distros or projects. I'm not sure what "regular openstack" is to be honest. :)
My example is Kata Containers who do performance testing to ensure that they are not losing 1% a month. I'd just like to see some trending data.
I see your point now , thanks . Yes , the goal of having a dashboard that track performance of STX daily will be very usefull for the project . Without comparing to other projects we will be able to detect regresions on perfomrance ( something we currentlyd ont have ) Regards
Thanks, Curtis
I will keep you posted
Regards
Numan/ Peng thanks a lot for the document, after reading I realize it is focused on Tempest ( which is used by OPNFV but as part of Functest, which is the test framework provided to answer the question: Does my infrastructure simply work as expected?.
Regards
BTW, sorry for the long delay of an update for this topic
Victor Rodriguez
[0] https://www.youtube.com/watch?v=RxMbF8ILl-M
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
-- Blog: serverascode.com
-- Blog: serverascode.com