Posted by christianchristensen on November 1, 2012 at 10:02pm
Last updated by christefano on Sat, 2014-01-25 21:01
Last updated by christefano on Sat, 2014-01-25 21:01
TODO:
- System under test
- Take a copy of a big camp website (BADCamp, Atlanta) sanitize it and have it in a distributable form.
- Alternatively: Distro {Commons, COD, …} with devel generate (or a SQL dump scenario)
- Basic Chef config to build a LAMP stack...
- Competition Rules
- Teams and/or Individual entries?
- Tests - partner with an established testing/perf company?
- Basic performance based metrics (ab, curl, ...)
- Basic PHPUnit tests (exercise site functionality)
- Get a trophy!
- Choose a more inclusive name? per http://groups.drupal.org/node/265533#comment-848438
Name suggestions
- Drupal Drag Race (rgristroph)
- Golden Streak Award (christefano)
- Drupal Hypermile Award (christefano)
- Golden Slipstream Award (christefano)
- Golden Hamster (kscheirer) - w/ hamster wheel trophy
- Drupal Speedtest (kscheirer)
- Drupal Siege (kscheirer) or Drupal Under Seige?
- Drupal Dangerzone (syakely)
- Drupal Server Shootout (kscheirer)
- Extreme Drupal Games (christefano)
When will we run this?
- SANDcamp, Jan 24-26 in San Diego, CA
- Portland
- The Extreme Drupal Games at GLADCamp, the Greater Los Angeles Drupal Camp, on March 7-9, 2014
Contest goal:
Performance test that will give (or extend) feedback to speed up an existing site -- the winner of the contest will use known best practices.
Contest participation
Submissions are git diff’s/PR’s that have the diff that will document the performance improvement.
Contest would take place on the training day... we would need a list of registered participants
Original Doc:
https://docs.google.com/document/d/14PLnDI-G4b-d_jYbEImSpVTT7gZ7L1b2yaiB...
Comments
I love it!
How do we get involved? We're building the new sandcamp site now, we could add a landing page to promote this event at SANDcamp.
I'd like to participate as well, are teams allowed or just individual entrants?
The next step is to make a load test / scoring script, I think
I think the next step is to have some plausible way to run a load test and get a score. I'll be working on that this weekend, something simple launched from Jenkins. If we have that, we can probably work around almost anything else, launching contestant's VMs by hand if needed and etc.
I'll also make a list this weekend of tasks we need to do, that way we can split it up.
I want to have that in place before putting any announcements up though, so that I know we can run at least something.
I think we should have team and individual entries, with a separate trophy for each. I would expect that a lot of the learning and benefit of the contest would come from working with people with complementary skill sets.
But no set of rules has been formally aggreed upon or published, so please suggest whatever you think would be best.
In ref to testing. I'm not
In ref to testing. I'm not sure how you plan to judge the sites, but one idea might be that some testing service would be interested in sponsoring the event.
ie - during the "judging" part of the event, the sponsor's testing tools would be on display for everyone to see how great they are. I'm thinking something like Sosta, Blazemeter, or Blitz.io.
Testing
Testing anonymous users is boring. Long story short you're testing how well the web sever stack works (varnish or nginx). What is interesting is testing logged in users.
Yes
There's one variant of this contest that's about setting up a good anonymous caching setup, but that's mostly old hat at this point, I agree. ;)
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Cheat Sheet
This would be my cheat sheet for the event: http://groups.drupal.org/node/210683
Good idea
We should probably provide a link to that list and any other short, quick performance checklists to contestants, so that the inexperienced feel welcome to participate, and at least get the gratification of making some improvements. Then they can take those back to their own sites and make them all better :)
We could have a n00b category
We could have a n00b category specifically for the non-experts. They could get their own award. +1 for making new performance experts :) Chess tournaments usually have a trophy for the "Best Unrated" player, meaning someone with no tournament experience. Maybe some volunteer n00b coaches? This requires the cheat sheet of course, which is a very good idea.
Good idea!
May I gently suggest a more inclusive name?
Suggestions ?
Yeah you are not the first to observe that. Do you have any good ideas for a better name ? Drupal Drag Race maybe ?
Agreed! I absolutely love the
Agreed! I absolutely love the spirit and ideas behind this initiative but would also like a name change.
"Neckbeard" is a fairly derogatory word and I'd like to contribute to this initiative in ways other than making an effort to re-appropriate the word in order to change the stigma around it.
I've added a new section under TODO with a few name suggestions.
Adding a TODO item is a great way to handle this
Way to make it actionable, christefano :)
Golden Hamster?
The trophy could be a little cute hamster pushing his wheel to make the internets go...
I'm a bit unclear on what
I'm a bit unclear on what would be tested:
A) My tricked out (a.k.a hacked) Drupal Core
B) My tricked out (a.k.a hacked) Drupal Core + my tricked out V.M.
C) My tricked out (a.k.a hacked) Drupal Core + my tricked out V.M. + my large racks of hardware.
Presumably C would be less useful/fun. If A, what other services are available (memcache, etc.)?
Any contrib modules in play (presumably you could swap out core functionality with a contrib module)?
What page is being load tested (In the past I've enabled all core modules, enabled all blocks, enabled all permissions, disabled the page cache, and hit the homepage)?
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
I think we'll be testing B.
I think we'll be testing B. Core hacks are allowed as long as all simpletests still pass (any other requirement?). But mostly we're after config changes on your server, but all VMs will be exactly the same specs (OS/CPU/RAM/Disk) - anything else?
I think it is more: D) My
I think it is more:
D) My tricked out (a.k.a hacked) Drupal Core + my tricked out Drupal Contrib + my tricked out V.M.
From the last conversation I was involved in, the Drupal environment is likely to be a distribution with devel-generated data in it (a basic site built on COD, Commerce Kickstart, OpenPublish or similar), so lots of contrib stuff to play around with too. The tests should reflect a user's path through the site, IMO.
Selenium to simulate a user workflow?
If there was a standardized Selenium script, that would help exercise the system and simulate a user's experience. It would also shake out some goofy edge cases if someone cut one too many corners.
Yes, we need to test site functionality as well
Yeah, the test will need to excercise the site enough that you can't just reduce it to static html or turn off tons of modules or anything like that. I think having some selenium tests that register new users and save content that run concurently with load tests is probably the way to go.
Testing tools
Might want to use https://github.com/LionsAd/xhprof-kit or a variant for performance testing. The advantage is that XHProf report is available, so you can analyze why it is faster.
I would say to disallow caching before Apache/PHP or split things up into server level vs. code level.
Though probably partnering with New Relic would be the most fun :).
Love the idea.
The Extreme Drupal Games are March 7-9th, 2014
It's on!
We're having our first competition at GLADCamp, the Greater Los Angeles Drupal Camp, this March 7-9th, 2014, in Pasadena, California. We're going with the name "Extreme Drupal Games" and have a page up at https://gladcamp.org/2014/extreme-drupal-games
I've cross-posted this wiki page to the GLADCamp planning group and everyone is welcome to join it at https://groups.drupal.org/gladcamp, or continue here in the Golden Neckbeard and High Performance groups. A separate discussion specific to the GLADCamp edition of the competition is going at https://groups.drupal.org/node/400568