Based off the discussions from http://groups.drupal.org/node/160134, we are having a code sprint!
Who is this for?
Everyone who wants to play an active role in the Drupal community. If you want to learn code come and join us. If you want to learn how to compose better documentation and fix existing documentation that works as well. If you want to look over the shoulders to see what happens and see users helping users in action, join us as well!
If possible, please bring your laptop and (this one is a requirement) a lot of enthusiasm.
Location:
Riot Games
2450 Broadway
Suite 100
Santa Monica, CA 90404
Food, drink, and parking (From dragonwize: If you park in the garage just grab a ticket and it will be validated when you arrive.) all provided thanks to dragonwize and his workplace, Riot Games :D
Comments
Oh darn...Santa Monica. I'd
Oh darn...Santa Monica. I'd go far, but that's a bit too far for Thursday this week :(
So, see you guys at the camp.
WizOne Solutions - https://wizone.solutions - Drupal module development, theme implementation, and more
FillPDF Service - https://fillpdf.io - Hosted solution for FillPDF
What time
Do we know what time the code sprint starts yet?
Paul Chernick
CEO
Chernick Consulting
(310) 569-2517
I'm waiting to hear back from
I'm waiting to hear back from dragonwize on what time works for him and his workplace to let us in.
Time & Address
Time: 7pm - til
From the feedback on the other thread 7pm seems like a good time to give people plenty of time to get here from where ever.
Address:
Riot Games
2450 Broadway
Suite 100
Santa Monica, CA 90404
I have food and drink prepped and am working out the rest of the details. I will post or let you know the exact details on parking but there is lots of street parking as well as a huge garage below the complex with entrances on all sides. I am working on the validation details and will post as soon as I have them.
Parking Validation
If you park in the garage just grab a ticket and it will be validated when you arrive.
awesome! thanks for making
awesome! thanks for making this happen! hope to see you tomorrow. I'll prob head over in the afternoon and find a cafe to work at nearby until the doors open.
--
mike stewart { twitter: @MediaDoneRight | IRC nick: mike stewart }
I'll miss this time, more's the pity...
Hi all,
I won't manage to make it there, this time. I'm on public transit
until I find remunerative work locally. I've relocated recently, from
the Philly area and currently live in Monrovia. I've discovered that,
even if I leave promptly at 9 (what a drag!), I still run the risk of
missing the last bus on the last leg of my journey home which, if
I caught it, would be a half hour bus trip and then a half hour on
foot anyway.
I hope to catch the next gathering, and meet those of you with
whom I'm not yet acquainted. I've gotten comfortable with Drupal
-- as a newbie -- with the Pro Drupal 7 Development Study Group
that Paul (pcher1bw) so graciously leads at Droplabs, pretty easily
because I've worked with so many software systems over the years.
Have a good session!
You can hitch a ride
John I will be driving out from El Monte if you would like to hitch a ride. I will also have my 12 year old daughter in tow with me if nobody objects. She will probably sit in a corner and read a book.
Can't this time, thanks, but wanna keep you in mind...
Hi Terry,
Timing would be too tight, coming back, for this event, but I'd love to take you up on the offer for a different event if we cross paths. Based on your profile, I'd enjoy to get acquainted. El Monte isn't a bad commute for me.
Thanks for the initiative.
I would like to come tonight,
I would like to come tonight, but I am afraid as a relative Drupal newbie that I will be more of a hindrance than a help.
Are you serious about the "if you want to look over the shoulders to see what happens" part, or are you just saying that to be nice. :-)
FWIW, my goal is to learn as much as possible about Drupal 7 for a project I am working on.
Cheers.
I am enthusiastically serious!
I can't talk for anyone else, but I've found it a good learning experience in the past and encourage newbies to jump in :)
Don't worry about being a hinderance :) Coding for Drupal (especially at sprints) should be about having fun. If you're not coding, Christefano will be there to help you get set up on how to work on Drupal. So even if you can't help out this time, the next time(?) you will be more prepared. And if not next time, then after that (or the time after). But it'll come.
If you want to learn about Drupal 7 and you are able to make it out tonight, I hope that the sprint will help you get a better understanding of what goes into making Drupal what it is (and maybe learning some of the inner workings of Drupal in the process :))
Everyone is welcome. Come to
Everyone is welcome. Come to help code, come to ask questions, all are welcome.
Anyone getting here early,
Anyone getting here early, message me and I will give you a tour.
Just a suggestion for the future
Hi All,
I look forward to seeing you all in similar circumstances in the future. I think we can help Drupal a lot if we have regular patching parties! I'd like to see us do at least one patching party a month. I volunteer Droplabs as a venue to host the patching party either every other month or every third month. I'd like to start the planning for an August patching party soon, perhaps we can have 2 in August, one at DCLA and one at another date and time.
I'd like to suggest that we list all of the things we want to work on in the code sprint / patch party prior to the meeting. I would also suggest that the person adding something to the list put a priority on it, 1 to 5, with the highest priority being 1 and lowest being 5. Last night the website for DCLA may have been the single highest priority. I would have gladly helped Mike with that one last night if he needed help.
If we have a list of things to work on, people can start working as soon as they arrive.
Would it be a good idea to define teams that can work on an item. I think a team would be a coder, a reviewer, a tester and someone to do any documentation. If someone would like to define a team another way, feel free.
Any other thoughts on the subject are welcome!
Paul
Paul Chernick
CEO
Chernick Consulting
(310) 569-2517
It was great seeing / meeting
It was great seeing / meeting all of you at the code sprint last night. I think a list is a good idea for targeting specific areas to focus on. However, if the community can't get a list together in time for the sprint, Christefano's suggestion of patch bingo (http://drupal.org/patch-bingo) could serve as an easy jumping off point to get started.
Kris
Code Sprint Meeting Minutes
Hi Everyone,
Here is a brief review (cough, cough) of the night's happenings.
Our first time at Riot Games Venue
Riot Games is an awesome venue. Parking is easy under the building at the Yahoo Center. Alan is a terrific host. The tour of the facility - wow. Nice game area, kitchens, dual screens, and almost ready theater seating review room with 20 inch, ported subwoofer. Every wall was "active" - white boards and pin boards galore - all in use. Riot Games is hiring, and want some Drupal experts, maybe even more than some. View their web site for many offered jobs. Our long time member Mikey is working there now. Mikey likes it.
Discussed Queued Issues
The way Drupal does fixes was reviewed and many questions got answered for attendees.
Drupal.org queued issue numbers that were listed on the white board are:
Core: 1015916 - maxlength for html attribute 'title=' of upload files - changed from 80 to 1024, and avoid error message of failed to write to database.
Contrib: One big fix was broken into four patches: 1212392, 1212096, 1212396, 1191324 - D7 Review Signup Fix
Christefano was concerned about these issues: 968994 and 194242 dealing with displaying Drupal Association badges for employees.
To view these issues go to http://drupal.org/node/#####, where ##### is replaced with the above number.
Due to a hiccup with the wireless connection, the middle part of the meeting was without net access. We had lots of other discussions, which are covered below. A few members left when the wireless went down. At the end, eight of us went to McCabes, see After Dark for details.
git primer
A primer on git was done by Ashok with help by Mikey and Mike.
Miscellanous
Module 'dreditor' and Firefox 'greasemonkey' - I forgot the content these were mentioned. It was important enough I wrote them down. This means I need to look into them, and install them and use them. So, do you?
Cache - Is this word Overloaded? Or Understanding Performance Tuning Questions
Oh, this was a long discussion. No way to cover it all, but the final beliefs, some that need confirmation, and many questions.
The word 'cache' is so easily uttered, but what does it truly mean, particularly regarding RAM usage? Below is background information, most of it firm, but do provide corrections, along with several big questions. Answers to these questions will dictate if configuration files need to be edited, or if some or all installation default values are acceptable for terrific performance.
Points decided are 'cache' does not always mean in RAM, but can still mean speed optimization. There are so many variations possible, on machines with no "free" RAM, or with excessive RAM "free." What does "free" mean was also an issue, and is covered half dozen paragraphs below.
An attempt has been made in this writing to make the terminology available for the lay Drupaler, or those new to OSes, or MySQL, etc. So many fields of expertise came into this discussion, no final answers were concluded. And we want to know.
Basic question: Where is RAM used to optimize reads and writes of what Drupal "data?" What parts of Drupal, memcache, MySQL, the OS, are involved in decisions to hold what data, and keep it in RAM for how long? Details wanted.
What "parts" of Drupal tables "cache_*" are copied to RAM by module 'memcache' (when the OS has installed the package 'memcache'), so cached static blocks, regions, pages, session data, and more have faster reads and writes. Writes are not to slow hard drive devices, and RAM collects writes until flush time to the slower device. This Drupal "cache" is not stored in RAM. These cache_* tables are on slow hard drive devices.
However, memcache or MySQL might optimize speedy reads and writes by keeping cache copies. Is this right? To what degree? What are the details?
Memcache stores keyed data in RAM. And allows network access to other machines, like web front end clients to a backend, dedicated database server. Cool. So memcache has it's "cache" forced into RAM. What processes decide to request memcache keyed data? The module "memcache?" If so, then what data is stored? Compiled MySQL queries? Drupal's tables cache_*? Which ones? How much? Which records? Aging/expiration algorithm(s)?
Does Drupal's module "memcache" use RAM for itself? Or does this module merely request the OS installed package "memcache" to store certain records? Which "records?" As the drupal.org/project/memcache documentation literally has no descriptions of any optimizations it provides, will the source code have to be read? Are there any web pages documenting module memcache's optimizations?
MySQL will hold some of its data in RAM, to speed reads and writes. How much data, what data, had several members voice their best recollection of their documentation reading. Does MySQL have some default 'hard coded' numerical value for the minimum size of RAM it will use, or does MySQL use an algorithm to determine the minimum RAM amount it will use for RAM caching?
For some clarification, the OS allocates RAM for each process per it's requests. What RAM is left over, the OS will allocate most of it to use for its optimization for slow file system devices, called "IO Cache" and "File Cache." The OS will reduce the size of it's allocation when other processes, like MySQL, request more ram. The amount of "free" RAM does not include the RAM used for IO and File Caches. Yet, the OS will release IO and File Cache space for other processes.
A big question of the night was how much RAM will MySQL use for its cache? Could MySQL, with default install configuration values, decided to hold "all" table data in RAM, if there is lots of available RAM? For example, 20 Drupal web sites, with 500 meg of table space, hundreds of meg of temporary space, index tables, and the like, on a machine with 32 gig of RAM, and 28 gig is "free" - will MySQL decide to keep RAM copies of ALL it's tables? Or is there a maximum value? What max? Hard coded numerical value? Or algorithmic?
Another big question was MySQL aging algorithm for what it holds in its RAM cache. Does this algorithm look at "free" RAM, and if tons of RAM is available for other processes, will MySQL decide to hold large amounts of cache data in longer? That is, not expire its RAM cached records and tables?
Does MySQL cache in RAM whole tables, or just frequently used rows?
After Dark
McCabes http://www.mccabesbar.com/ was having a Meetup of http://www.gamedevdrinkup.com/. McCabes closes at 2, or is that 4 AM? We enjoyed black and tans. I owe someone for mine. Mikey?
All grammar errors and typos are mine.
Peter
LA's Open Source User Group Advocate - Volunteer at DrupalCamp LA and SCALE
Update on MySQL optimizations
I've done a lot of reading since this meeting, and learned MySQL 5.1 installation sets no meaningful default values for optimal performance. Installing never looks at the machine's RAM amount.
IMHO, for Drupal using MySQL, one must always edit and add my.cnf values for MySQL optimization based upon the number of Drupal websites hosted (increases over time, so my.cnf must be edited monthly or yearly for performance reasons). my.cnf might be edited if RAM changes to avoid crashing the OS.
I've kept my post here brief, as the initial details of my recent learning and edits to my web host I posted some of it, the "open files" limit changes, here:
http://groups.drupal.org/node/165109#comment-561994
I'm wondering if there is a GDO Group for MySQL Optimization? So, this information can be centralized for all MySQL admins to read.
Pete
Peter
LA's Open Source User Group Advocate - Volunteer at DrupalCamp LA and SCALE
MySQL /etc/my.cnf and MySQL uses OS "File Cache" to hold tables
This post is a follow up to questions I asked in my Code Sprint Meeting Minutes. I've found answers. And I include here information about how Operating Systems, kernels, supervisors and optimized features found in most of these work.
I've found all the members I've talked to have not known about how the Operating System (OS) has a "File Cache" feature, and how this feature works to speed access to file records/blocks using the computer's RAM to optimize all read and written files.
Reading MySQL.org 5.1 online manual I confirmed MySQL depends on the Operating System's "File Cache" feature to load into RAM all MySQL tables. This means in systems with free RAM, and not using swap space, will have MySQL table files always in RAM.
Most OS File Cache work as follows. At least once, each set of table files (.frm, .MYD, .MYS), as MySQL server opens each file, many OSes will "read ahead", and copying the entire file into RAM, into the memory reserved for File Cache. The File Cache feature will age parts of the file, and the entire file, over time, per it's algorithm, and kernel parameter values. If the File Cache never fills up, then it's likely the files loaded for MySQL will never be expired, and it's File Cache blocks will not be freed for other newly read files to use.
Configuring my.cnf is not necessary to have the OSes File Cache do this for MySQL.
Configuring my.cnf to allocate enough RAM to load all the high speed search indices MySQL, created per the data definition for each table, is key to faster queries that use these indices.
Also, configure my.cnf to cache result sets for commonly used queries, which includes the WHERE clause explicit conditional values, that is numbers and text strings.
A few more File Cache details, critical for understanding when UPDATES get written to the hard drive. The File Cache will not only be used for reading files from disk to RAM, but it's RAM "records/blocks" get updated whenever the same file is written to. The cool part is the OS will use these updated "files" in File Cache RAM, whenever the file needs to be read again, even if the updated records have not been flushed to the disk. The "disk" file fully resides in RAM, for reads and writes, and is shared between all processes, that need that one file. RAM acts like the disk, invisible to the application code. The code runs faster.
This flushing, in Linux, is done by the process pdflush, or if journaled, then kjournald process. Why does it work this way? Over the decades this algorithm was determined to be highly optimal for best use of RAM to speed file IO, both reading and writing files to slower devices.
There are other algorithms that might come into play, like memory mapping the device, or IO Cache. IO Cache was invented first, before File Cache. Then came memory mapping, where disk space is mapped into RAM. Both these process use RAM to speed access to slow storage devices. Both are provided the OS.
IO Cache consists of read and write buffers for as a bi-directional transfer between application RAM and disk. In the old days, not so long ago, there could be in RAM 3 copies of each record read or written between processes and hard drives. With reading, the first copy created is in IO Cache, as the OS reads the hard drive blocks, and copies the block into an IO Cache buffer. This buffer is then copied into the Virtual Space of the application that requested the record, into an space allocated by the language in use, say C, for double buffering IO to/from the OS. The language will then copy that very same record into the user requested allocated space, in a way that it is contiguous, for random access by your code. Three copies. Very slow, even if two of the copies are RAM to RAM.
Memory mapping is much like File Cache, only faster, as it has only two copies of each disk block in RAM. The copy normally existing for the bi-directional transfer to and from disk to RAM, is what memory mapping eliminates. The first copy in RAM exists in the application's IO buffer space, and the second copy is the user requested allocated space, which is not really IO related, but is for code to randomly access.
Now a days, many algorithms can still be in use. Unless OS, compiler, or sometimes the coder, uses special OS API functions and compiler/link loader options. MySQL is one such application that uses optimized read/write functions. So, MySQL may only have two copies of the file in RAM if the OS is using File Cache, or the single copy of memory mapped files, even faster.
At least this is my current understanding, as complex as the documentation on these features is. I'd appreciate any corrections, feedback, or URLs or book references where these features are explained with a higher degree of readability. :-)
There is also L1 and L2 hardware cache, higher speed RAM between the CPU registers and RAM. Describing L1 and L2 is outside the scope of this post. Plus, it's rare the programmer has access to these caches. I mention them as these are the first instances of the word "cache" I came across with my first 486.
When a Drupal webmaster mentions "cache" they are not speaking about any of the above 'older' cache mechanisms I learned first. Drupal core does something that offers even higher performance gains than the hardware and OS caching mentioned. Eliminating repetitive SQL queries, that would use these mentioned HW/OS caches, means Drupal avoids slow device access for entire blocks, page sections, and even the entire web page. This Drupal definition of the word cache is nice. :-)
What Drupal does is save the HTML rendering of SQL result sets, when it knows a block can be entirely saved, or a region, or page. Drupal will "save" the HTML snippets into a database table whose name is prefixed with "cache_". Drupal will then read from these tables to create regions and pages to serve to the end user.
The cool part about this, is these cache tables use the mentioned HW/OS cache features. So, as Drupal first starts a newly created website, and renders HTML, saving the HTML snippets into tables named cache_*, the rows of these tables will eventually be written to the hard drive, but using free time of the computer to do so. Thus, applications are not slowed down by writing table rows to disk.
Now, the clever part. These disk files, the tables rows on disk, are never read off the slow device. Why? The needed rows are already in RAM, in File Cache, or Memory Mapped, and then, another copy, in MySQL Query Cache in RAM, for the fastest possible 'read' into Drupal code to serve to the web browser. (Yes, three copies of the same records in RAM.)
Now, the rest of the story is finished with memcache, varnish, PHP op-code, and similar add on layers, that further optimize the RAM usage. These topics have been posted on many times, through GDO and DO, but not from my perspective.
I have learned what each of these layers do, from a conceptual viewpoint, and next is become seeped in their terminology, and fine tuning their configurations. I'll post in a month or two after I read this goal. And one day, will gather all my performance posts into one single expert wiki page.
Why do this? How better to "know" I have comprehensive understanding of the full set of possible Drupal optimizations than by publicly posting in a wiki, and asking the community to maintain it by making updates as technology changes.
Peter
LA's Open Source User Group Advocate - Volunteer at DrupalCamp LA and SCALE