This is a log of an IRC chat I had to day with someone interested in contributing to the API module, where I gave an overview of the API module.

This log has been somewhat edited (some things are out of sequence so that they make more sense, due to the other person in the chat asking good questions). And I edited out the other person's questions, and fixed one or two things. It is still a bit rough... probably useful though!


(09:59:22 AM) jhodgdon: So the basic idea of the API module is:
(09:59:47 AM) jhodgdon: a) During cron runs, the module parses the code and comments in PHP files (mostly) and saves information in the database
(10:00:02 AM) jhodgdon: b) Then when someone visits api.drupal.org, they get a parsed view of the API documentation


(10:16:54 AM) jhodgdon: The content from the pages comes from special comments in the code.
(10:17:02 AM) jhodgdon: For instance, take a look at this code from Drupal Core:
(10:18:02 AM) jhodgdon: http://cgit.drupalcode.org/drupal/tree/core/modules/node/node.module#n189
(10:18:31 AM) jhodgdon: And here is what this node_title_list() function looks like on api.drupal.org:
(10:18:32 AM) jhodgdon: https://api.drupal.org/api/drupal/core!modules!node!node.module/function/node_title_list/8
(10:19:03 AM) jhodgdon: So the @param documentation in the Drupal Core code comment is shown in the Parameters section on the api.drupal.org page, and so on
(10:19:27 AM) jhodgdon: Basically, in Drupal Core code, any comment that starts with /** rather than just // or /* is parsed by the API module
(10:19:37 AM) jhodgdon: and this turns into the documentation pages on api.drupal.org
(10:20:28 AM) jhodgdon: also it parses the code itself too
(10:20:44 AM) jhodgdon: you can see in the Code section on that page that it has turned a bunch of stuff into links
(10:21:26 AM) jhodgdon: anyway you can see in that code listing that for instance \Drupal is a link that takes you to that class.
(10:21:42 AM) jhodgdon: and there are reverse links too -- there is that section that say s"2 calls to node_title_list()" on that page
(10:21:48 AM) jhodgdon: that shows you other functions that call this one
(10:21:57 AM) jhodgdon: that comes from parsing the code
(10:22:24 AM) jhodgdon: which is why the "parse the file" is kind of complicated. All of what you see in node.module file in Drupal Core has to be turned into nice pages on api.drupal.org.
(10:22:28 AM) jhodgdon: and all the other files


(10:10:40 AM) jhodgdon: I guess I better talk about nodes.
(10:10:47 AM) jhodgdon: The api.drupal.org site supports comments
(10:10:58 AM) jhodgdon: in Drupal 7 and prior, comments can only be on Nodes.
(10:11:09 AM) jhodgdon: So the API module creates a fake "node" for each page that can be commented on
(10:11:32 AM) jhodgdon: that means each function, class, interface, method on a class, constant, etc. -- everything has a page and can be commented on.


(10:02:37 AM) jhodgdon: There are concepts of "project" and "branch" in the API module.
(10:02:54 AM) jhodgdon: Drupal is a project, and you could also define a project for Views etc.
(10:03:11 AM) jhodgdon: Within each one, you might have a 7.x branch or a 7.3.x or whatever. You can have more than one
(10:05:15 AM) jhodgdon: go look at api.drupal.org
(10:05:23 AM) jhodgdon: it has separate sections for Drupal 7.x and 6.x and 8.x
(10:05:30 AM) jhodgdon: those are the branches
(10:05:42 AM) jhodgdon: you do not want to mix them together


(10:00:43 AM) jhodgdon: So, back to (a) the cron runs...
(10:01:32 AM) jhodgdon: http://cgit.drupalcode.org/api/tree/api.module#n2889
(10:01:44 AM) jhodgdon: This is hook_cron(), which is Drupal's cron hook.
(10:02:13 AM) jhodgdon: Basically, it calls api_update_all_branches(), which is in the include file called parser.inc.
(10:05:57 AM) jhodgdon: http://cgit.drupalcode.org/api/tree/parser.inc#n2915 is the update branches function
(10:06:18 AM) jhodgdon: basically all it does is to check whether it's time to do an "update" on each branch that is defined. If so it adds that to a queue.


(10:06:30 AM) jhodgdon: So the API module uses the Drupal queue system.
(10:08:47 AM) jhodgdon: Basically the code can "add" jobs to the queue, and then the Drupal queue system will periodically (during cron or independently if you tell it to via Drush) use "worker" functions to process the queue jobs


(10:06:48 AM) jhodgdon: The API module defines 3 queues. http://cgit.drupalcode.org/api/tree/api.module#n2906  is what defines them
(10:09:08 AM) jhodgdon: so that api_cron_queue_info() function there defines the 3 queues and what the "worker" functions are that process them.
(10:10:02 AM) jhodgdon: For instance, the worker for the "update branches" queue is api_queue_update_branch().


(10:13:09 AM) jhodgdon: So this is the basic structure:
(10:13:29 AM) jhodgdon: - cron does "See if any branches need updating, and add them to the update branch queue".
(10:14:12 AM) jhodgdon: - update branch queue does: "See if any files have changed, and add them to the parse queue. Also if any files have been deleted, add entries to the node delete queue for everything that file used to contain".
(10:14:50 AM) jhodgdon: - parse queue does: "Parse a file and update/create entries for everything found in the file. Also add entries to the node delete queue if something that used to be in the file is no longer defined in that file"
(10:15:18 AM) jhodgdon: - node delete queue does: "Delete nodes that have been identified as obsolete"
(10:15:33 AM) jhodgdon: The hard part is the parsing.


(10:12:14 AM) jhodgdon: A note about node deletes: During both "update branch" and "parse a file" operations, sometimes code has been removed and the class/function/etc. does not exist (either its file has gone away or it no longer exists in the file). In that case, the fake node associated with that has to be deleted.
(10:12:51 AM) jhodgdon: node deletion is time consuming in Drupal (lots of hooks), so rather than deleting the node directly (especially if there are a bunch of them to delete), they are added to the node_delete queue


(10:16:06 AM) jhodgdon: And then... after all that, there is another side to the code, mostly in the api.pages.inc file, which generates the actual pages on the site from the stuff that the parse functionality saves in the database.
(10:25:22 AM) jhodgdon: If you have done some Drupal development, then you can probably figure out how the pages are generated by looking at hook_menu() and seeing what the page generating functions are, theme templates, etc.

(10:23:43 AM) jhodgdon: The API module also has a comprehensive set of automated tests (using the Simpletest framework in Drupal, which hopefully you are familiar with?)
(10:24:21 AM) jhodgdon: this helps us be sure that when we make changes to the module, we don't break anything. Normally before I commit changes/patches to the code, I run all the tests and make sure they all pass.