Research on two HTTP libraries

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
dixon_'s picture

Me and Dave Hall (@skwashd) spent part of our weekends looking through the HTTP request handling code in both the Zend and Symfony frameworks. We may have been little bit biased by the good help from Lukas Smith (@lsmith77) of the Symfony community, but we've tried to be fair in the comparison of the 2 projects. We encourage feedback on our findings.

Assessment Criteria

We were guided by the assessment criteria discussed at the WSCCI meeting on 2 August 2011. The criteria was as follows:

  • Dependencies
  • Security
  • Handling of request variables
  • Handling and parsing of headers
  • Handling of the request body
  • Licensing and contribution terms

Secure by Default

Neither of the frameworks are “secure by default” as they take no extra steps to sanitize data. We don’t feel that this is a problem as we are dealing with the HTTP requests. The implementation of security controls should be done where security is a problem, such as the storage or presentation layers.

Concepts

Zend framework seems to be very big with an crazy array of different tools. But their HTTP Request component (links earlier in this thread is to their HTTP client) seems very simple in comparison (which might not be a good thing). All their component are loosly coupled, but as the Controller component is one of the foundations of their MVC architecture, it relies on it heavily,which we'll explore further down.

Symfony on the other hand, has a smaller array of tools, but has a richer HTTP Foundation. In fact, their whole framework seems very HTTP centric, all the way from HTTP Kernel and up. They've put a lot of effort into separating their components. They aren't loosely coupled, they are decoupled. All their components are in separate repositories and they are presenting and promoting them as stand-alone products. On top of this they have a community and culture of building distributions and other independent frameworks on top of their decoupled components. See Silex and the Content Management Framework initiative.

Zend

Code weight is around 50 KB. The Zend framework's HTTP component is fairly small and simple relatively to the rest of the framework.

Dependencies

Like most Zend classes the Http class depends on the Zend Config class to provide basic configuration information. Zend Uri is required for URL parsing.

In Zend's AbstractRequest class which the Http class extends, there are some MVC-related code which should concern us when it comes to being cleanly extensible. But from what we could tell it's not a hard dependency though. So it should still be possible to extend.

<?php
public function getControllerName() {
if (
null === $this->_controller) {
   
$this->_controller = $this->getParam($this->getControllerKey());
}
return
$this->_controller;
}
?>

Request Variables

Zend provides (raw) access to GET, POST, COOKIE and SERVER.

Headers

Zend doesn't do any useful parsing of headers like Accept or Language, other than just returning the desired key [ref]. Interestingly it seems to only parse HTTP_* headers, and forgetting about CONTENT_TYPE, CONTENT_LENGTH, AUTHORIZATION etc. if Apache aren't used.

Zend only provides simple key retrieval for COOKIEs.

Body

Provides access to the content body as a string [ref].

Others

The Request component doesn't handle SESSION or FILES. These are handled by separate components [ref, ref].

HTTP client

A quite solid HTTP client is provided as a separate component [ref].

Licensing and Contribution Terms

Zend Framework is licensed under the terms of the new BSD license. All contributors to the ZF project are required to sign a Contributor Licensing Agreement, even for trivial patches. This could make it difficult for the Drupal community to contribute back to Zend.

Symfony

Code weight is around 180 KB. Symfony has better methods for mocking requests [ref, ref]. Essentially, they can easily do sub requests to other controllers in the system [example]. This is their internal ESI system! We found this very interesting.

Dependencies

The HTTP Foundation makes the assumption that you use its built in session handling. If we decide not to use it, we could work around it by extending the class and add a Drupal specific session handler.

Request variables

Symfony provides (raw) access to GET, POST, COOKIE and SERVER.

Headers

Symfony provides parsing of complex headers like Language and Accept [ref, ref]. It doesn't forget about CONTENT_TYPE, CONTENT_LENGTH and ATHORIZATION [ref], as opposed to Zend.

Symfony provides some extra parsing of COOKIE headers too [ref].

Body

Provides access to the content body, both as a string or as a file resource [ref].

Others

Symfony provides a really neat SESSION handling interface [ref], with a PDO implementation [ref].

Symfony provides some simple functionality to deal with FILE transfer. Nothing around multipart though, from what we found.

HTTP client

Symfony doesn't provide any type of HTTP client.

Licensing and Contribution Terms

Symfony is licensed under the terms of the MIT license. Some methods have been taken from Zend Framework, but this is clearly documented. The project appears to have an open contribution policy which allows any one to freely contribute to the code base. It’s done via pull requests on GitHub, like this: https://github.com/symfony/symfony/pull/1386

Conclusion

Both HTTP components seems to do the job quite well. But the Symfony HTTP Foundation is more self contained and provides a bit richer tools for parsing headers and such. Also, Symfony provides handling of SESSION and FILES as part of their HTTP Foundation. That could be interesting to think about while we're at it.

Here is the drupal.org issue on the matter: http://drupal.org/node/1178246

Comments

functional testing

lsmith77's picture

Just wanted to mention that Symfony2 HttpKernel comes with an HTTP client for functional testing: https://github.com/symfony/HttpKernel/blob/master/Client.php

Most Symfony2 developers use https://github.com/kriswallsmith/Buzz if they need a real HTTP client.

As for multipart its generally supported, though the form related stuff is in the Symfony2 Form component.

As for Session its very extensible via the SessionStorageInterface.

Accept-* parsing

lsmith77's picture

I noticed in the ticket that Accept-* header parsing is mentioned. This is also covered in Symfony2.

Elaborate

Crell's picture

Can you elaborate on what you mean by "covered"? What exactly does it do?

there is

lsmith77's picture

there is https://github.com/symfony/HttpFoundation/blob/master/Request.php#L1023
and getLanguages(), getCharsets() and getAcceptableContentTypes().

Now in order to "negotiate" between the header and the server you will still need some logic. For example I wrote a listener to handle the format negotiations (currently you define a global preference list, but eventually we will allow overriding this on a per controller action basis):
https://github.com/FriendsOfSymfony/FOSRestBundle/blob/master/EventListe...

The implementation of

greggles's picture

The implementation of security controls should be done where security is a problem, such as the storage or presentation layers.

For SQL Injection or XSS and maybe a few other classes of vulnerabilities I do agree with this perspective.

There are at least two security related issus that could be handled earlier in page processing.

First, CSRF is something that would be great to move earlier. I'm not sure whether this is the right point, but some discussion on a proposal to add csrf protection to the menu system could be helpful.

Second, Drupal 7 and below don't do a very good job of ensuring that random extra bits of text at the end of a URL are real/valid. See http://drupal.org/sandbox/pwolanin/1197046 for one solution to the idea.

Symfony2 handles CSRF

lsmith77's picture

Symfony2 handles CSRF protection inside the Form and Security components. XSS is handled by the PHP and Twig template engines and SQL injection is handled by Doctrine2. Some more advanced security measures like cookie signing, https/http redirection is covered by a recently published Bundle https://github.com/nelmio/NelmioSecurityBundle

release cycles

catch's picture

One thing that hasn't been discussed yet is the release cycles of these projects. It looks like the gap between Symfony 1.0 and Symfony 2.0 was about three years (and Symfony 2.0 was just released a couple of weeks ago).

My personal guess for an 8.x release is some time in 2013. We would need to aim to be in sync with the most recent Symfony 2.x version at that point, but we're looking at supporting Drupal 8 until something like 2017 at our current release rate. I personally have no idea how Symfony manages its release cycles, but it would be a shame to have Drupal 8 using something that's only going to be supported for a year or so then be deprecated (as currently happens with jQuery). I'm sure we can deal with this issue but it's something we should try to bake in.

Valid point

Crell's picture

A valid concern. Lukas, what is Symfony's development roadmap, especially with regards to backward compatibility? Or are you like Drupal and eschew the concept? :-)

Currently the plan is to make

lsmith77's picture

Currently the plan is to make the Symfony2.1 release our LTS release with support for at least 3 years (though we might go to 5 years). 2.1 will come out sometime this year. I do not think that this has been clearly announced publicly anywhere.

I can tell you that people in the Symfony community are quite excited about possible cooperation with Drupal. So we can also take a look at supporting the components you guys choose for a longer period potentially. But thats not my call to make.

So the time scales are

catch's picture

So the time scales are interesting. I think we could use more information though, it might help to explain how it works in Drupal as well.

For minor releases of core (7.1, 7.2, 8.1, 8.2) we can only do the following:

  • fix bugs.
  • add new features if they don't break bc (new API functions, extra function arguments, not a lot else)
  • break APIs if it's absolutely necessary to fix a critical bug

While jQuery is completely different in terms of integration, this means that when 8.x comes out we'd aim to be on the very latest stable release, but after release we never update the version that core ships with, and instead there is the contributed jquery_update module that allows you to stay up to date on specific installs.

It looks to me like Symfony might have a different cycle, in that you'd have a 2.0.x bug fix release, then 2.1 could contain new features and API changes? Where it gets interesting for me is in how feasible it would be for Drupal 8 to include new releases from the 2.x series of Symfony, vs. being stuck frozen 2.1 (or 2.3 or wherever). That may or may not be possible depending on the scope of changes.

Major, minor, and bugfix releases

fabpot's picture

One of the fundamental change between symfony1 and Symfony2 is how we will manage BC breaks. For all the components that have a tagged @api, we guarantee that the class/method name, the signature, and the behavior won't change in any version of Symfony 2.x. That's a strong commitment that will give us stability over a very long period of time (and this is indeed very important for the HttpFoundation and HttpKernel components as they represent the low-level architecture of Symfony2 but also other frameworks/applications/products based on Symfony2 -- interoperability is a must here).

For minor versions (2.x), we can add new features and we can break BC but only for things that not tagged with @api, or if there is a security issue that cannot be fixed without breaking BC. 2.x versions will also receive all bug fixes done in 2.0.x releases.

As far as security fixes is concerned, we will apply patches to all LTS versions even after they EOLed.

So this sounds good from the

catch's picture

So this sounds good from the point of view of the Drupal release cycle since it would be at least feasible to updated the component library in a point release.

We tend to point and laugh at people when they ask about release dates for the next major version, but at risk of that happening to me, do you have any vague ideas when work on Symfony3 might start? Even whether that starts within the next two years or not is useful information.

Next major versions of Symfony

fabpot's picture

If we are talking about the core components like HttpFoundation or HttpKernel (or even ClassLoader), I don't see why we would like to have another major release in the next two years. So, it is highly improbably that these components will change drastically in the foreseeable future even if we start working on Symfony3.

The next two years is fine,

catch's picture

The next two years is fine, it's more the window in between a stable release of Drupal 8, but before Drupal 9 comes out where I'd be concerned about getting stuck with something old (not necessarily unsupported, just something where people would potentially want to use the new stuff instead). However it sounds like this will be fine, thanks for the feedback on this!

ESI and reverse proxies

fabpot's picture

You mention that Symfony2 supports ESI (it is implemented in the HttpKernel component). You should also probably mention that the HttpKernel component comes with a pure PHP implementation of an HTTP reverse proxy (and of course the reverse proxy also supports ESI). That means that even if you cannot afford using a proxy like Varnish (think shared-hosting), you can still benefit from ESI. That's also great for testing your caching strategy on your development machines.

From my limited experience we

lsmith77's picture

From my limited experience we always ended up with so much custom configuration in varnish that the internal reverse proxy was only of limited use. However the reverse proxy is certainly helpful, if not for debugging its also a great option to improve performance in setups that cannot afford setting up a varnish reverse proxy.

I looked briefly at the

sdboyer's picture

I looked briefly at the baked-in Symfony2 ESI strategy a few months ago. Seemed to basically be what's done with Boost. Yeah?

Basically ESI (Edge Side

exlin's picture

Basically ESI (Edge Side Includes) is different than Boost.
You might want to check out esi_api module or read more from my recent blog post

Yeah no, I got it. For some

sdboyer's picture

Yeah no, I got it. For some reason I'd read "ESI" and thought "proxy caching." Thanks.

Yikes. nvm, pulled my head

sdboyer's picture

Yikes. nvm, pulled my head out of my ass, read the OP and actually remembered what "ESI" means. Derrr.

lol

mikeytown2's picture

you had me wondering there for a minute.

Symfony2 as a community

fabpot's picture

Being the lead developer of the Symfony project, I can also tell you that we are a very open community, willing to collaborate with other Open-Source projects. As a matter of fact, the foundation of Symfony2 has been thought to be a good starting point for end-user applications like CMSes. The phpBB team has decided to use Symfony2 for their next major version (phpBB4), and it would be a great news for us if Drupal would use some of our components.

One aspect that could

lsmith77's picture

One aspect that could additionally be interesting for Drupal. As Symfony2 essentially transform a Request instance into a Response instance, if Drupal were to adopt HttpFoundation it would mean that it would become trivial to embed Drupal inside a Symfony2 app and vice versa.

Conclusion

Crell's picture

So, I think my recommendation at the moment is going to be for Symfony, for the following reasons:

1) No CLA. I understand why Zend has it, but I really want us to be able to work with the upstream library on improvements and bug fixes, and that CLA is a huge barrier for a community like Drupal.

2) It does more. As above, it looks like Symfony handles more of the necessary processing of the raw HTTP data to make it useful. The less of that we're writing ourselves the better.

3) More robust approach. From the writeup above and talking to Lukas and Fabian, it looks like Symfony takes a much more HTTP-centric view of the world. Considering that is the same direction we're trying to move Drupal, it seems a more natural fit.

4) Collaboration potential. I'll be honest, the fact that we have two Symfony lead developers here in the thread answering questions uninvented is rather impressive, and reassuring. I hope that they continue to be involved, or at least available. :-)

5) More decoupled. We don't know yet if we're going to want to use anything more than the HTTP library from either framework. We may, we may not. The fact that Symfony seems to be more completely decoupled makes it easier for us to cherry pick components later.

Unless I hear a strong reason to the contrary I'm going to officially recommend to Dries that we pull in Symfony's HTTP library along with its autoloader. We can then build our context system atop that. I can sort out the MIT/GPL licensing issues with Dries.

RTBC?

dixon_'s picture

Too me it sounds like most people in this thread would agree on what you are saying here. Good conclusion. Can one mark a g.d.o discussion as RTBC? ;)

Can I agree with you ?

DjebbZ's picture

Can I agree with you ? Especially #4 and #5. Lukas and Fabian, thank you already for your invaluable feedback, and thank you in advance for your collaboration ! Exciting to see 2 big PHP communities getting closer :)

Yes on the HTTP library. I

sdboyer's picture

Yes on the HTTP library.

I think the autoloader may bear some tweaking, though.

Possibly

Crell's picture

Could well be. But we need to start somewhere, and you seem to favor the Symfony autoloader over SplClassLoader. :-) We can always add more classes and/or subclass the Symfony ones as appropriate later. For now, we just need an autoloader that works.

Good thing & great news!

mdhooge's picture

My opinion is obviously biased since I use both drupal & symfony… but I think this is a good news! :-)
And I already dreamt of using symfony components within drupal, but unfortunately I painfully lack time to dig it further in the time being.
Michel

"me too" comment

tmuras's picture

The same here, I'm using both Drupal & Symfony and would love to see as much of Symfony used in Drupal. It would be great if Drupal switched to Doctrine at some point as well but I doubt that would happen too quickly.

I doubt Doctrine would ever

sdboyer's picture

I doubt Doctrine would ever happen. Least I hope it won't. Drupal is generally not a fan of ORMs, and we've got DBTNG. If nothing else, we've invested way too much time in that system to chuck it anytime soon - in particular, getting Views to work with it. And we've got our own schema system that we've been using for years, which has a lot of specific rules (that the whole community knows) on how we manage updates. It'd be a major effort to see any of these things change, and I can't imagine the gains would make it worthwhile.

How about Silex

jmolivas's picture

Drupal + Silex since is based on Symfony2 Components

http://silex-project.org/

--
http://jmolivas.com/
@jmolivas

I do not think that Silex

lsmith77's picture

I do not think that Silex makes that much sense. Silex is essentially the Components that Drupal is considering plus Routing + EventDispatcher + Pimple. Now especially the later is quite central as a simple Dependency Injection Container in Silex (though I guess it could also easily be removed) but the question is then why even bother? That being said, Silex is a good example to play around with to see how the chosen Components behave outside of Symfony2.

I really wonder why both

Damien Tournoud's picture

I really wonder why both Symfony and Zend wrote their own thing here instead of promoting a unified approach in PHP, a la Ruby's Rack and Python's WSGI. Very sad.

Damien Tournoud

Not so sad

mdhooge's picture

Different company, different target business, different history...
Symfony started 6 years ago! Many things were done since that...

And PHP is a really old web scripting language, that lacked for years a real OOP answer. So many different people provided their own way to work around that.

But I don't think it's that sad. Symfony/Fabien has an "opportunistic" behaviour, re-using as much as possible from others. Taking from PHP but also porting from other languages (e.g. http://fabien.potencier.org/article/42/parsing-xml-documents-with-css-se...). So the effort is not wasted re-inventing the wheel.

I think this is that approach that pleases so many developers. And also that symfony manages to federate so many good developers and ideas. And dropping the past to go forward with OOP.

My 2 cents :-)

RE: why both

wilmoore's picture

As one of the people in the ZF community that has instigated this very same discussion a few times, I can tell you why. While there are other reasons (easy to tame), the main reason is that Zend has a CLA and wants to cater to customers that require this. For this reason, ZF and Symfony will likely never be compatible (unless Zend decides to make a fundamental shift in this regard).

I love both frameworks, but I really think Zend is shooting themselves in the foot with this CLA business. I don't mind it so much, but the fact is, it creates a wall in the community an that is going to end up doing ZF more harm over time.