Charging clients for when Drupal security updates cause incompatibility issues

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
itomic's picture

(please note that I write this article as a business owner, not an experienced Drupal dev!)

Our company charges an annual fee for identifying and applying security updates/patches for the Drupal sites we've designed, built, host and maintain.

A small % of the security updates we apply cause minor or major incompatibility issues with the site. To resolve these issues might take a few minutes, or a good few hours. My understanding is that incompatibility issues are more likely to arise on Drupal sites that have been heavily customised, compared with sites that are very 'standard', i.e. non-custom-coded. But even where a site contains little or no custom code, the potential for incompatibility issues still exists.

We're comfortable that our current pricing for identifying and applying security updates covers us for just that - identifying and applying the updates. What is does not cover us for is when these incompatibility issues arise. This represents a significant risk for us, because of course it's difficult to insure ourselves against an event that is unknown in advance in terms of size (labour hours) and frequency.

So to better insure ourselves against this incompatibility risk, we're planning on adjusting our policy to state that, if/when any incompatibility issues arise, then the resolution of the issues is chargeable to the client on a time & materials basis (at some agreed rate), i.e. on top of the fixed priced we're already charging our clients for the base service (identification and application of updates).

I've spoken to a couple of current clients on this topic and proposed pricing+policy... one gets it, one doesn't. Putting myself in the shoes of a client, I can see how this might make some of them nervous, because now there's a (slightly) "blank cheque" aspect to the deal, i.e. they now face the prospect of receiving invoices from us to fix incompatibility issues that, for example, we say took 3 hours to resolve - and they just have to take our word for it.

I'm very keen to learn how others deal with this issue from a pricing and policy perspective.

Thanks,

Ross.

Comments

I cross-posted this to the

greggles's picture

I cross-posted this to the consulting group because it's more of a business question than a security question. The same question applies to the situation where 5.x became unsupported and clients are urged to upgrade or a module is marked unsupported (for any reason) and clients become a one-off situation.

I'm not aware of anyone who has a great solution to this. One thing you could do is make sure you have tests (preferably automated) that cover the major uses of the site. When you upgrade a contrib or core you can run those tests to reduce the effort in verifying whether a new release will work well.

Our company charges an annual

Shai's picture

Our company charges an annual fee for identifying and applying security updates/patches for the Drupal sites we've designed, built, host and maintain.

I use this same approach as a feeelancer with very small clients. I charge yearly (and at a fixed price determined in advance) for that set of services.

My whole sales approach on this method is that my clients won't have to worry (both about problems with sites AND about unexpected costs). This is one of several reasons they are paying a lot more than the $125/year shared hosting they often had prior to coming to me. They just want their site to work; my clients are generally too small to have tech people on their paid staffs, and so I'm providing an end-to-end service for them.

I think the answer to the question is in pricing appropriately for the all-in-one hosting/code-security product, and then potentially raising the yearly inclusive hosting + security price, but only yearly. Think of yourself as an insurer; if a bad accident happened that you did have to pay for (e.g. a nasty contrib upgrade mess), you could recoup some of your loss through increased fees in the future.

One thing I do some of but want to do more: as I am working on the initial proposal, I want to more explicitly tie the complexity of a solution to increased annual hosting/cost-security fees. I'm pretty good at explaining that more complexity/code customization will likely lead to more fiddling which will mean ongoing expensive programming costs. But I also want to start emphasizing that there are also increased hosting/security etc. costs as well when implementing relatively more complex solutions and/or adding custom coding.

The bigger unsolved issue for me is Drupal 8. If Drupal 8 comes soon, I'm dead in the water. My customers are not going to be able to afford major upgrades (based on my limited and fraught experience so far upgrading sites from 6 to 7). I'm hoping for at least 2 more years before Drupal 8 is released. Or, as an alternative, if the Drupal community decided to continue to support Drupal 6 even with Drupal 8 released, I would be overjoyed. If, however, Drupal 8 comes out in the next year and Drupal 6 becomes unsupported, I don't have a clue as to what I would tell my customers who are on Drupal 6.

You might say, "duh, this release/support approach is not new." I think I have overestimated how much my clients can actually spend on their web projects and significantly underestimated how hard it would be to implement a major D6 - D7 upgrade.

But there must be a lot of other people in my situation.

I'm sorry if I've gotten off track a bit... it is connected to the original post, but maybe deserves its own thread.

best,

Shai

Shai Gluskin
Content2zero

I do this as well, but when I

gordon's picture

I do this as well, but when I offer these support contracts to customers I tell them it is for minor upgrades and major upgrades like going from Drupal 7.x to 8.x is not included.

That kind of upgrade is most of the time not necessary and right now unless they are on Drupal 5.x it is not needed to upgrade to the latest version.

If they are on 5.x then you can't reliable offer support since it is no longer supported. so other than just charging for hosting you can't put them on a support contract, and any issues will be time and materials.

--
Gordon Heydon

Michael Hofmockel's picture

With every build contract I offer a support contract (retainer). I recommend a certain number of hours per month based on the complexity of the site at a reduced rate. It is then up to the client to decide how many hours they would like to buy. If they don't have enough hours because of a major conflict I advice the client that they have run out of hours and they will be charged a higher rate. In this way they run the risk, not me.

In order to sell a complete coverage package you must pad sufficiently to cover these kinds of issues and even then it is a educated betting game on your side. Propose it to them both ways and they will take the cheaper route where they take on the risk every time.

Regards,
Michael Hofmockel

Open Source || Open Access || Open Mind

Mixed approach.

pearcec's picture

I am very upfront about explaining the potential risk. I explain to them our solution for mitigating it. Like Michael, I recommend they set aside a certain amount of hours a year, something like 6-32 depending on what they are looking to do. This point is key, it lets them know they should expect to spend ongoing money to maintain and fix minor issues with their site.

We do quarterly updates at a fixed price. If they don't want to do the updates, we provdide ad-hoc updates at a greater cost. I explain that if you keep current it is less likely you will run into a severe issue. If you wait, it is more likely a critical remote security attack will come out, we upgrade your site that hasn't been touched in 2 years, and you are faced with greater costs.

We do the upgrade in a staging environment. They have to sign off on it. Then we do the upgrade. We also keep a copy of the old site up and running. So we don't run into the situation of: "But it worked before." Having the old site running makes it real easy to prove it right or wrong. If we find something quick and easy, we usually just take care of it. If not we provide them with an estimate and let them decide. Perhaps they want to do away with functionality, or change functionality while they are spending the money.

So far this approach has worked. But seems few if any of my customers want to pay the regular maintenance. But at least we had the conversation and tried to do our best to inform them.

--
Christian

@pearrec, are your clients'

Shai's picture

@pearrec, are your clients' sites on your server or theirs?

Shai Gluskin
Content2zero

They are on ours. They have

pearcec's picture

They are on ours. They have to be dialed into our process. We tried taking on a couple sites we didn't build. That was a train wreck.

--
Christian

"They have to be dialed into

Shai's picture

"They have to be dialed into our process." Right, that is the same for me. Are all of your clients on their own dedicated servers? Because it sounds like you give them the choice to not buy into your quarterly plan. In my case I have almost 15 clients on one server. They do not share any Drupal code, but they are sharing resources. I feel that I would be risking some clients if I didn't require them all to stay updated on security upgrades.

I just couldn't stand those repeated explanations of why I wanted to "upgrade" their site even though they wouldn't see any benefit. I'm not averse to explaining; I came from being a teacher. But it was a waste of time. I'd rather make that sales pitch as part of my getting the client initially and say, "this is the way I do business... here are the benefits." They sign on. Done. The yearly hosting/code security bill provides a yearly opportunity to review this.

But like the original poster, the gray areas are the hardest, like when a module security upgrade breaks the site. In my case, because I've made the initial sale on providing an end-to-end solution, if they aren't getting any new benefit, I have to pay for it.

Shai Gluskin
Content2zero

Thanks for your comments,

itomic's picture

Thanks for your comments, people. Sounds like others are grappling a little towards a "perfect solution", which of course isn't really possible in the sense that some clients understand and empathise with the issue, and others less so, despite best efforts to educate them.

But certainly up-front education is critical.

And keeping a (temporary) copy of the old site in an attempt to better manage the "but it worked before" question sounds smart.

Thanks again.

p.s. I'd be interested to

itomic's picture

p.s. I'd be interested to learn from others about the frequency with which a security update causes incompatibility issues with an existing site. How often does this happen (e.g. 10% of the time on average?), and whether, like us, you find that the solution might take only a few minutes to implement, or a good few hours. Anything more you/we can do to mitigate against this risk from a technical perspective, as opposed to a billing perspective? Because of course if we can minimise this risk from a technical perspective, then it reduces the need for an billing policy around it.

Periodic fee

joseonate's picture

We offer a flat periodic fee to keep the site updated with security patches, and also with regular module updates at our discretion (if we find that a non-critical module update generates conflict, we’ll often revert to the previous version until further review.)

Each case is unique. We base our fee mainly on the following two parameters:
1. Risk: How complex is the website?
Amount of modules + amount of obscure modules + custom stuff = higher risks of issues and incompatibilities.

  1. How often will we perform updates?
    When updates are performed regularly, it is easier to keep a handle on the site and there’s less work to do on each update round.

From this analysis we do an estimation of work hours that includes an expectation of running into problems and finding solutions. For the client, there is peace of mind in knowing that the site is kept up to speed at a regular periodic cost that is easy to put in the budget.

Needless to say, all of this requires being familiar with the website at hand. So far we have only offered this arrangement to our existing clients.

Likelihood of incompatibilities:
I have personally run into issues caused by a module update more times than I care to remember. I don’t know that I can give you an intelligent percentile estimate of the likelihood of problems, but they are common enough that I have not considered relying on some automated process to perform module updates.

Fixed-fee isn't viable/sensible IMO

Andy Inman's picture

Interesting topic. I've only recently started offering general server/site maintenance services (in direct response to demand). Previously I only offered to maintain my custom code, and that done at an hourly rate (example upgrading a whole bunch of custom modules from D5 to D6.)

My approach with the relatively few sites I'm now maintaining is partially a technical one - all have a staging and live site - a duplicated installation (separate Drupal code and database.) So we run all upgrades on the staging site before touching the live site, if anything obvious breaks the live site is intact, (albeit perhaps insecure!) Then decisions can be taken relatively at leisure. Of course, it's always possible that something less than obvious gets broken, in which case we may not notice until it's "too late", but this approach at least avoids WSOD and similar problems where you then need you to rush around madly trying to fix it or restore from backup.

In practice, security updates are not always vital - quite often the particular situations needed to exploit a vulnerability doesn't/can't exist on a specific site. Many small sites have only a handful of registered users and all are trusted - many vulnerabilities are only exploitable by a logged-in user.

So, for now I charge for everything on a time basis (with a minimum hours retainer.) Of course clients want everything to be fixed-price, but it's not always viable. What about OS and LAMP upgrades? They might break a Drupal site too, albeit relatively unlikely. I don't think I would ever want to offer a fixed-price irrespective of what might break. I would put a disclaimer in the agreement, definitely.



Currently part of the team at https://lastcallmedia.com in a senior Drupal specialist role.

We do a mix of the

freelock's picture

We do a mix of the above:

  • Flat rate maintenance plans to apply updates, which includes deploying to a test/stage copy of the site first. We treat it like insurance, and we exclude interactions with custom modules -- e.g. if you have a custom module that breaks as the result of an upgrade, we're charging hourly to fix it. Otherwise, as long as the module maintainer has a current release and an upgrade path, it's on us to get it working if it breaks as the result of an upgrade.

  • Retainer hours for maintaining custom items, at a reduced rate for however many hours customers want to reserve on our calendar. Basically if a security upgrade breaks something, and it's related to something custom, we provide some choices: Turn off the broken functionality, or pay to fix.

Oh, and we also require that the site is on a server where we can install our deployment tools, we configure it to pull from our company git repo for most standard modules, and a site must have a baseline review/initial project for us to get familiar with how it's put together and properly document unusual configurations.

We like fixed fees for maintenance because it incentivizes us to develop tools and processes to make the job go quicker, properly protect the site and be able to roll it back, and generally spreads the cost of fixing broken upgrades across a much broader customer base.

And we're picking up quite a few customers who see value in our approach -- we just picked up a bunch from a PR firm who had a customer site hacked, came to us to clean them up and protect them... When we went ahead and applied the recent CKEditor update when it came out right after we had finished cleaning up one hacked site, they suddenly saw the value in having us manage it!