TLS/SSL

Varnish needs TLS/SSL.

It's the elephant in the room that nobody wants to talk about.

The world is not the same as it was in 2006. Varnish is used for more and more sensitive sites. A larger percentage of Varnish installations now have some sort of TLS/SSL termination attached to it.

TLS/SSL has been a controversial issue in the history of Varnish Cache, with PHK (Principal architect of Varnish Cache - https://www.varnish-cache.org/docs/trunk/phk/ssl.html) being an outspoken opponent of adding TLS in Varnish. There are valid reasons, and heartbleed has most certainly proven many of PHK's grievances right. But what does that matter when we use TLS/SSL anyway? It's already in the stack, we're just closing our eyes to it.

Setting up nginx in front of Varnish to get TLS/SSL, then nginx behind Varnish to get TLS/SSL... That's just silly. Why not just use nginx to cache then? The lack of TLS/SSL in Varnish is a great advertisement for nginx.

There are a lot of things I dislike about TLS/SSL, but we need it anyway. There's the hitch project (http://hitch-tls.org), but it's not really enough. We also need TLS/SSL to the backends, and a tunnel-based solution isn't enough. How would you do smart load balancing through that? If we don't add TLS/SSL, we might as well just forget about backend directors all together. And it has to be an integral part of all backends.

We can't have a situation where some backend directors support TLS/SSL and some don't.

Varnish Software is already selling this through Varnish Cache Plus, their proprietary version of Varnish. That is obviously because it's a deal breaker in a lot of situations. The same goes for basically any serious commercial actor out there.

So we need TLS/SSL. And we need it ASAP.

Better procedures for VCL changes

With every Varnish version, VCL (The configuration language for Varnish) changes either a little bit, or a lot. Some of these changes are unavoidable due to internal Varnish changes. Some changes are to tweak the language to be more accurate (e.g. changing req.request to req.method, to reflect that it's the request method).

If Varnish is part of your day-to-day work, then this might not be a huge deal. You probably keep up-to-date on what's going on with Varnish anyway. But most users aren't there. We want Varnish to be a natural part of your stack, not a special thing that requires a "varnish-admin".

This isn't necessarily an easy problem to solve. We want to be able to improve VCL and get rid of old mistakes (e.g., changing req.request to req.method is a good thing for VCL). We've also changed the way to do error messages (or custom varnish-generated messages) numerous times. And how to create hitpass objects (a complicated aspect of any cache).

A few simple suggestions:

• All VCL changes reviewed in public as a whole before the release process even starts. To avoid having to change it again two versions down the line.
• Backward compatibility when possible. With warnings or even requiring an extra option to allow it. E.g.: req.request could easily still work, there's no conflict there. Never for forever, but perhaps to the end of a major version. Not everything will be backwards compatible, but some can.

I've had numerous complaints from highly skilled sysadmins who are frustrated by this aspect of Varnish. They just don't want to upgrade because they have to do what feels like arbitrary VCL changes every single time. Let's see if we can at least REDUCE that.

Documentation?

There's a lot of documentation for Varnish, but there's also a lot of bad documentation. Some issues:

• People Google and end up on random versions on varnish-cache.org. No, telling people "but there's a version right there so it's your own fault!" is not an acceptable solution. Varnish Software them self recently had a link in their Varnish Book where they used a link to "trunk" instead of "4.0", whereupon the "here is a complete list of changes between Varnish 3 and Varnish 4" link was actually a link to changes betwen Varnish 4.0 and the next version of Varnish.

• "user guide" and "tutorial" and "installation"? Kill at least two and leave the others for blog posts or whatever. Hard enough to maintain one with decent quality.

• Generated documentation needs to be improved. Example:

  Prototype
          STRING fileread(PRIV_CALL, STRING)
  Description
          Reads a file and returns a string with the content. Please
          note that it is not recommended to send variables to this
          function the caching in the function doesn't take
          this into account. Also, files are not re-read.
  Example
          set beresp.http.served-by = std.fileread("/etc/hostname");
  

  PRIV_CALL should clearly not be exposed! Other examples are easy enough to find.

  In addition, the Description is a mixture of reference documentation style and elaboration. Reference documentation should be clearly separated from analysis of consequences so technical users don't have to reverse-engineer a sentence of "don't do this because X" to figure out what the code actually does.

  And where are the details? What happens if the file can't be opened? What are the memory constraints? It says it returns the content of the file as a string, but what happens with binary content? There's clearly some caching of the file, but how does that work? Per session? Per VCL? Does that cache persist when you do varnishadm stop; varnishadm start? That's completely left out.

• Rants mixed in with documentation? Get rid of "doc/shpinx/phk" (https://www.varnish-cache.org/docs/4.0/phk/) and instead reference it somewhere else. Varnish-cache.org/doc should not be a weird blog-space. It clutters the documentation space. Varnish is not a small little project any more, it's grown past this.

VMOD packages

Varnish vmods are awesome. You can design some truly neat solutions using Open Source vmods, or proprietary ones.

But there are no even semi-official package repositories for the open source vmods. Varnish Software offers this to customers, but I really want it for the public too. Both for my own needs, and because it's important to improve Varnish and VMOD adaption.

Until you can do "apt-get install varnish-vmod-foo" or something like that, VMODS will not get the attention they deserve.

There are some projects in the works here, though, so stay tuned.

TLS/SSL

In case you missed it, I want TLS/SSL.

I want to be able to type https://<varnish host>

BTW: Regarding terminology, I decided to go with "TLS/SSL" instead of either "SSL" or "TLS" after some feedback. I suppose "TLS" is correct, but "SSL" is more recognized, whether we like it or not.