Policy versus mechanism

• We design mechanisms and leave policy to users
• What policy? (I don't know, I just work here)

Examples

/etc/exim4/update-exim4.conf.conf

Do we get a update-update-exim5.conf.conf.conf in the next release?

kristian@kit:/etc/apache2$ find -type f | wc -l
109
kristian@kit:/etc/apache2$ grep -v '^ *#' *conf conf.d/* \
        sites-enabled/* mods-enabled/*  | wc -l
526
(for hello-world)

$ egrep -v ' *#' /usr/share/compiz/*xml  | wc -l
76898

kristian@kit:/etc/exim4$ find -type f | wc -l
43

Apologies

Exim, Apache and Compiz are just random examples, nothing personal!

Example - Varnish

My private server with special cases for munin, stress-testing, debian/ubuntu-mirroring, purging, couple of comics, and more -- 5 vastly different backends:

kristian@kit:/etc/varnish$ grep -v '^ *#' default.vcl | wc -l
94

kristian@kit:/etc/varnish$ ls
default.vcl  default.vcl.dpkg-dist  secret

Varnish Configuration Language

• Tiny language for Varnish
• Translated to C, compiled and linked at run time
• Mostly readable without an extensive introduction (in the “I can read <language>, but I can't really write it.”-sense)
• Simple constructs to build complex policies
• Escape chute: In-line C in VCL!
• Super-fast

Implement mechanisms

• What does the software really do internally?
• Expose it!
• The answer to "Should we support A or B?" is almost always "Both", "neither" or "POTATO POTATO POTATO TOMATO!".
• The more power you give to the configuration mechanism, the more you can focus on the hard problems.
• Good managers/developers make good decisions. Smart managers/developers ensure that they don't have to.

Some examples

• Hot-link-blocking:

  if (req.url ~ "\.jpg" &&
           req.http.referer !~ "^http://mysite.example.com") {
          error 403 "Hot-linking forbidden";
  }
  

• Rewriting:

  set req.url = regsub(req.url,"^/sprot/","/sport/");
  

More examples

backend lo { .host = "localhost"; }
backend backup { .host = "varnish-cache.org"; }

sub vcl_error {
        if (req.restarts < 5) {
                return(restart);
        }
}

sub vcl_recv {
        if (req.restarts > 4 &&
                        req.url ~ "^/cool-stuff/" &&
                        req.http.user-agent !~ "IE6(?i)")
        {
                set req.backend = backup;
        }
}

The result

varnishd -C -f foo.vcl

Varnish Run-Time (VRT)

• Vcl does not fiddle directly with internal data structures
• VRT provides an API and abstraction-layer between VCL and varnishd
• bin/vanrishd/cache_vrt.c
• lib/libvcl/* for normal VCL

Why in-line C?

• Because we can
• It's simple
• It's cool
• For obscure or complex policy-related decisions we can leave it to the user to implement it himself (GeoIP, syslog, cryptographic analysis of input-data, etc)
• Acknowledgement that we will never be able to predict all the possible ways people want to configure Varnish

How?

backend vc { .host = "varnish-cache.org"; }

C{ #include <stdio.h> }C
sub vcl_recv {
        C{
                printf("Hello mom, I'm on the internett!\n");
        }C
        set req.http.host = "www.varnish-cache.org";
}

How it works

1. Varnish reads a VCL file, translates it to C
2. Compiles it
3. Links against it
4. Uses it/runs (..)
5. A request arrives
6. Varnish parses the HTTP
7. VGC_function_vcl_recv() is run - as specified in VCL
8. The compiled VCL does it's thing, calls VRT_*() functions to modify data
9. Upon completion, VGC returns control to varnishd telling it where to go next.
10. Varnish looks the item up in cache (..)
11. Cache miss or hit? Run VGC_function_vcl_{miss,hit}()
12. ... etc etc

Varnish 3.0

• VMODS!
• Extends VCL with modules written in C

Questions?