Kristian Lyngstøl's Blog

Magic Grace

Posted on 2015-09-25

I was hacking together a JavaScript varnishstat implementation for a customer a few days ago when I noticed something strange. I have put Varnish in front of the agent delivering stats, but I'm only caching the statistics for 1 second.

But the cache hit rate was 100%.

And the stats were updating?

Logically speaking, how can you hit cache 100% of the time and still get fresh content all the time?

Enter Grace

Grace mode is a feature Varnish has had since version 2.0 back in 2008. It is a fairly simple mechanic: Add a little bit of extra cache duration to an object. This is the grace period. If a request is made for the object during that grace period, the object is updated and the cached copy is used while updating it.

This reduces the thundering horde problem when a large amount of users request recently expired content, and it can drastically improve user experience when updating content is expensive.

The big change that happened in Varnish 4 was background fetches.

Varnish uses a very simple thread model (so to speak). Essentially, each session is handled by one thread. In prior versions of Varnish, requests to the backend were always tied to a client request.

  • Thread 1: Accept request from client 1
  • Thread 1: Look up content in cache
  • Thread 1: Cache miss
  • Thread 1: Request content from web server
  • Thread 1: Block
  • Thread 1: Get content from web server
  • Thread 1: Respond

If the cache is empty, there isn't much of a reason NOT to do this. Grace mode always complicated this. What PHK did to solve this was, in my opinion, quite brilliant in its simplicity. Even if it was a trade-off.

With grace mode, you HAVE the content, you just need to make sure it's updated. It looked something like this:

  • Thread 1: Accept request from client 1
  • Thread 1: Look up content in cache
  • Thread 1: Cache miss
  • Thread 1: Request content from web server
  • Thread 1: Block
  • Thread 1: Get content from web server
  • Thread 1: Respond

So ... NO CHANGE. For a single client, you don't have grace mode in earlier Varnish versions.

But enter client number 2 (or 3, 4, 5...):

  • Thread 1: Accept request from client 1
  • Thread 1: Look up content in cache
  • Thread 1: Cache miss
  • Thread 1: Request content from web server
  • Thread 1: Block
  • Thread 2: Accept request from client 2
  • Thread 2: Look up content in cache
  • Thread 2: Cache hit - grace copy is now eligible - Respond
  • Thread 1: Get content from web server
  • Thread 1: Respond

So with Varnish 2 and 3, only the first client will block waiting for new content. This is still an issue, but it does the trick for the majority of use cases.

Background fetches!

Background fetches changed all this. It's more complicated in many ways, but from a grace perspective, it massively simplifies everything.

With Varnish 4 you get:

  • Thread 1: Accept request from client 1
  • Thread 1: Look up content in cache
  • Thread 1: Cache hit - grace copy is now eligible - Respond
  • Thread 2: Request content from web server
  • Thread 2: Block
  • Thread 3: Accept request from client 2
  • Thread 3: Look up content in cache
  • Thread 3: Cache hit - grace copy is now eligible - Respond
  • Thread 2: Get content from web server

And so forth. Strictly speaking, I suppose this makes grace /less/ magical...

In other words: The first client will also get a cache hit, but Varnish will update the content in the background for you.

It just works.


What is a cache hit?

If I tell you that I have 100% cache hit rate, how much backend traffic would you expect?

We want to keep track of two ratios:

  • Cache hit rate - how much content is delivered directly from cache (same as today). Target value: 100%.
  • Fetch/request ratio: How many backend fetches do you initiate per client request. Target value: 0%.

For my application, a single user will result in a 100% cache hit rate, but also a fetch/request ratio of 100%. The cache isn't really offloading the backend load significantly until I have multiple users of the app. Mind you, if the application was slow, this would still benefit that one user.

The latter is also interesting from a security point of view. If you find the right type of request, you could end up with more backend fetches than client requests (e.g. due to restarts/retries).

How to use grace

You already have it, most likely. Grace is turned on by default, using a 10 second grace period. For frequently updated content, this is enough.

Varnish 4 changed some of the VCL and parameters related to grace. The important bits are:

  • Use beresp.grace in VCL to adjust grace for an individual object.
  • Use the default_grace parameter to adjust the ... default grace for objects.

If you want to override grace mechanics, you can do so in either vcl_recv by setting req.ttl to define a max TTL to be used for an object, regardless of the actual TTL. That bit is a bit mysterious.

Or you can look at vcl_hit. Here you'll be able to do:

if (obj.ttl + obj.grace > 0s && obj.ttl =< 0s) {
        // We are in grace mode, we have an object though
        if (req.http.x-magic-skip-grace-header ~ "yes") {
                return (miss);
        } else {
                return (delier);

The above example-snippet will evaluate of the object has an expired TTL, but is still in the grace period. If that happens, it looks for a client header called "X-Magic-Skip-Grace-Header" and checks if it contains the string "yes". If so, the request is treated as a cache miss, otherwise, the cached object is delivered.


Varnish Wishlist

Posted on 2015-09-19

I recently went back to working for Redpill Linpro, and thus started working with Varnish again, after being on the side lines for a few years.

I've been using Varnish since 2008. And a bit more than just using it too. There's been a lot of great change over time, but there are still things missing. I recently read and while I largely agree with Kacper, I think some of the bigger issues are missing from the list.

So here's my attempt to add to the debate.


Varnish needs TLS/SSL.

It's the elephant in the room that nobody wants to talk about.

The world is not the same as it was in 2006. Varnish is used for more and more sensitive sites. A larger percentage of Varnish installations now have some sort of TLS/SSL termination attached to it.

TLS/SSL has been a controversial issue in the history of Varnish Cache, with PHK (Principal architect of Varnish Cache - being an outspoken opponent of adding TLS in Varnish. There are valid reasons, and heartbleed has most certainly proven many of PHK's grievances right. But what does that matter when we use TLS/SSL anyway? It's already in the stack, we're just closing our eyes to it.

Setting up nginx in front of Varnish to get TLS/SSL, then nginx behind Varnish to get TLS/SSL... That's just silly. Why not just use nginx to cache then? The lack of TLS/SSL in Varnish is a great advertisement for nginx.

There are a lot of things I dislike about TLS/SSL, but we need it anyway. There's the hitch project (, but it's not really enough. We also need TLS/SSL to the backends, and a tunnel-based solution isn't enough. How would you do smart load balancing through that? If we don't add TLS/SSL, we might as well just forget about backend directors all together. And it has to be an integral part of all backends.

We can't have a situation where some backend directors support TLS/SSL and some don't.

Varnish Software is already selling this through Varnish Cache Plus, their proprietary version of Varnish. That is obviously because it's a deal breaker in a lot of situations. The same goes for basically any serious commercial actor out there.

So we need TLS/SSL. And we need it ASAP.


After speaking to PHK, let me clarify: He's not against adding support for TLS, but adding TLS itself. Varnish now supports the PROXY-protool which is added explicitly to improve support for TLS termination. Further such additions would likely be acceptable, always doing the TLS outside of Varnish.

Better procedures for VCL changes

With every Varnish version, VCL (The configuration language for Varnish) changes either a little bit, or a lot. Some of these changes are unavoidable due to internal Varnish changes. Some changes are to tweak the language to be more accurate (e.g. changing req.request to req.method, to reflect that it's the request method).

If Varnish is part of your day-to-day work, then this might not be a huge deal. You probably keep up-to-date on what's going on with Varnish anyway. But most users aren't there. We want Varnish to be a natural part of your stack, not a special thing that requires a "varnish-admin".

This isn't necessarily an easy problem to solve. We want to be able to improve VCL and get rid of old mistakes (e.g., changing req.request to req.method is a good thing for VCL). We've also changed the way to do error messages (or custom varnish-generated messages) numerous times. And how to create hitpass objects (a complicated aspect of any cache).

A few simple suggestions:

  • All VCL changes reviewed in public as a whole before the release process even starts. To avoid having to change it again two versions down the line.
  • Backward compatibility when possible. With warnings or even requiring an extra option to allow it. E.g.: req.request could easily still work, there's no conflict there. Never for forever, but perhaps to the end of a major version. Not everything will be backwards compatible, but some can.

I've had numerous complaints from highly skilled sysadmins who are frustrated by this aspect of Varnish. They just don't want to upgrade because they have to do what feels like arbitrary VCL changes every single time. Let's see if we can at least REDUCE that.


There's a lot of documentation for Varnish, but there's also a lot of bad documentation. Some issues:

  • People Google and end up on random versions on No, telling people "but there's a version right there so it's your own fault!" is not an acceptable solution. Varnish Software them self recently had a link in their Varnish Book where they used a link to "trunk" instead of "4.0", whereupon the "here is a complete list of changes between Varnish 3 and Varnish 4" link was actually a link to changes betwen Varnish 4.0 and the next version of Varnish.

  • "user guide" and "tutorial" and "installation"? Kill at least two and leave the others for blog posts or whatever. Hard enough to maintain one with decent quality.

  • Generated documentation needs to be improved. Example:

            STRING fileread(PRIV_CALL, STRING)
            Reads a file and returns a string with the content. Please
            note that it is not recommended to send variables to this
            function the caching in the function doesn't take
            this into account. Also, files are not re-read.
            set beresp.http.served-by = std.fileread("/etc/hostname");

    PRIV_CALL should clearly not be exposed! Other examples are easy enough to find.

    In addition, the Description is a mixture of reference documentation style and elaboration. Reference documentation should be clearly separated from analysis of consequences so technical users don't have to reverse-engineer a sentence of "don't do this because X" to figure out what the code actually does.

    And where are the details? What happens if the file can't be opened? What are the memory constraints? It says it returns the content of the file as a string, but what happens with binary content? There's clearly some caching of the file, but how does that work? Per session? Per VCL? Does that cache persist when you do varnishadm stop; varnishadm start? That's completely left out.

  • Rants mixed in with documentation? Get rid of "doc/shpinx/phk" ( and instead reference it somewhere else. should not be a weird blog-space. It clutters the documentation space. Varnish is not a small little project any more, it's grown past this.

VMOD packages

Varnish vmods are awesome. You can design some truly neat solutions using Open Source vmods, or proprietary ones.

But there are no even semi-official package repositories for the open source vmods. Varnish Software offers this to customers, but I really want it for the public too. Both for my own needs, and because it's important to improve Varnish and VMOD adaption.

Until you can do "apt-get install varnish-vmod-foo" or something like that, VMODS will not get the attention they deserve.

There are some projects in the works here, though, so stay tuned.


In case you missed it, I want TLS/SSL.

I want to be able to type https://<varnish host>

BTW: Regarding terminology, I decided to go with "TLS/SSL" instead of either "SSL" or "TLS" after some feedback. I suppose "TLS" is correct, but "SSL" is more recognized, whether we like it or not.


Cycling Norway

Posted on 2014-08-14

About 4 years ago I decided to get in better shape. I took my bike for the longest ride I had ridden at the time, which was about 7km I believe. This summer, I took the train to Stavanger and cycled back to Oslo along the coast. All in all, it was a little under 500km and took me a little over a week (8 days of cycling), with the longest single stretch being 131km (final stretch home).


I had done the same trip, more or less, the year before and had learned a lot. But frankly, preparations were pretty simple:

  1. Alert friends and family
  2. Make sure I have a functional bike.
  3. Make sure I've got luggage sorted out.
  4. Pack light.
  5. Book train ticket (Probably the hardest part)
  6. Ride the bike a lot.

I ride a relatively cheap Merida Cyclocross 3 for these sort of trips, which gives a nifty mix of speed and durability. And lets me attach a luggage rack. This year, I was trying out slick tyres, as I knew most of the route was going to be asphalt.

Stage 0: Getting to Stavanger

Strava log:

As it turns out, this was harder than it should have been. NSB (Norwegian Rail company, what the proper English name is is anybody's guess, since the "Company Information" page they provide in English is blank) provide separate tickets for bikes, and you can book it ahead of time. There's a max price which means that for long-ish distances, bringing your bike is pretty cheap (175kr or something like that is the max price). In theory, this is all pretty awesome. However...

  • Problem 1: Can't book bike tickets on-line.
  • Problem 2: Construction work on parts of the final stretch to Stavanger. Buses instead of trains.
  • Problem 3: NSB didn't seem able to book my bike past the point where the buses start replacing the trains. E.g.: Bike on train == fine, bike on bus == ????.
  • Problem 4: Guy at the station doesn't actually know how far the train will go. Sells me tickets for Egersund, while the train now goes to Bryne as I later discover. Tells me to speak to the bus drivers regarding the bike and assumes it'll be no problem to take the bus with my bike, despite no official ticket/booking.
  • Problem 5: Wait, the trains goes to Bryne? That means no buses in Egersund, where I have tickets for.
  • Problem 6: NEXT guy, day later, doesn't realize this is a problem created by NSB, so I have to buy a NEW set of tickets for Bryne...
  • Problem 7: Wait, what, my bike has paid 90,- NOK extra for free coffee, news papers and wifi? And has a seat reserved for it, not a place in the goods carriage?

In short: One big mess.

Thankfully, NSB customer services sorted it all out once I wrote them an e-mail, and I got my extra money back, got a ride to Bryne (where my brother picked me up), and had two seats to myself since my bike still had one reserved...

By the way: I lied, I rode from Forus, not Stavanger. Don't tell anyone.

Stage 1: "Stavanger" - Egersund

Strava log:

In short: Wicked rain followed by great sun, two flat tires on the back wheel, lots of nice, flat terrain, swearing about stupid detours triggered by bicycle routes being made for sightseeing, not transport, then this:


Some times, detours are worth it.

Also ran into parts of what's called "Den Vestlandske Hovedvei" (The western main road). For anyone cycling, please be aware: That's signage-code for "Ridiculously steep hills, bad, unpaved road, road stops at the bottom of hills, completely suboptimal cycling terrain, and also very scenic.



/pics/jpegs/horsy.jpeg /pics/jpegs/horsy2.jpeg

They never moved. I'm not particularly afraid of horses, but I'm reluctant to pass behind horses I'm not familiar with and who are not accompanied by a human, so I ended up going around them in the ditch.


Stage 2: Egersund - Flekkefjord

Strava log:

In short: No rain, until the last 30 minutes, lots of steep hills, both up and down, switch the actual tyre this time after the third tube change, staying with Bendik in Flekkefjord was great(Heck, I had a whole floor to myself).


The Jøssingfjord is famous due to the Altmark Incident (, a skirmish during World War 2 where Norwegian neutrality was breached.

/pics/jpegs/jossingfjord-memorial.jpeg /pics/jpegs/double_roof_small.jpeg

And finally, Flekkefjord:


Not exactly the best weather, but my host was great and good company easily makes up for bad weather!

Stage 3: Flekkefjord - Mandal

Strava log:

In short: Forecast: 30°C and sun. Reality: CONSTANT RAIN.

Started the day by getting a lift from my generous host. Cycling Flekkefjord - Lyngdal using the official cycling route is a gigantic detour since they recently built a large amount of tunnels and a GREAT new bridge across Fedafjorden. None of which you are allowed to cycle on. Then the official cycle route take you along the coast on mostly gravel roads. Wasn't doing that again.

Since the forecast was so good, I figured the little rain I had during the start of the ride was just left-overs from the thunderstorm we had during the night. So I only used a rain jacket. No rain pants. No rain covers for my shoes. BIG BIG mistake, as it was raining through the entire ride. Just as I was entering Mandal the weather cleared up.

As for the route: Good pick. All downhill on GOOD unpaved roads to Lyngdal. One big hill up from Lyngdal, then mixed. The last stretch is steep climbing on wooden roads. If allowed: Cycle on E39! It'll save you a LOT of energy.

Stage 4: Mandal - Kristiansand

Strava log:

In short: GREAT ride. Great scenery, good roads, good weather, little or no traffic. Almost lost the Strava log, but managed to resurrect it when I got home.


Also got to spend the eveing with Vegard and his family, which were nice enough to take me on a boat trip in the Kristiansand area. I even got one of those so called selfies:


Stage 5: Kristiansand - Grimstad

Strava log:

In short: Great weather, great road, then "Vestlandske Hovedvei" again. Yeah, turned out to be an other stretch of steep climbs on bad gravel road. Oh well.


Stage 6: Grimstad - Risør

Strava log:

In short: Nice ride (I finally figured out Tvedestrand), heavy rain during the last 20 minutes left me completely soaking wet, but otherwise happy.


This probably takes care of tailgaters.

Stage 7: Risør - Kragerø (sort of)

Strava log:

In short: Nice ferry ride, by far the shortest ride of the trip (hardly even counts), met an other cyclist on the ferry from Risør and we cycled together to Stabbestad. Good roads, but somewhat boring scenery.


Stage 8: Kragerø - Oslo (sort of)

Strava log: (slightly broken).

In short: "Let's just do this and get home." Ridiculously warm. Dad gave me a lift from Tåtøy to Helgeroa, then Garmin lost the data between Helgeroa and my first pit-stop in Larvik.

Tip: Do NOT follow the official cycle route here if you want to make good progress. It's very scenic and nice, but also very slow as it takes you through forest paths and whatnot. Following my route from Helgeroa to Horten was very fast and easy (flat).

The only picture I really took was from the ferry crossing from Horten to Moss:


(That's Horten in the background).


  • Bike: Merdia Cyclocross 3 (2013 model)
  • Tyres: Continental slicks, then some Maxxis rear tyre.
  • Cheap bags from G-Sport/GMAX. Worked OK, but the lack of a proper stiff plate between the bag and the wheel meant that the bags gradually got closer to the wheels as the trip progressed. Not a problem now, but I probably wont take them for an other long trip.
  • Shoes: Bright orange Giro shoes with MTB cleats/pedals. Bought because they have a Vibram sole, which means two things: Good grip when you're OFF the bike, and you can walk around without sounding like you're wearing slalom boots or something like that.
  • Garmin Edge 810, with maps from (OSM is THE best source for cycling maps). The Edge 810+OSM combo works great for long travel, but I did have to resort to using my phone + Google maps once or twice. All in all not bad. Really nice to have a map in front of you when riding on unknown paths. As for the Edge itself... It shut down spontaneously 4-5 times (good battery), lost the Helgeroa-Larvik stretch and ALMOST lost me the Mandal-Kristiansand stretch (Non-technical users would probably not have realized you could recover it). But I'd still use it again.
  • Lights: Knog lights. This is not for night cycling (we have something like 4 hours of night-time at this time of year), but for tunnels and really bad weather. The type of lights don't matter that much, but you'll feel a lot better if you bring them and then need them.
  • Cloths: I typically ride with a regular cycling bib (Assos) under some sort of loose terrain cloths. I feel better off the bike with "normal" cloths, and it feels good on the bike anyway.
  • Kindle!!!! I read a lot on my rides (well, in the evenings and when I stop for a break, anyway).
  • Camera (Brought a huge SLR this time. Total overkill, but meh)
  • "Civilian cloths": This makes the end points much nicer. The trick is to have overlapping cloths. A fleece sweater works fine when playing cards on a late night, and is a good backup for really cold weather, for example. But proper civilian cloths makes the trip significantly more leisurely (and you're less of a burden to your hosts).

Next time: Smaller camera. Less "Stuff". Stronger rear tyre from the start.


The trip was a really great success. What really made it, though, was all the stops along the way.

I want to thank everyone who let me stay with them, often on short notice.

My next multi-day trip is probably going to be mountain biking, but one thing is certain: I've come a long way since I decided to get in shape 4 years ago.


The Architecture the Varnish Agent

Posted on 2013-02-15

Designing software architecture is fun.

The Varnish Agent 2 was written as a replacement for the original Varnish Agent. They both share the same purpose: Expose node-specific Varnish features to a management system. They are design very differently, though.

In this post I'd like to explain some choices that were made, and show you how to write your own code for the Varnish Agent 2. It's really not that hard.

The code can be found at:

Why C ?

The choice of C as a language was made fairly early. One of the main reasons is that Varnish itself is written in C, as are all the tools for Varnish. This means that the by far best supported APIs for talking to Varnish are written in C.

But an other reason is because C is a very good language. It has become a false truth that you "never write web apps in C", more or less. There are good reasons for this: It takes time to set things up in C, C isn't very forgiving and perhaps most importantly: people generally suck at C.

In the end, we chose C because it was the right tool for the job.


When designing a new system, it's important to know what you're trying to achieve, and perhaps just as important to know what you're /not/ trying to achieve.

The Varnish Agent is designed to:

  • Manage a single Varnish server.
  • Remove the need for management frontends to know the Varnish CLI language.
  • Expose log data
  • Persist configuration changes
  • Require "0" configuration of the agent itself
  • Ensure that Varnish works on boot, even if there is no management front-end present.
  • Be expandable without major re-factoring.
  • Be easy to expand

What we did NOT want was:

  • Support for running the agent on a different machine than the Varnish server.
  • Elaborate self-management of the agent (e.g: support for users, and management of them).
  • Mechanisms that are opaque to a system administrator
  • Front-end code mixed with back-end code
  • "Sessions"

We've achieved pretty much all of these goals.

The heart of the agent: The module

At the heart of the agent, there is the module. As of this writing, there are 14 modules written. The average module is 211 lines of C code (including copyright and license). The smallest module, the echo module, is 92 lines of code (the echo plugin is an example plugin with extensive self documentation). The largest modules, the vlog and vcl modules, are both 387 lines of code.

To make modules useful, I spent most of the initial work on carving out how modules should work. This is currently how it works:

  • You define a module, say, src/modules/foobar.c
  • You write foobar_init(). This function is the only absolutely required part of the function. It will be run in the single-threaded stage of the agent.
  • You either hook into other modules (like the httpd-module), or define a start function.
  • After all plugins are initialized, the start function of each plugin is executed, if present.

That's it.

Since a common task is inter-operation between plugins, an IPC mechanism was needed. I threw together a simple message passing mechanism, inspired by varnish. This lives in src/ipc.c and include/ipc.h. The only other way to currently talk to other modules is through httpd_register (and logger(), but that's just a macro for ipc_run()).

If you want your foobar.c-plugin to talk to the varnish CLI, you want to go through the vadmin-plugin. This is a two-step process:

int handle;

void foobar_init(struct agent_core_t *core)
    handle = ipc_register(core, "vadmin");

This part of the code gives you a socket to talk to the vadmin module. Actually talking to other modules in foobar_init() is not going to work, since the module isn't started yet.

And proper etiquette is not to use a global variable, but to use the plugin structure for your plugin, present in core:

struct foobar_priv_t {
        int vadmin;
void foobar_init(struct agent_core_t *core)
        struct foobar_priv_t *priv = malloc(sizeof(struct echo_priv_t));
        struct agent_plugin_t *plug;
        plug = plugin_find(core,"foobar");
        priv->vadmin = ipc_register(core,"vadmin");
        plug->data = (void *)priv;
        plug->start = NULL;

In this example, we have a private data structure for the module, which we allocate in the init function. Every function has a generic struct agent_plugin_t data structure already allocated for it and hooked on to the core->plugins list. This allows you to store generic data, as the core-data structure is the one typically passed around.


The varnish agent uses a lot of assert()s. This is similar to what Varnish does. It lets you, the developer, state that we assume this worked, but if it didn't you really shouldn't just continue. It's excellent for catching obscure bugs before they actually become obscure. And it's excellent for letting you know where you actually need proper error code.

Let's take a closer look at the generic struct agent_plugin_t:

struct agent_plugin_t {
        const char *name;
        void *data;
        struct ipc_t *ipc;
        struct agent_plugin_t *next;
        pthread_t *(*start)(struct
                            agent_core_t *core, const
                            char *name);
        pthread_t *thread;

The name should be obvious. The void *data is left for the plugin to define. It can be ignored if your plugin doesn't need any data at all (what does it do?).

struct ipc_t *ipc is the IPC-structure for the plugin. This tells you that all plugins have an IPC present. This is to allow you to run ipc_register() before a plugin has initialized itself. Otherwise we'd have to worry a lot more about which order modules were loaded.

Next is *next. This is simply because the plugins are par of a linked list.

the start() function-pointer is used to define a function that will start your plugin. This function can do pretty much anything, but have to return fairly fast. If it spawns off a thread, it's expected that it will return the pthread_t * data structure, as the agent will later wait for it to join. Similar, *thread is used for the same purpose.

Using the IPC

You've got a handle to work with, let's use it. To do that, let's look at the vping plugin, starting with init and start:

static pthread_t *
vping_start(struct agent_core_t *core, const char *name)
        pthread_t *thread = malloc(sizeof (pthread_t));
        return thread;

vping_init(struct agent_core_t *core)
        struct agent_plugin_t *plug;
        struct vping_priv_t *priv = malloc(sizeof(struct vping_priv_t));
        plug = plugin_find(core,"vping");

        priv->vadmin_sock = ipc_register(core,"vadmin");
        priv->logger = ipc_register(core,"logger");
        plug->data = (void *)priv;
        plug->start = vping_start;

vping_init() grabs a handle for the vadmin (varnish admin interface) plugin, and the logger. It also assigns vping_start() to relevant pointer.

vping_start() simply spawns a thread that runs vping_run.

static void *vping_run(void *data)
        struct agent_core_t *core = (struct agent_core_t *)data;
        struct agent_plugin_t *plug;
        struct vping_priv_t *ping;
        struct ipc_ret_t vret;

        plug = plugin_find(core,"vping");
        ping = (struct vping_priv_t *) plug->data;

        logger(ping->logger, "Health check starting at 30 second intervals");
        while (1) {
                ipc_run(ping->vadmin_sock, &vret, "ping");
                if (vret.status != 200)
                        logger(ping->logger, "Ping failed. %d ", vret.status);

                ipc_run(ping->vadmin_sock, &vret, "status");
                if (vret.status != 200 || strcmp(vret.answer,"Child in state running"))
                        logger(ping->logger, "%d %s", vret.status, vret.answer);
        return NULL;

The vping module was the first module written. Written before the varnish admin interface was a module. It simply pings Varnish over the admin interface.

This also illustrates how to use the logger: Grab a handle, then use logger(handle,fmt,...), similar to how you'd use printf().

The IPC mechanism returns data through a vret-structure. For vadmin, this is precisely how Varnish would return it.


ipc_run() dynamically allocates memory for ret->answer. FREE IT.

The logger also returns a vret-like structure, but the logger() macro handles this for you.

Hooking up to HTTP!

Hooking up to HTTP is ridiculously easy.

Let's look at echo, comments removed:

struct echo_priv_t {
        int logger;

static unsigned int echo_reply(struct httpd_request *request, void *data)
        struct echo_priv_t *echo = data;
        logger(echo->logger, "Responding to request");
        send_response(request->connection, 200, request->data, request->ndata);
        return 0;

void echo_init(struct agent_core_t *core)
        struct echo_priv_t *priv = malloc(sizeof(struct echo_priv_t));
        struct agent_plugin_t *plug;
        plug = plugin_find(core,"echo");
        priv->logger = ipc_register(core,"logger");
        plug->data = (void *)priv;
        plug->start = NULL;
        httpd_register_url(core, "/echo", M_POST | M_PUT | M_GET, echo_reply, priv);

This is the ENTIRE echo plugin. httpd_register_url() is the key here. It register a url-base, /echo in this case, and a set of request methods (POST, PUT and GET in this case. DELETE is also supported). A callback to execute and some optional private data.

The echo_reply function is now executed every time a POST, PUT or GET request is received for URLs starting with /echo.

You can respond with send_response() as demonstrated above, or the shorthands send_response_ok(request->connection, "Things are all OK!"); and send_response_fail(request->connection, "THINGS WENT BAD");.


Currently all http requests are handled in a single thread. This means you really really shouldn't block.

But make sure it's written with thread safety in mind. We might switch to a multi-threaded request handler in the future.

Know your HTTP

"REST"-interfaces are great, if implemented correctly. A short reminder:

  • GET requests are idempotent and should not cause side effects. They should be purely informational.
  • PUT requests are idempotent, but can cause side effects. Example: PUT /start can be run multiple times.
  • POST requests do not have to be idempotent, and can cause side effects. Example: POST /vcl/ will upload new copies of the VCL.
  • DELETE requests are idempotent, and can have side effects. Example: DELETE /vcl/foobar.

Test your code!

Unused code is broken code. Untested code is also broken code.

Pretty much all functionality is tested. Take a look in tests/.

If your code is to be included in an official release, someone has to write test cases.

I also advise you to add something in html/index.html to test it if that's feasible. It also tends to be quite fun.

Getting started

To get started, grab the code and get crackin'.

I advise you to read include/*.h thoroughly.


The Varnish Agent 2.1

Posted on 2013-01-31

We just released the Varnish Agent 2.1.

(Nice when you can start a blog post with some copy/paste!)

Two-ish weeks ago we released the first version of the new Varnish Agent, and now I have the pleasure of releasing a slightly more polished variant.

The work I've put in with it the last couple of weeks has gone towards increasing stability, resilience and fault tolerance. Some changes:

For a complete-ish log, see the closed tickets for the 2.1 milestone on github.

This underlines what we seek to achieve with the agent: A rock stable operational service that just works.

If you've got any features you'd like to see in the agent, this is the time to bring them forth!

I've already started working on 2.2 which will include a much more powerful API for the varnishlog data (see docs/LOG-API.rst in the repo), and improved HTTP handling, including authentication.

So head over to the demo, play with it, if you break it, let me know! Try to install the packages and tell me about any part of the installation process that you feel is awkward or not quite right.