Planetary mutex?

[Marius Gedminas]

Last weekend an Ethernet switch started sulking, and the machine that
runs my Planet from a NFS-mounted filesystem was stuck without
networking for about twelve hours.  When I kicked the switch and logged
in to take a look, I discovered twelve instances of the hourly cron
script, trying to update the planet all at the same time, and playing
havoc with the server's load average.  Naturally I killed them all (with
a simple killall -TERM python), and started a fresh one.

This is where I discovered that Planet uses the infamous Berkeley DB for
its caches, from the "your Berkeley DB is corrupted, mwahahahahah, mine
is an evil laugh!" message (I'm paraphrasing).

  (Aside: I'm so happy I converted all my Subversion repositories to
  fsfs after that one time when Berkeley DB ate the repository where I
  kept my precious home directory backups.  I try to stay well clear off
  BDB since then.)

It wasn't all that hard to recover: rm cache/*; rerun Planet; look at
the total number of entries (eight) in disbelief; read the config file;
discover new_feed_items = 2; realize that I'm aggregating 4 other
planets, and 2 * 4 == 8; bump new_feed_items to 50; rerun Planet and
live happily ever after.

So, when Planet acquires a website with a bugtracker (Trac is nice and
easy to set up; hint, hint), I'll suggest some sort of locking to
protect from several instances of a cron script trying to muck up with
the caches/output simultaneously.

Cheers,
Marius Gedminas
-- 
lg_PC.gigacharset (lg = little green men language, PC = proxima centauri)
	-- Markus Kuhn provides an example of a locale
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: Digital signature
Url : http://lists.waugh.id.au/archives/devel/attachments/20060308/11be5c50/attachment.pgp