Venus and a single cache directory?

[Sam Ruby]

Jeff Waugh wrote:
> <quote who="Sam Ruby">
> 
>> So... if you had (for example) three config.inis which specified three
>> (possibly overlapping) sets of feeds, but specified the same
>> cache_directory, and you defined a fourth config.ini which contained the
>> union of feeds and specified the same cache_directory, you could "spider"
>> the latter (fetching the data), and then serially "splice" using the
>> original definitions (producing the output).
> 
>>From this paragraph, it sounds like sharing the cache directory results in
> the same behaviour we get with Planet 2.0 today -- after retrieving the feed
> for the first time, subsequent processes will benefit from I-M-S and E-Tag
> optimisation, but still do the request. The rest of your email describes a
> way of dodging multiple requests. Right?

That behavior is still retained by Venus.  Most, but not all, feeds 
these days support either I-M-S or E-Tag.

Additionally, you Venus provides additional options.  If, for example, 
you want to refetch a single feed and republish your planet, you can do 
the following:

   python spider.py config.ini http://example.com/feed.xml
   python splice.py config.ini

- Sam Ruby