Re: DNS primer

[Dave Howe <[EMAIL PROTECTED]>]

Roland Perry wrote:
> I'm struggling to 
> reconcile your description of a secondary that both isn't updated unless 
> there's a need, and which also automatically gets its data from the 
> master. 
This is actually a feature of the protocol. The secondary queries the 
master every 'n' seconds - definable in the zone, on a per-zone basis.
here is an example start-of-authority record for an random zone:
linuxops.net.  38400  IN  SOA  ns1.linuxops.net. hostmaster.linuxops.net. (
        2004091902  ; Serial
        28800  ; Refresh (8 hours)
        7200  ; Retry (2 hours)
        604800  ; ExpTTLire (7 days)
        86400 )  ; TTL (24 hours)

and to break it down:
> linuxops.net.
(name of zone, note trailing period)

> 38400
TTL override for this one record - in this case, 640 minutes (don't ask 
me why, this isn't my zone)

> IN  SOA
record type - Start of Authority

> ns1.linuxops.net.
primary nameserver - note trailing period again, and note this may need 
a glue record as it is below the waterline.

> hostmaster.linuxops.net.
responsible person - in this case, [EMAIL PROTECTED]

> ( 2004091902  ; Serial
Unique serial number - normally date of last update in ISO order, 
followed by a single digit to allow up to 10 updates per day

> 28800  ; Refresh (8 hours)
how often a secondary should check for updates - every 8 hours in this case.
> 7200  ; Retry (2 hours)
how often a failed request should be repeated

> 604800  ; Expire (7 days) how long a record should be distributed 
from "stale" cache if it cannot be refreshed (applies to both cache 
servers and secondaries)

> 86400 )  ; TTL (24 hours)
record expiry for caches

now, once every <refresh> minutes, a secondary will go and check the SOA 
for the zones it is secondary for. if the serial number differs (and is 
larger than the one it has) it will re-download the zone; otherwise, it 
will reset its counter and carry on happily.
The new DNS-SEC stuff includes digital signature support, which adds a 
few more wrinkles, but on the whole this has been the Way Things Work 
for over a decade that I know of, and probably longer


> The maintainer has to make decisions about when the data is 
> updated, and where from - and as I described before it's traditionally a 
> snapshot process, not a dynamic database.
true enough, but as we are talking a DNS style index of data, rather 
than data itself, a snapshot is probably fine. how often do you invisage 
the "master server" for a given patient's record will change?

> > In theory, the cache expiries are specified explicitly by the owner of
> > the zone - in practice, ISPs often hardcode this in their proxies for
> > their own benefit.
> Bending the rules like that (or not) is not relevant to the mechanisms 
> for updating the zones.
No, they are - because if your caches honour the author's wishes, he can 
set the refresh expiry interval to a reasonable compromise between load 
and freshness.

> > no, it doesn't, and change propagation can be a bugbear of major DNS
> > changes; a propagation time of two or three *days* is not uncommon,
> > although few ISPs push the envelope for DNS expiry beyond that. However,
> > the fact that *your* isp's cache of a record may be inaccurate is not
> > the fault of that record's owner, and in fact it is possible to bypass
> > your isp's cache and obtain data from primary or secondary servers 
> > directly.
> Glad we agree.
I am not sure we do. you seem to believe that the staleness problem is 
entirely due to the system itself, rather than the fact several ISPs 
abuse the protocols by ignoring the exipiry times for their own benefit.

> > yes, it is. if your records change rapidly, you specify a short expiry -
> > one hour, say - so that no record in cache can live more than one hour
> > before being refreshed from the master. as this is an index rather than
> > a data source, such delays are acceptable - you can live with having to
> > not switch off a webserver for two days if that is how long you continue
> > to receive requests after the record goes stale, but there is always a
> > tradeoff between freshness of records and bandwidth usage, and the DNS
> > system does well.
> So you think the DNS would be a suitable prototype for a distributed NHS 
> database [remember, that's the proposition we are discussing] if it was 
> set to a cache time of (say) 10 minutes [the time it takes you to walk 
> from the prescribing GP to the pharmacist].
Certainly. remember, the DNS aspect is merely an index record to tell 
you where to find the data - actually obtaining the data is done using a 
different protocol, or often a choice of different protocols.
As the locate-a-server glue for the internet, DNS is surprisingly 
resilient; it handles index lookup for online and offline protocols 
(such as HTTP and email) and is being extended into protocol-specific 
support, digitally signed record support (for prevention of spoofing 
attacks, not that you are likely to see them) and is proposed as a 
repositary for authenticated email and a few other things.

> > > My DNS servers do none of this. It may well be that some DNS servers
> > > are migrating to such a system, and that's why I qualified my earlier
> > >  remarks with:
> > depends on what dns software you are using - its a feature of bind 9,
> > and not always turned on.
> Glad we agree.
Again, secondary polling has been in place for generations - its only 
the secondary notification that has been recently added, and is a "turn 
on" feature for bind 9.

> > Then whomever set up your DNS server is an idiot (and I can say this
> > unreservedly; automatic secondary updating by pull architecture has been
> > a feature of a properly configured DNS install for more years than I can
> > conveniently remember, certainly since the days I was still using bind 
> > v4)
> Not my DNS server, but those of (eg) cctlds.
For the TLDs, then possibly so - there are advantages to using a non-DNS 
upload protocol for really, really large zonefiles exported from a 
database, and that is how most TLDs are indeed handled. however, note:

co.uk.  25102  IN  SOA  ns1.nic.uk. hostmaster.nominet.org.uk. (
        2004110601  ; Serial
        7200  ; Refresh (2 hours)
        300  ; Retry (5 minutes)
        2419200  ; Expire (28 days)
        172800 )  ; Minimum (2 days)

the final entry for TTL specifies that a cache may hold any given record 
for two days - so it is quite possible (and even within spec) for your 
query to *seem* to take two days to update, but not actually do so. Any 
secondaries that use this
I am not aware of a manual update process being used for co.uk - but it 
is possible; a lot of the really large TLDs are database generated, and 
it is simpler to farm the output out directly to the nameservers than to 
rely on the autoupdater pulling a huge file by DNS protocols twice per 
day. co.uk appears to have nine servers to hold its records, so that 
would save quite a lot of dns traffic on the master. just because they 
don't always use the mechanism though, doesn't mean that they *have* to 
manually update, nor that the mechanism is faulty in any way. I can try 
and find out just how co.uk is updated, but TBH I am not sure I know 
anyone at that TLD to ask :)

> > > > DNS as an index though is pretty static - and a DNS record will
> > > > *even worst case* be updated and propagated across the world faster
> > > > than that packet of papers can make it from your old GP to your new
> > > > one.
> > > At 21 days I think it might be a tie; but it's not about propagating 
> > > single copies, is it? It's about being in the casualty department and
> > >  them being able to access the constantly updated central record
> > > which will reveal what exactly pills it was that the GP prescribed 4
> > > hours ago that you are now suffering the serious allergic reaction
> > > to.
> > I would be interested in what ISPs actually cache dns data for three
> > weeks in the face of a stale copy - I have heard of some badly
> > configured dns zones that *specify* a record expiry of three weeks, in 
> > the mistaken belief it makes their servers somehow more stable, but 
> > never seen it myself and considered it a urban myth...
> Read about the post 9/11 situation, then. As a major source of 
> protracted-loss-of-connectivity it rattled quite a few cages.
just done a little googling, and it seems to confirm my own fairly vague 
memories of the incident - that several sites had trouble because their 
listed nameservers were unreachable after the WTC went "off air" and a 
lot more found that rushing in new servers on new addresses were bitten 
by cascade lookup problems - TLDs being cached for days, per-server 
records cached for weeks - all as specified in their "oh, this is ok as 
it never changes" DNS records.
I don't recall (and can't find) any mention of sites being out for over 
two weeks - even those forced to rebuild due to loss of hardware - but 
presumably you can provide a few URLs for my enlightenment?

> I'm not sure how useful that is, given that everyone will know that the 
> record in question is in a bunker in the East Midlands (and its backup).
In the context of the original question - ie, could a distributed 
database where the GPs hold on to their own records locally actually 
work - they wouldn't be in such a bunker, but possibly the index that 
links them to the hospital records and/or blunkettcard would be.

> Or were you thinking that every doctor or hospital department I'd ever 
> visited would hang onto its fragment of my medical record on a local 
> server, and these would somehow get patched together into a coherent and 
> holistic view by some master-index?
why not? unless you need all records to be centralized for some other 
reason, it is as viable an approach as the master-bunker one, allows 
local authorities to retain access locally (which, lets face facts here, 
is where it will be of benefit to the patient) if the network as a whole 
faces disruption, and allows neighbouring authorities to arrange their 
own replication and disaster recovery by interdependency contracts 
rather than (almost certainly) funding a centralized solution with no 
idea of how they are supposed to access it if it suddenly pings out one 
january morning....
The internet has proven to be a resilient carrier for very similar 
traffic to that proposed; diverse path and backup link management is 
well understood, most ISPs offer it for relatively low cost, and it is 
proven technology based on open standards. Even several formerly 
"secure" private networks have seen the advantages of switching to 
either VPN or similar layered protocols over the public network (eg, 
EDI) and fully integrated into VoIP and data networks, many metropolitan 
authorities could replace their primary links with microwave links for 
long term savings and vastly increased data bandwidth.
That is not to say the NHS should just use the internet - but the same 
ISPs now offer ADSL packages on a Virtual internet circuit, where only 
employees of the same company can "see" each other, and of course their 
employer's firewall.... and not the internet at all.