Too big for its boots

In a delightful way, from a writing perspective, my last few data analysis topics have first synergised with one another and then led naturally on to the consideration of Big Data.

What exactly is "big data"? The answer, it should come as no particular surprise to hear, is "it depends". As a broad, rough and ready definition, it means data in sufficient volume, complexity and velocity to present practical problems in storage, management, curation and analysis within a reasonable time scale. In other words, data which becomes, or at least threatens to become in a specific context, too dense, too rapidly acquired and too various to handle. Clearly, specific contexts will vary from case to case and over time (technology continuously upgrades our ability to manage data as well as generating it in greater volume) but broadly speaking the gap remains – and seems likely to remain in the immediate future. The poster boys and girls of big data, in this respect, are the likes of genomics, social research, astronomy and the Large Hadron Collider (LHC) whose unmanaged gross sensor output would be around fifty zettabytes per day.

There are other thorny issues besides the technicalities of computing. Some of them concern research ethics: to what extent, for example, is it justifiable to use big data gathered for other purposes (for example, from health, telecommunications, credit card usage or social networking) in ways to which the subjects did not give consent? Janet Currie (to mention only one recent example amongst many) suggests a stark tightrope with her "Big data vs big brother" consideration of large scale pædiatric studies. Others are more of concern to statisticians like me: there is a tendency for the sheer density of data available to obscure the idea of a representative sample– and a billion unbalanced data points can actually give much less reliable results than thirty well selected ones.

Conversely, however, big data can also be defined in terms not of problems but of opportunity. Big data approaches open up the opportunity to explore very small but crucial effects. They can be used to validate (or otherwise) smaller and more focussed data collection, as for instance in Ansolabehere and Hersh’s study ^[1] of survey misreporting. As technology gives us expanding data capture capabilities at ever finer levels of resolution, all areas of scientific endeavour are becoming increasingly data intensive. That means (in principle, at least) knowing the nature of our studies in greater detail than statisticians of my generation could ever have dreamed. A while back, to look at the smaller end of the scale, I mentioned ^[2] the example of an automated entomological field study régime simultaneously sampling two thousand variables at a resolution of several hundred cases per second. That’s not, by any stretch of the imagination, in LHC territory but it’s big enough data to make significant call on a one terabyte portable hard drive. It’s also a goldmine opportunity for small team or even individual study of phenomena which would not long ago have been beyond the reach of even the largest government funded programme: big data has revolutionised small science.

There is, in any case, no going back; big data is here to stay – and to grow ever bigger, because it can. Like all progress, it’s a double edged sword and the trick as always is to manage the obstacles in ways which deliver the prize. [more]

---

[1] Ansolabehere, S. and E. Hersh, Validation: "What Big Data Reveal About Survey Misreporting and the Real Electorate". Political Analysis, 2012. 20(4): p. 437-459.

[2] Grant, F., "Retrieving data day queries", in Scientific Computing World. 2013, Europa Science: Cambridge. p. 10-12..

Pretty as a picture

Talking about statistical work by nonstatisticians, recently ("Stats for the million", 14 June), I mentioned the importance in that context of graphical visualisation of data. It goes well beyond that, however.

On the one hand, fuelled by the ever-accelerating growth curve in computing power per unit of investment, visualisation has progressively moved to the core of exploratory and analytic strategies. The effects on traditional methods are profound, as separate work phases collapse into continuous cybernetic feedback loops and statisticians develop increasingly immersive relationships with their raw material. On the other, data visualisation has penetrated mainstream discourse to become an integral part of vernacular literacy – “one of the genuinely new cultural forms enabled by computing” as Lev Manovich ^{[1, 2]} describes it.

Those two aspects, the technical and the vernacular, are not separate; they are two sides of the same coin. They are beginning to interpenetrate with other developments such as direct onscreen haptic manipulation of program interfaces and may in the long run turn out to be the most far reaching and profound effect of the scientific computing revolution.

At the heart of this lies the capacity of inexpensive desktop, laptop or even handheld devices to manipulate graphics in real time response to user curiosity. When I started writing for Scientific Computing World, back in the 1990s, it was possible to represent three data variables as a scatter plot cloud, or as a fitted surface, on x, y and z axes, but changing the viewpoint or scale usually involved typing new parameters into a settings box and watching the screen progressively redraw. It seemed pretty cool, then. I remember my excitement when the major statistics packages, one by one, added the ability to grab the plot with a mouse click and intuitively apply zoom, pitch, roll and yaw by dragging. Nowadays, I can do the same on a pocket tablet or even a cellphone by simply sliding my fingers around the image itself. On a desktop, laptop or heavier tablet machine I have access to considerably more than three dimensions, not to mention different display types such as vector flows in the same visualisation as positional points, planes or volumes.

Not that such impressive psychoperceptual pyrotechnics are always necessary or even desirable in every context. Detailed 2D presentation of very traditional plots of the kind that would have been familiar to my primary school self in the late 1950s are, in many circumstances, still the best visualisations of real world situations. The miracle of current software is that those two extremes, and everything between, are available off the shelf to suit the needs of the moment. [more...]

---

1. Lev Manovich, “Data visualisation as new abstraction and anti-sublime” in Small tech: the culture of digital tools, electronic mediations, B. Hawk and D.M. Rieder, Editors. 2008, University of Minnesota Press: Minneapolis.
2. Lev Manovich, Software takes command : extending the language of new media. International texts in critical media aesthetics.  9781623568177.

Full references list here

Stats for the million

Most of my consultancy work boils down to devising ways in which clients can maximise the validity and quality of work by lay staff or volunteers whilst minimising the impact of inevitable errors. And my email inbox is rarely without a plaintive request (open or veiled) for informal guidance or reassurance on a conclusion in which the sender lacks the confidence necessary for exposing it to peer scrutiny.

In the real world, whether we statisticians like it or not, an overwhelming majority of statistically based decisions and statistically oriented actions are taken by nonstatisticians.

Having made such a sweeping statement, it would be prudent to take a moment for definition of terms. What, exactly, is a nonstatistician?

In my own practice, there is no single clear cut answer to that. The word is not the binary divisor which it pretends to be: just a shifting pointer on a continuum. That’s why I used the alternative description "lay staff" in my first paragraph, above: because it’s less complicated. All too often, individuals with the same level of expertise will place themselves very emphatically on opposite sides of the pointer. In one market segment with which I am very familiar, professionals who are hazy about the difference between mean and mode are counted as statisticians and make million euro decisions based on statistical grounds. In another, science graduates whose degree transcripts include passes in all the usual statistics courses claim to be completely baffled by the subject.

Reading through the literature of any professional area, but particularly in the medical and life sciences, one finds frequent reference to this. [more...]

Retrieving data day queries

Perhaps the most famous data retrieval case in the history of science comes from sixteenth century orbital mechanics. Copernicus had laid the foundations for a viable heliocentric system; Kepler stood ready to finalise it. Between the two, both problem and solution, lay the mysteries of Mars: "the wanderer planet". The data which Kepler needed already existed, in a database of naked eye observations painstakingly constructed over two decades by Danish philosopher Tycho Brahe.

The problem was twofold. Brahe had nailed his colours to a mixed system at odds with that of Copernicus; and his data were his claim to posterity. He employed Kepler as an assistant, but jealously guarded access to the full observational data set.

Kepler did, eventually, gain access to the data. It wasn’t easy, nor always amicable (though allegations that he murdered Brahe to achieve it have been discounted), but it was done. He still had to learn how to retrieve it productively, but six years of mining and analysis finally bridged the gap to produce a final, successful, validated model.

Things have changed almost unrecognisably over the four or five centuries since Copernicus, Kepler and Brahe, but some features recognisably remain amid the new. Investment in research is balanced against the advantages of shared access. Boundaries, proprietary or otherwise, remain between researchers and data repositories. Murder and less extreme espionage methods may be rare (though not unheard of) as means of gaining access to data stores, but Kepler would no doubt recognise in essence the processes of negotiation and persuasion which allow those boundaries to be permeated.

The biggest early twenty first century data retrieval issue, however, is a different one. Acquisition in large quantity is becoming ever easier. Storage is, in relative terms, becoming cheaper. The headache often becomes how to ensure that one retrieves the right data for particular purposes from the ever ballooning volumes which are thus becoming available.

And then there is the problem of storage format obsolescence. Unlikely as it may seem, digital information which is by definition recent and (you might think) ought to be more easily accessible, and more carefully curated, can sometimes be harder to reach than older analogue stores. [More...]

Omnilingual word spotting

In yesterday's out take from my recent “Joy of text” piece in Scientific Computing World, I mentioned in passing the term “word spotting”.

Word spotting looks for visually similar discrete components within a text, and classifies them using statistical comparisons. In approximate human terms, it is treating words (or combinations of words) as ideograms rather than as phonetic constructs.

Word spotting is not limited to written or printed material, though that is the context with which I'm concerned here: it also applies, for example, to speech recognition. Nor, intriguingly, is it necessarily limited to words of known meaning; it can equally well be applied to semantic units of entirely unknown signification. It could, as an extreme example, be used to analyse the manuscripts from a lost extraterrestrial civilisation in H Beam Piper's classic science fiction story Omnilingual.

Reverse the Omnilingual example to imagine a hypothetical extraterrestrial archaeologist trying to study post apocalyptic remains of our own cultures. It will be obvious from context that certain signification units are associated with the physical sciences: "volts" on electrical signs and appliances, just to pick one.

Our xenoarchaeologist (who may not have alphabetic scription systems, or even vocalisation, and certainly cannot assume that each letter represents a sound) looks at a mass of textual matter whose content, subject matter, purpose and reliability are unknowable. Where should attention be concentrated? The only certain knowledge is that words are identifiable visual entities found in isolation and occurring with spatial separation in books.

Word spotting, with no semantic assumptions, quickly shows that some books contain many instances of the visually related signifiers "volts" "volt", "voltage", "voltmeter" and so on, suggesting that those sections contain material related to electricity; others do not. There will, of course, be false hits such as "Voltaire" and "revolt", but as one discriminator amongst others in a multiple sieving process it would nevertheless be invaluable.

Handwritten notes and journals would be less amenable than printed books (in my own handwriting, for instance, computer transcription systems have trouble separating "volt" from "bolt" and sometimes even "void"), but could still be sieved using multiple discriminators in the same way.

---

• Piper, H.B., Omnilingual, in Astounding Science Fiction. 1957, Dell Magazines: Northwalk CT.

Making sense of the census

In preparing an article on any topic, there is often more good stuff than makes it into the final edit.

In the case of my recent “Joy of text” piece, about text analysis for Scientific Computing World, one of the out takes was a prototype content-based image retrieval system framework developed for census searches by Kenton McHenry and his ISDA group at NCSA. The problem to be solved is computerised searching of large volume handwritten census returns.

A user inputs a handwritten query – I might, for instance, write "Grant". The system derives a numerical feature vector which describes that input, then seeks occurrences of similar vectors within the image database.

The system is designed to self-validate, by recording which returned entities are selected by the user. I will not, for example, select false hits such as "Grand" or "Ghent" and the system will note which results I do or do not follow up. Over time, other Grants will make similar decisions and increase the system's confidence in selecting some hits for return and not others; gradually, those writing “Grant” as their query will see fewer and fewer offers of documents containing similar looking words.

The computer analytic process behind all this is progressive.

First, the lines and boxes on the census forms are used to carve up the content into image segments (for instance, surname will be in a box at the same location on each form and will become a data entity). Each segment is then converted into a numerical feature vector representing its appearance, and similar feature vectors are grouped hierarchically. Two million XSEDE (Extreme Science and Engineering Discovery Environment) CPU hours have been requested for initial record processing.

When the search query is entered, word spotting is used to compare its vector with those stored in the database, seeking matches within statistically defined limits of similarity. The search is not a blind one from the beginning of the database through seventy billion image segments to the end; the hierarchical grouping guides greatly reduces the number of entities which need to be compared.

The joy of text

It's a mundane truism, not normally worth mentioning, that words and phrases as signification units in natural language have only the fuzziest of relations to that which they signify. It is, nevertheless, a live issue for the many researchers attempting to computerise data analytic activity using text as raw material. It's also a truism of which I have been reminded afresh as I discussed the topic with practitioners and consumers of textual analysis, no two of whom used the term in exactly the way.

Strictly speaking, textual analysis describes a social sciences methodology for examining and categorising communication content. In practice, though, it is widely used to cover a range of activities in which unstructured or partially structured textual material is submitted to rigorous analytic treatment. What they all have in common is a desire to wrestle the petabytes of potentially valuable information locked up in an ever inflating text reservoir (blogs, books, chat rooms, clinical notes, departmental minutes, emails, field journals, lab notebooks, patents, reports, specification sheets, web sites and a million other sources) into a form which is susceptible to useful, objective data analytic treatment. Temis, of whom more later, have on their website a headline which sums it up neatly: "Big data issue #1: a lot of content and no insights". Text mining, the consequent knowledge bases, and analysis of the results have become a major component of biomedical and pharmaceutical research.

For our purposes here, I have taken it to mean analysis whose purpose is to extract scientific value from texts, to examine those texts scientifically, or some combination of the two. [More]

Origin and OriginPro 9

Origin and OriginPro both went 64-bit in their previous release (8.6) and it was a welcome advance, but this is the update in which the effects really start to show off. On the most intensively demanding graphing tasks you don’t need a benchmark test to show their speed gain; it’s clearly visible as you work. One complex 3D operation which used to allow me time for a swig of coffee is now complete before I lift my finger from the mouse click which triggered it. … … … Exactly how much of this is a result of refined 64-bit utilisation and how much is down to default OpenGL graphics for 3D work I can’t say, but the net result is startling and impressive.

Putting aside the speed issue, my personal favourite picks were additions which make work flow more smoothly and, therefore, render data exploration and interpretation more transparent. [more]

Ode to OCR

Preparing an article on text analysis for the upcoming issue of Scientific Computing World (watch this space) I've been revisiting the various ways of getting physical text into digital form.

Optical Character Recognition (OCR) is the workhorse of text transcription and, while we all grumble about its very real real short comings shortcomings, it really does a remarkably good job, most of the time, of rendering graphic images of printed fonts into digitised text for analysis.

Even at the lowliest manual level, and with all its admitted faults, OCR is a useful tool. A colleague and I recently had to add a six hundred and fifty page eighteenth century text to a digitised textbase for analysis. The rare and valuable paper original which we located was in a library, and could not be removed. Filing a request for the digitisation to be carried out would take weeks. With the consent of the library we used a smartphone, a ten year old copy of ABBYY FineReader 5 (now in release 11, and correspondingly more developed, as part of a software range for different text tasks) and a netbook. Even allowing for manual error correction, we had our validated data within three hours and the library added a copy to its own digital records. A similarly sized text already available as graphic only PDF was transferred more quickly still.

Going places

As a child I lived, for a while, on open plains at opposite sides of the planet where (like all my schoolmates) I was fascinated by glimpses of how transport might be in our future. The P1127 (an experimental VSTOL aircraft which would later become the Harrier); the Belvedere (a twin rotor helicopter, already ageing and soon to be swept away by the Chinook); and, most exciting of all, something which we heard but never saw.

This gave rise to one of my earliest serious data gathering and analysis projects: a notebook in which I religiously recorded details of everything mechanical that moved and, with particular attention to detail, rocket test firings. The date, time and direction for every roaring launch, every thud of a concrete warhead returned to earth, went into my little book. After a while, though I didn’t yet know the term “data analysis”, I was able to tell my friends with fair accuracy when the next launch was likely to be, how long the flight would take and where the impact would probably be.

These rockets had mysterious names like Honest John, Thor, Thunderbird, and Bloodhound. Fifty-or-so years on, another rocket bears the Bloodhound name but without the warlike aura. This one aims to transport a human being, not an explosive charge, in an attempt upon both the land and low level aviation speed records at 1·4 times the sea level speed of sound. At the same time, it is generating data and methods for spin off into wider scientific and technological theatres and scientific computing is at its heart. [more]

MapleSim 16

After an initial lockstep phase in which MapleSim upgrades were closely tied to those for Maple, release 6 was held back and appeared shortly after Maple 16. It may be that this marks the point when the simulation package asserts its independent existence as an application in its own right, rather than a younger sibling in the shadow of its older stable mate.

Not that the links with Maple have loosened in any functional sense; in fact, their potential for integration continues to grow. Maple can now transparently open MapleSim (and Modelica, of which more in a moment) models within its worksheets and make thread safe MapleSim function calls from its TPM (the high level, multithreaded, Task Programming Model). Information from the MapleSim model, including system equations, can be accessed, as can MapleSim's full solver repertoire. And Maple can set or change MapleSim parameters and solver settings, so is able to operate as a symbolic front end. The two products remain firmly complementary, but not co-dependent; MapleSim is developing a wider vision.

[more]

Timeslips (1)

This is, functionally and organically, an extension of my first Vie Hebdomadaires post, “The puzzle that is me”, from 30 July. Perhaps extension isn't the right word; ‘descendent’ or ‘inheritor’ might be better. It's likely to be quite short; partly because I have only as long as it takes me to eat a bowl of soup, and partly because I don't yet know exactly where it is heading. It will have to be the first fragment of a self exploration in progress.

I said, back then in July, that a particular science fiction novel read in my early teens “changed me fundamentally and shaped my whole subsequent life”. To a more diffuse but equally real extent, that is also true of the whole gamut of science fiction which I read between roughly thirteen and sixteen. While this is a fact of which I've been generally aware for some time, over the past three months I've been thinking about it consciously and explicitly.

In second hand bookshops, charity stalls, digital reissues and the always excellent Marx Books, I've been deliberately looking out for copies of those titles and authors which I can retrieve from memory. Not according to a grand strategy; each quest is triggered by a chance association from another book read, by a day to day experience, by a remark in conversation.

I was a voracious reader at that time. Though many of the books concerned are, I now discover, quite slim, I nevertheless cannot any longer devour six or seven of them in a day. It's perhaps fortunate, then, that I can't remember enough bibliographic details to track down more than a few of them. Some I remember clearly and can go straight to them in a catalogue; Geoffrey Hoyle's October the first is too late was one such. Others I can find by following trails of breadcrumbs backward ... I could remember neither author nor title for Frederik Pohl's A plague of pythons, for instance, but searching online for a combination of things I did remember (coronets, mind control, science fiction novel, hoaxer, etc) brought them up fairly quickly. For some I have to rely on the black arts of which Ray Girvan is master; he has located several titles which defeated me completely, from almost no clues at all, and is currently chasing another for me.

There are several things that particularly interest me about delving back into what I read at a formative time in my life. The most obvious and superficial is finding out what it was, exactly, that so strongly influenced me – and why. There is also a historical fascination in looking back to how things were then. Even more interesting, though, is to discover what I was like then: to see what sort of teenager I must have been for this fiction to have precisely that effect upon my developing psyche. It's a fascinating voyage of self discovery.

I have to pick up my stylus and walk, now. A lecture hall full of eager (or perhaps somnolent) students awaits me. Rather than stow this and continue it later, this seems like a good point to break and post. I'll be back, in other (no doubt equally fragmentary) posts, to follow some of the threads I've started to gather here.

---

• Fred Hoyle, October the first is too late. 1966, London: Heinemann.
• Frederik Pohl, A plague of pythons. 1966, London: Gollancz.

Fire, flood, pestilence and war

Statisticians, despite popular perceptions to the contrary, are as fond of play as the next person – and for a statistician there is no bigger playground than ecology. It starts off at the same size as the planet, layers dimensionally upwards downwards from human scale, has multiple expression in uncountably many scientific study domains, extends backwards and forwards in time. And throughout all of that, it is intrinsically statistical.

The surge of public environmental attention in the 1960s and 1970s after publication of Rachel Carson's Silent spring was a key factor in making me a statistician, and many others of my generation make the same admission. Hippy illusions gave way to irresistible glimpses into endless unexplored vistas of complexly related data ... and then came scientific computing to make the adventure feasible.

Ecology is, in conceptual essence, a statistical study of chreodic systems in perpetual flux with dynamic equilibrium frequently rearranged by catastrophic sheer planes.

Read more...

---

• Rachel Carson, Silent spring. 1962, Boston, Mass.: Houghton Mifflin.

What are your thoughts?

“I used to think I was indecisive, but now I’m not so sure”. So runs one of the oldest entries in the bumper fun book of psychology jokes. It parallels one in the statistician’s equivalent volume: “statistics means never having to say you’re certain” (which, if you are too young to remember Ali McGraw and Ryan O’Neal^[1] in Love story, plays off the strapline “Love means never having to say you’re sorry.”) And both have a serious echoes in professional practice: because psychology is an area in which data trees are always compromised by a forest of confounding factors, and samples can often be small. In most pure, classical psychology, to an even greater extent than in the physical sciences, only statistical analysis can (to mix metaphors) tease out the pure signal from the white noise with confidence.

Even in its pure and classic form, the concerns of even the most abstract psychological research are rarely far from pragmatic utility. Roland Bremond, a mathematical morphologist currently concerned with the very down to earth business of transport planning, neatly encapsulated the data analytic nature of the beast^[2] in Saussurian terms: “La «réalité», du monde des objets, en psychologie expérimentale, est définie statistiquement comme l’ensemble des propriétés perçues partagées par tous les sujets «normaux» ... ... ... Autrement dit, la réalité est une propriété statistique, sur une population...” (In experimental psychology “reality”, the world of objects, is defined statistically as the set of properties shared by all perceived “normal” subjects ... ... ... or, to put it another way, reality is a statistical property of a population...)

Not that pure classical psychological research is the only strand or even the most evident. Without straying too far into monist and dualist controversies it’s fair to say that complete separation of mind from body was abandoned a long time ago, and psychology interpenetrates deeply with neuroscience and biochemistry amongst many other physical areas.

In its classic guise, especially in combination with modern computerised statistics, psychology is one of the areas of science where real, useful, original research can still be done by the solitary researcher without institutional funding or resources. In my freelance consultancy life I regularly work with research psychologists and frequently encounter such individuals, most of them right at home in one data analysis software package or another. In the run up to this article several lines of thought came from lone workers including the occasional pre-university student. [More]

Solas 4.0

Missing data values are the bane of a statistician’s life. At best, when the distribution of missing values is truly random, they reduce sample size which has a knock-on effect on conclusions and confidence. At worst, when there is an extensive pattern of systematic omissions, they can render a whole data set completely useless.

At the same time, attempting to fill in (or, in the language of the trade, impute) the missing values is a fraught business full of pitfalls. There is an ever present danger of building assumptions into the data which are then expressed in analyses.

Very often, however, the benefits of imputation in retrieving at least part of a study for useful non-content-robust analysis outweigh purist qualms. The question is then how to go about the process in the best way and this is where Solas comes in... [more]

AdviseStat

In an ideal world, everyone who uses statistics or statistical methods (for whatever purpose) would fully and thoroughly understand them, making the best possible decisions in every instance. The real world, alas, is not like that. We all have to make use of disciplines which are not our own, making the best decisions we can in partial or imperfect knowledge. Data analysis is a discipline so thoroughly essential in almost every area of science that it is bound to be widely, though unintentionally, misapplied.

There is, therefore, a great need for software tools which try to guide uncertain users into best possible practice. All statistics software has, for a long time now, been exploring and developing ways to do this. Some have made it their first principle and AdviseStat is one such solution... [more]

Going up

[this is a copy of my second post as a guest on Vie Hebdomadaires]

---

It’s been a heavy day, time is short, and I’m afraid it’s not going to be a long post tonight.

On Sunday, my friend Gayle Reynolds alerted me to advance publicity for release of the Berkeley Earth Surface Temperature (BEST) study. Yesterday, the results were made available so I duly collected a copy but didn’t get a chance to read it until today.

BEST analyses temperature data over a period of more than two hundred and fifty years. Though its declared aim to make a transparent database of records, calculations, methods and results publicly available is valuable in itself, the view has been obscured by a blizzard of accusations and counter accusations flying in both directions between the climate change accepters and sceptics, additionally fuelled by funding from sources associated with aggressive global warming denial.

In the event, the report firmly underwrites the anthropocentric temperature increase model and can find “… no significant correlation between the land surface temperature history of the last 250 years and the solar forcing history…”

And along with the report I’ve downloaded the data sets (they are , as promised, publicly available on the BEST site), so that I can play with them myself over the weeks ahead … which, in itself, makes this a good day.

- Felix

---

• Robert Rohde, et al., A new estimate of the average earth surface land temperature, spanning 1753 to 2011, 2012. Available from http://berkeleyearth.org/pdf/results-paper-july-8.pdf

Navigating the sea of genes

Over my working life, statistical data analysis has explosively expanded in significance across every area of scientific endeavour. Over the last couple of decades, computerised methods have ceased to be an aid and become, instead, simply the way in which that statistical data analysis is done. Partly as a result, and partly as a driver, data set sizes in many fields have grown steadily. As with every successful application of technology, a ratchet effect has kicked in and data volumes in many fields have reached the point where manual methods are unthinkable.

Some areas of study, however, have seen their data mushroom more than others. Of those, few can match the expansion found in genetics, a field which has itself burgeoned alongside scientific computing over a similar time period. The drive to map whole genomes, in particular, generates data by the metaphorical ship load; an IT manager at one university quipped that “it’s called it the selfish gene for a reason, and not the one Dawkins gave: it selfishly consumes as much computational capacity as I can allocate, and then wants more”. [more]

Perfecting pills and potions

“You are here to learn the subtle science and exact art of potion making ... the delicate power of liquids that creep through human veins, bewitching the mind, ensnaring the senses ... I can teach you how to bottle fame, brew glory, even stopper death.”^[1]

J K Rowling's potions master, Professor Severus Snape, is talking of magic but his approach is in many ways more scientific (“there is little foolish wand waving here”, he warns) than muggle explanations for drug effects which were accepted professional consensus until quite recently in the scheme of things. In anything like the present day usage of the term, pharmacology is (give or take an argument or two over detail) a hundred and sixty five years old. For much of that short history, pharmacological data analysis was externally observational: the body was a black box, with the drug as input and detected effects as outputs. The current receptor based molecular approach, though it has its theoretical roots in the early twentieth century, only really dates from after the second world war. [more]

---

1. Rowling, J.K., Harry Potter and the philosopher's stone. 1997, London: Bloomsbury. 0747532699 (cased) 0747532745 (pbk.).

A book of Eliot, a flask of Château Rieussec, and thou...

In a comment to my post noting a PNAS paper about rhythm structures, Julie Heyward asked:

Why does this feel a little like saying that a glass of Château Rieussec smells of ripe pineapple and honey because it contains ethyl butyrate and phenylethylic alcohols? [Not that I've ever had a glass of Château Rieussec or know that they smell of ripe pineapple and honey, much less that it's because of the ethyl butyrate and phenylethylic alcohols – I just read and repeat this stuff ... and I hope you appreciate how long it took me to type ethylyl butyrate and phenylethylic alcohol three times.]

I didn't even know what Château Rieussec was, at the time, though of course I guessed that it was a wine. I've now looked it up, here.

I confess that all wines, and beer for that matter, taste to me like variations on the same sour, unpleasant theme. None of the finer details of aroma and bouquet reach me through the general sense that something unpleasant is rotting in my mouth. Which probably makes me a Philistine, but there you go (though while we're at it, may I point out that the Philistines are victims of a JudaeoChristian smear campaign). Brandy is a different matter ... but, I digress.

I do appreciate Julie's heroism in typing ethyl butyrate and phenylethylic alcohol three times, though I confess that I haven't typed either myself – simply copied and pasted them.

Anyway ... to return to the issue at hand ... Julie has a good point, though not an uncomplicated one.

Many moons ago, as a teenager studying English literature at A-level, I found my natural enjoyment of poetry and drama, in particular, eroded by continual analysis. In the year after I finished that course, I read almost nothing. Later, though, I returned to literature and discovered that the habits of analysis had now internalised and operated as an enriching layer which fed, rather than opposed, my enjoyment. The crucial difference was that I no longer stopped reading T S Eliot to think “hmmm ... Eliot is here echoing the contemporary debates around the meaning of time and the psychology of mind, through reference to Dante and the Vedas...” but, rather, allowed that knowledge (where it existed) to thrum away in my subconscious and generate overtones which enhanced the conscious reading ... or not, if I didn't have that knowledge available.

That distinction is one which I have since applied to every cultural interaction. Added layers of meaning are valuable if they enhance the "reading"; undesirable if they interfere with it.

I've made reference from time to time to habitat theory, particularly as espoused by Jay Appleton and proposed in his The experience of landscape. I find Appleton's analyses, and habitat theory generally, fascinating as an entirely self contained set of ideas separate from my twin loves of walking and art. When I am enjoying either a hike through the Wicklow mountains or a Maggi Hambling sunset painting, I never ever think consciously “aha ... what we have here is a set of three interlocking prospect symbologies...” because to do so would spoil either experience. On the other hand, the subconscious knowledge that such an additional dimension exists, and is being synchronously processed at another level, does add a layer of chromatic complexity to both experiences which increases my enjoyment of them. I don't offer this as a general rule, though; it works for me, but may not work for others. My ex wife, a painter and sculptor of great passion, always refused to read Appleton and was right to do so – because, although she had a rich and complex intellectual life, and would cheerfully argue the intertwining of history and myth in Shakespeare's Richard III, she knew from experience that some arts of it had to be kept away from her creative self if she was to avoid interference of the type which afflicted my A-level English lit course.

Coming back to Bach and Château Rieussec, similar principles apply. As I listen to the Matthaus Passion, I shall spare not one fraction of a second for thoughts of the power law underlying its rhythm structure ... but I can still enjoy that knowledge as a separate delight. It's possible that the knowledge will subconsciously add to my conscious and instinctive enjoyment of the music; it's also possible that it won't; but it certainly will not be allowed onto the stage where it can interfere. If I am ever forced, by politeness, to sip wine (Château Rieussec or otherwise), on the other hand, I shall probably find conscious, centre stage thoughts of ethyl butyrate and phenylethylic alcohol (again copied and pasted, not typed) a useful distraction. Transferring the spirit of the Château Rieussec argument to something I do sensuously enjoy, I have to agree that neither ascorbic nor salicylic acids have any part in my sensations as I consume a mango or a tomato ... although I find the properties of those ingredients interesting at other times, when not lost in the fruits themselves.

In general, then, my reply to Julie's comment is that parallel knowledge can, in the right circumstances and depending on the individual, be a useful amplifier of experience ... but only as long as it doesn't usurp it. The trick is ensuing that each stays in its place and time.

---

• Post title with apologies to Fitzgerald's translation of The rubaiyat of Omar Khayyam.
  
• Jay Appleton, The Experience of Landscape. 1986, Hull: Hull University Press. 0859584615. [Originally 1975, London: Wiley. 0471032565.] [Most recent edition 1996, Chichester: Wiley. 0471962333 (hbk) or 047196235X (pbk).] [Now out of print.]