Visual representation of data
Posted by Arjendu on December 14, 2007
I read a booklet by Edward Tufte last night as preparation for the writing seminar I am attending. As usual, whenever I re-read Tufte, I find his arguments a fascinating reminder of how the representation of data can be so crucial in analysis and persuasion. Of course, analysis is persuading yourself of the validity of your hypothesis.
The workshop itself today had some discussion about graphical representation and some about finding and using data sets of various sorts. The latter was fascinating, but in a way completely unrelated to my work as a physicist — we live in a truly data-rich world nowadays, and there’s information of various sorts that make you go ‘oh, cool, I didn’t know I could figure that out’. There was even a quick discussion of something called Swivel (which was termed the ‘you-tube’ of data, and it really is! Their tagline is ‘tasty data goodies’).
I was sitting with Josh, a mathematician, and Greg, a political scientist who is pretty quantitative and perhaps our table was getting a little obnoxious, given that we all play with numbers and graphs a lot more than many of our colleagues. Ah well.
Sort of coincidentally, I got email back from my friend Arnaldo in Brazil about a project I’ve been working on with him for, umm, let’s see, almost 5 years now, since we started it when he visited me, and I remember that my daughter had just been born that winter. We’ve generated one paper from it, and I’ve been sitting on some other results for quite a while now, since I can’t quite explain what we have. The ‘sort of’ coincidence is because one of the key points of this paper is pretty visual.
To understand this, take as fact at the moment that if you look at the difference between the quantum prediction and the classical prediction for the behavior of a system, as a function of various parameters — such as size and temperature — of the system, you get a fairly complicated relationship. And why would we care about this difference? Well, classical behavior is very different in principle from quantal, the latter is turning out to be very useful in all sorts of ways, and it would be nice to know when we crossed over from one behavior to the other.
You have to imagine here that this distance function is plotted on one axis, and the different parameters are along the other two axes — we’ll stay in three dimensions for now. And so you’ve got some weird looking surface in your 3-dimensional plot. But if you search for ‘scaling’, that is, if you rotate and squish the data in certain ways, this data collapses into a single curve. It’s a pretty dramatic effect visually, it says something deep about the way quantum and classical behavior is different and it’s not entirely clear to me why it does this. In short, a perfect thing to think about for years on end.
Here’s the ‘back’ story. The idea of looking for scaling came to me in the middle of a boring colloquium at Rice University — I remember doodling it on my notepad. I worked on testing the basic idea with my friend Bala and his student Ben, and we showed that it, in fact, worked in two systems. Since then some other people have also found that scaling exists in other systems, which was great, so I know we are not just fooling ourselves somehow. Next I asked Arnaldo if he’d like to help me with trying out new ways of testing for scaling. Turns out it still works, in fact the new way works even better, but we still can’t quite say why it works in the particular way it does. That is, why does the single curve we land up with have the shape it does?
Arnaldo’s nominally the computational expert on this, while it’s nominally ‘my’ project, that is, it’s in my field, so my being stuck is not good news. I have a very smart student, Parin, working on this, but he’s only 3 years out of high school, so …
I keep waiting for inspiration, but I might take my friend’s advice and rope in one of my competitors to see if they’d like to be a collaborator on this, in case they understand the result. That’s one of the pleasures, as mentioned earlier, of this stage of my professional life — I care more about understanding the physics than trying to hoard the credit for a paper or something. Losing the fear of competition is very liberating.