« June 2007 | Main | August 2007 »

July 31, 2007

Long Tail Statistics (Kilkki's Formulation Hides Latent Demand)

The expression “The Long Tail” first started to buzz with the WIRED article of that name by Chris Anderson a couple of years ago.  Later, he expanded the article into a book explaining the ideas more fully and giving examples from book sales, music and movies.

The idea is explained in many places.  Start with Wikipedia.  I won’t try to reproduce a basic explanation.

However, recently, I read a paper by Kalevi Kilkki titled "A practical model for analyzing long tails". In it, Kilkki explains an alternate Long Tail distribution to the traditional Power Law distribtution.

I don’t see Kilkki's motivation--or rather, I don't trust it. He seems to be blindly "refining" the distribution to get more precision.  I don't trust this for two reasons (1) precision isn't going to help prediction or design (without understanding underlying processes, at the very least) and (2) I think his quest to line up the dots with the lines hides important effects. We will see. Finally, I think Kilkki over emphasizes the differences and benefits of his model. To try out my main critique, I built a (rough) Mathematica notebook to look at Kilkki’s model and the traditional power law.

As usual, I generated more questions than answers, but, unless I really misunderstood Kilkki, I don’t see the benefits he is pointing to.  I hope you enjoy a quick look at The Long Tail and send comments if you have any thoughts.

July 23, 2007

Biofeedback Experiments 2

Can we see any difference in IHR between meditation (such as my skills are) and reading Ann Coulter's web site?  It turns out there are clear differences.

IHR Comparison Plot 

The y-scale is instantaneous heart rate (IHR).  The Red curve is
"Reading Ann Coulter" while the Blue curve is "Meditation."  Each curve
respresent the same amount of time recording with the
LightStone (see previous post).

 

First you can see that nearly all of the points on the Red curve are well above the Blue curve.  My average heart rate for the upper curve was 85.9; for the lower, 65.9.  (By the way, actual time in minutes is x-scale time divided by 60,000 so the elapsed time is about 3.5 minutes.) But also to variation in heat rate in the Blue curve is much more coherent.  To make this clearer, here is the power spectrum for each curve.

 

Power Spectrum
Power Spectrum [Abs(Descrete FFT)^2] from two equal time peiodes of
heart rate recording.  The upper curve (Blue) is meditating, while the lower (Red)
was recorded while I read Ann Coulter's web site.
  

 

The 3-4 strong peaks at the far left (the right half is a mirror image, an artifact of the FFT algorithm) in the upper power spectrum correspond the strong frequencies present in the HRV during meditation while the power spectum is spread into the upper frequencies in the lower plot.

Which is better for you? (We humans tend to like regualar patterns, so by that measure, meditation is "better".) Without knowing much more about the interactions of the systems are work here, it is hard to say.

July 21, 2007

Biofeedback Experiments 1

 

I sporadically practice meditation. As soon as I started, I was curious about what is going on with the brain and body during meditation. The biofeedback game, The Journey To Wild Divine,  uses a simple USB 3-finger sensor called the LightStone that measures heart rate and skin conductance give a simple answer to that question. These are two readily measurable quantities with patterns people associate with meditation practice.

Here is a nice description of The Journey To Wild Divine, reviewed for the Mac.

I tried out a friend’s copy late one Friday night and was out getting my own within a few weeks.  After a few sessions, I completed the game and came back to practice some of the exercises with the data recording feature turned on ($).

The JWD game exports the instantaneous heart rate (IHR), the (calculated) heart rate variation (HRV) and skin conductance level (SCL).  There are some open source programs for acquiring and analyzing the LightStone data. I just dumped the data into Mathematica via a Python import script.

The game presents visual “puzzles” (e.g. stack levitating rocks, unlock doors, etc.) one “solves” by controlling the readings on the LightStone.  Some require becoming more energetic (agitated?), and many require great relaxation and steady attention. Below are the recordings from (what I found) a challenging exercise in the game.  It took maybe 10 or more minutes.  It is hard to keep track of time when you are solving the puzzles, and I didn’t time myself.  (I haven’t taken the effort to figure out the time scale on the exported data yet.)

 SCl Plot

The most dramatic (graphically) change over the exercise was the decrease in SCL.  At the beginning of the exercise, SCL is relatively high and unsteady.  It decreased steadily with small sharp jumps upward until the puzzle was solved. (The change in pattern of all three quantities is visible when the puzzle was solved a little before the end of the recording.)

 

HRV Plot
 

 

The HRV scale is in arbitrary units (i.e., I don’t understand them yet).  There seems to be research and claims (and gadget-programs!) regarding the correlation between higher HRV and good health. Ah, causality.  Anyway, in order to solve the puzzle in JWD, it appears that I had to raise my time-averaged HRV. (I don’t know the actual formula.)

 IHR Plot

The last graph shows the IHR.  This one I think I understand.  JWD use a heart beat pattern recognition algorithm to detect the spike in every heart beat and use that to mark the time between individual beats. The IHR is proportional to1/(time between two adjacent peaks).

The IHR starts out erratic, but settles into a pattern that raises and falls in relation to my respiration (it seems).

This is fun. Measuring a couple of readily available metrics started me thinking about the possibilities or getting to key points in other systems simultaneously.  For example, it is easy to imagine that my endocrine system is involved, and so that metabolism is affected. And on and on…Our systems-level understanding of the body is very primitive, but the opportunities for deep understanding are starting to seem within reach.

July 07, 2007

Read in the last 100 days...

I don't intent to write book reviews here, but it might be interesting to see what I have been seen reading in the last 100 days because it has a lot with how and what I am thinking about the World. Briefly,   "Paleo" is a different and useful approach to thinking about training and diet, Nichols was good, heartfelt writing about my home state, Robb is on to something very important to planning and executing modern effective foreign policy, and "Six Degrees" is full of interesting network dynamics (some of the results have been reproduced here).

 

Paleo Diet for Athletes, The... Cordain PhD, Loren and
House of Rain: Tracking a va... Childs, Craig
Motherless Brooklyn... Lethem, Jonathan
Biomimicry: Innovation inspi... Benyus, Janine M.
On The Mesa... Nichols, John
Brave New War: The next stag... Robb, John
Six Degrees: The Science of ... Watts, Duncan J.
Astral Dynamics: A new appro... Bruce, Robert


Hosting by Yahoo!