Skippy Records

Candidates for Top 30 Conspiracy Theories - Link

Fri, 21 Nov 2008 13:16:08 -0700

As some of you have been dragged into conversations about conspiracy theories and why we love them, you will not be surprised that I plan to start writing about it here. This first post is an introduction and overview of some of the best-loved CSs. The pictures in this article make an easy staring point.

Search queries: Stock, recession, bailout

Mon, 17 Nov 2008 19:27:39 -0700

Using the same 90K searches that I have referred to here before, today I looked at search queries containing the words stock (and related terms), recession, and bailout. The left scale of the plot below is indicates the fraction of all queries for each day containing the terms. The right-hand scale is the NASDAQ100 index for the same period.

Interest in the stock market starts to take off in early September, just as the NASDAQ starts the dramatic decline. Curiosity and research on the bailout don't seem to be closely correlated with stock prices or interest in the stock market.

Interest in recession started back in June when the NASDAQ showed signs of weakness then declined. More recently, interest in recession has continued to since late October.

Are you already "socially networked" with everyone you know?

Sun, 16 Nov 2008 19:42:44 -0700

Facebook and LinkedIn Network Connections

Both my Facebook and LinkedIn social networks have gone through periods of rapid grown in the past year. Although I have been a member of LinkedIn for many years, my network expanded rapidly last year, approximately doubling in a couple of months. I have had a similar experience in the last few months with my Facebook network.

This is likely a convergence of factors. My contemporaries are reconnecting with high-school and college acquaintances on Facebook. More of the people I have worked with in the past have joined LinkedIn and started added colleagues from past employers. This has led to a critical mass of people both connecting and inviting others to connect.

In each case, it seems that when enough people in my circle of acquaintances are connected, new connections form rapidly because it isn't far (in terms of network hops) to other people I know.

So am I already "socially networked" with everyone I know? I think the answer is yes. Because I am not very active at inviting new friends, it might be more accurate to say that I could be completely socially networked with an evening's work. In terms of network dynamics, this is because the network of my local group of friends has undergone the phase transition network mathematicians refer to as "percolation."

A simple model of network percolation

Where are without a model? Not very far on this blog. Instead of modeling a complete social network, let's start with a simplified model and see if the behavior I want to describe pops out.

Here is a model that is easy to visualize. Imagine I am about to tile my kitchen floor. I draw a grid on the floor showing where each tile will be placed. Looking at the kitchen floor as I prepare to lay the tile, you see a simple grid of lines, like graph paper. Now, start adding tiles. But instead of working from one side of the kitchen to the other in rows (like a normal person), I add tiles randomly.

I lay one tile by the sink, then two by the fridge, then another by the stairwell, a couple in the center. The floor is getting cover red with tiles, but in a hodgepodge sprinkling. Not too many of them are sitting right next to each other at first. As I put down more and more tiles, however, there start to be little clusters of adjacent tiles. After a little while, the floor might look like this:

But then, along comes my nephew Hayden, wanting to go to the sink for a drink of water. I don't want him to walk where there are no tiles, because he will track glue everywhere. But he is small, and can't take steps bigger than one tile. So I need to hurry to put down enough tiles so he can walk to the sink. But I don't want to ruin my beautiful experiment with the random tiles. How many tiles do I have to put down (still placing them at random locations) before Hayden can walk on adjacent tiles from the living room to the sink to get a drink?

When he can do this, the edges of the network of tiles are connected. The network of tiles is said to have a percolating group of tiles.

Juan Enriquez on the Credit Crisis - Good Info and Entertaining

Mon, 03 Nov 2008 09:25:03 -0700

Entertaining... for the dismal science. This is worth your time as it adds a lot of data and context re the current situation.

Juan Enriquez (2008) Pop!Tech Pop!Cast from PopTech on Vimeo.

Query Count Update-Obama Pulls Away From McCain

Sat, 01 Nov 2008 17:34:20 -0700

The election is finally going to happen in a few days. Are people searching for the candidates more than the were last time we looked on the Obama-McCain Search Query Counts October 19 and Obama-McCain Search Query Counts October 21?

Interest in Obama continues to accelerate, while query counts for McCain have grown only slightly. Obama continues to widen the gap in the world of query attention.

Looking once again at the interest in McCain and Palin shows that they are running even.

Data

I added a few misspellings to the queries so, while the patterns are similar, the absolute numbers have likely changed a bit. Recent data is from pool of about 90K unique users per day searching on all major search engines and some little known engines as well. Searches on e-commerce sites are included as well.

Useful Political Discourse: Colin Powell Endorses Obama

Tue, 21 Oct 2008 18:35:48 -0700

Whether you support McCain or Obama, this is a well-spoken and nicely considered endorsement. Thank you Mr. Powell, for taking a moment to upgrade political discourse at a time when it has sunk low. Colin Powell endorses Barack Obama.

Hat tip to HalsHop.

McCain-Obama, McCain-Palin Query Counts Update - 21 Oct 2008

Tue, 21 Oct 2008 16:57:39 -0700

This post updates the data from the post Search Queries, Obama, McCain, Palin and Sex with the latest query counts and presents the data with a little refinement (no symbols and 5-day moving average).

Obama continues to outpace McCain in internet queries:

Gov. Palin continues to garner approximately as many query counts as McCain, but the trends for each are in the opposite direction.

The Direction of Time

Mon, 20 Oct 2008 20:56:31 -0700

What determines the direction of time? At first, this seems to be a non-question: of course time flows in the direction it flows. But physicists deal with many ideal systems that run equally well in both direction. The physics of the past look just like the physics of the future. This is true of colliding billiard balls and very simple planetary systems. If the evolution of a physical system is symmetric in time, what makes time flow one way and not the other?

The common explanation is that when we look at complex (real) systems, the Second Law of thermodynamics, that entropy--approximately disorder--increases with the flow of time, dictates the direction of time. Our habit of seeing this everywhere explains the fascination with a movie of spilled milk being played backward. It is surreal to see the milk go back inside the glass as it reassembles for the shattered pieces.

This explanation of time's direction is usually presented as a logical rather than causal explanation. Good for all you physics students who listened to your professors explanation of time's direction and the Second Law and said, hu, what?

In Ilya Prigogine's book The End of Certainty, he argues that the long-time explanation of the direction of time depending on the Second Law of thermodynamics requires an update. To illustrate the starting point, he builds a simple model with a deterministic (i.e. does the same thing given the same starting point) chaotic systems.

The argument behind the simulation explained below goes like this. There are structures that emerge and evolve in complex dynamical systems that (a) don't have any iterations between the particles and (b) evolve asymmetrically in time. Thus, the Second Law is not adequate to explain the direction of time.

The simulation below uses a very simple chaotic system, the Bernoulli Map. The equation of motion for each of an ensemble of particles is x(n+1) = 2x(n) mod 1. The new position, x(n+1), is calculated from the old position, x(n), by multiplying by two, then removing the integer part so the new position is between 0 and 1.

Chaotic systems are characterized by exponential divergence of trajectories that start out close together. Contrast this with a pendulum. When we have two pendulums of the same length and mass and we start one off at a slightly different point from the other, they just swing back and forth slightly off synchronization. But the gap doesn't grow over time.

In the Bernoulli Map, trajectories started out arbitrarily close diverge exponentially. The rate of divergence is measured by the Lyapunov exponent. The plot below shows the distance between two trajectories starting off very close and evolving according to the Bernoulli Map. The plot is on a semi-log scale; the slope of the line is the Lyapunov exponent.

Now we start many many particles together and watch the distribution of the particles along the x axis evolve. Notice that as time evolves (in the right direction!) the trajectories become uniformly distributed. This system does not evolve the other direction when run backwards, but rather continues to normalize the distribution. This is in spite of the fact that the particles do not interact or exchange energy (as they would in a gas coming to equilibrium, for example).

The Python code creating this example are available for download--Bernoulli Map.

Search Queries, Obama, McCain, Palin and Sex

Sun, 19 Oct 2008 19:11:01 -0700

Counting the occurrences of search terms enterred into major search engines can give hints to what people find interesting.

The plot below compares the fraction of queries containing the terms "McCain" and "Obama" since May, 2008. The search term "Obama" appears in about twice the queries as "McCain." Without digging further, there is not way to tell if this is favorable or negative interest.

Senator McCain's campaign got more attention from Web surfers after the announcement of Governor Palin as his running mate. The plot below shows the fraction of query occurrences of the term "Palin" compared to "McCain for the same period.

It is difficult to get a sense of scale on the interest in either presidential candidate. Comparing to a term that many Internet searchers are interested--yes, sex--gives some sense of relative interest. Below I plotted the fraction of searches containing the term "Sex" against the fraction of searches containing "Obama."

Inexplicably, "sex" lost favor over the summer up until the Democratic convention, when the frequency of the "Obama" term peaked. Not that neither candidate ever come close to matching our interest in sex. What does this mean for our democracy?

Recursive Play

Sat, 11 Oct 2008 20:09:45 -0700

A few days ago, I needed to collapse any depth of nested lists to a single list. Using the ability of a Python function to call itself (recursion), one is able to write a short and fairly straightforward routine for flattening a list containing any number of nested lists.

Here is the function I wrote:

def recursive_list_collapse(x):
    if type(x) == type('a') or type(x) == type(1) or type(x) == type(1.1):
        # if the object is already a charater,
        # just return it as a list of 1 char
        return [x]
    else:
        # if the object is a list, step through the objects
        # in the list and call myself with each object
        tmp = []
        for a in x:
            tmp += recursive_list_collapse(a)
    return tmp

Wikipedia has a nice article on Recursion.

It's not quite true that the method will collapse any number of lists--the depth is limited by the number of recursive calls to the function that the environment can support. Each time the method is called, the environment must store the information from the currently running method so that it can pick it back up again when the function call on the next to last line returns it's sublist. If you run out of scratch pad space to remember where you are, the methods cause an error.

The Wikipedia articles doesn't make much of a concept that is tied to recursion. Strictly speaking, recursion is infinite. But to do something useful, we need to stop at some point.

Pure recursion goes all the way down, but with self-reference (a model of self that allows decision making), recursion can be terminated and do something useful. It is difficult (impossible?) to find the "bottom" of the recursive method without some type of self-reference, without a model of at least part of self. In the method above, I use the ability of Python to look at the type of data in the list to determine if (a) it is another list--therefore, to recursively call itself with the sublist; or (b) if the item is a fundamental type, don't call the function again. Other recursive methods add a counter to the type passed in the recursive call--meta data--to keep track of self.

To explore the self-reference idea in another way, I rewrite the a non-recursive version of the method that uses a string manipulation algorithm. This output is identical to the routine above. This time, the method is to change representation to a string, manipulate the string to remove the "extra" list markers. But you can't just leave it a string, or it is not the same method. Here's the interesting self-reference part--Python allows you to execute the string as a program statement to turn it back into a list type using exec('r='+ string_manipulation(example)).

def string_manipulation(x):
    last = ''
    candidate = ''
    tmp = ''
    # walk through string with all spaces removed
    for c in str(x).replace(' ',''):
        if c == '[' or c == ']':
            candidate = ','
        else:
            candidate = c
        if last != ',':
            tmp += candidate
            last = candidate
        elif candidate != ',':
            tmp += candidate
            last = candidate
    return '[' + tmp[1:-1] + ']'

Neat.

P.S. This post was a strange coincidence as BFD emailed after I had started it to talk about recursion. Neat.

The Singlularity will always be near

Wed, 01 Oct 2008 13:29:43 -0700

Kevin Kelly's essay Thinkism is an important essay responding to apocalyptic thinking about the "Singularity." This is a very clear and considered essay, written with deep understanding and experience of how order emerges in complex systems. Related, I highly recommend his book Out of Control. It is a little dated, but if you hold this in mind as you read, it is all the more wonderful for Kelly's depth and prescience.

Acutally Useful Mortgage Bailout Idea

Wed, 01 Oct 2008 13:25:47 -0700

Art De Vany has a blog primarily focused on eating and exercising in a way compatible with the human metabolic system. But his day job used to be an economist. He proposes a bail out plan--the government buying options on excess housing inventory--that makes a lot of sense to me:

"So, who should be helped to clear out the over priced homes and underperforming mortgages? Leamer suggests the government might purchase the homes available for sale right now. It would take a purchase of half a million homes to solve problem. That would cost about $150 billion and would help homeowners directly, not Hank’s pals on Wall Street or those sketchy guys at Fannie and Freddie and Country Wide. My option approach would cost a lot less and put cash in the hands of homeowners and the banks right away. Follow the government buy out of overpriced homes with mortgages that are under water with tax credits for first-time homebuyers and a tax rebate for buyers of existing homes. This would clear out the stock of homes for sale quickly."

This plan clearly cuts out the greedy crew who invented timeless financial classics like the no-income-no-asset home loan (that they promptly sold the same day because of the out-sized risks). Why are we considering enriching Wall Street companies with a no-oversight $700B bail out? Are our leaders asking the wrong questions? Are they asking the wrong people?

Inevitable or successions of Adjacent Possible?

Sun, 28 Sep 2008 19:54:33 -0700

In Stuart Kauffman's book Reinventing the Sacred, he describes the concept of the Adjacent Possible. This refers to the idea that there are states of a system the follow immediately from the Current Actual state. And there are others, that do not. You sometimes here people say “You can't get there from here.” To get to states of the system that don't immediately follow from the Current Actual it may be necessary to follow successions of Adjacent Possibles.

Two problems face problem solvers:

Apprehending the Current Actual, and

Perceiving the Adjacent Possible.

Action becomes clear and easy to motivate (a choice rather than a struggle) given that a leader can accomplish both of these. The problem is that we don't have as direct a connection to apprehending the current actual (epistemology) nor as complete an understanding of the the adjacent states that are actually possible (science) as we sometime assume. That means particulars have to be worked out and that it is sometimes very difficult.

Fundamentalism ignores the problem completely. Instead of worrying about apprehending the Current Actual or perceiving the Adjacent Possible, fundamentalism both assumes the current state is an illusion and the final state will result from fiat transformation, in some cases no matter what we do (e.g. many Christian's contempt for stewardship of the Earth is based on a vague mix of a belief in the insignificance of human actions and conviction of an ultimate apocalyptic outcome).

When the end point is “known” to be inevitable, why worry about denial or realism? Why struggle against either? You don't need to hear the rest of the question when you know the answer. This may be a useful or even necessary state of mind for some (personal!) spiritual pursuits, but it is a bad way to make decisions about systems like the economy, the environment, the judiciary branch of government, foreign relations or the military. We've had 8 years of “principled” (read: willfully uninformed regarding the Current Actual and the Adjacent Possible) action from our executive leadership. Let's move ahead with something more hopeful.

Updates to Matchport AR Missing Tutorial

Sun, 28 Sep 2008 18:57:44 -0700

I was motivated to make some minor updates and clarifications to the MatchPort AR Missing Tutorial because it has already moved into the top ten files downloaded from Skippy Records. I just received the MatchPort b/g Pro in (what my buddy over at Joel's Garage affectionately refers to as) the "Brown Truck." I will start to extend the MatchPort AR how-to and getting started guide to include the wireless version later this week.

Useful Political Discourse - Lessig Video

Wed, 24 Sep 2008 10:37:43 -0700

I don't post much about politics here because I don't like the requirement of rediculous, overly simplistic side taking. But I have to give in on this one...

I am a big fan of Lessig. He is a key force in the creation of the Creative Commons and presistently in favor of smart thinking on intellectual property. He has created a nice presentation on Palin's Experience claims. First, I really love the preamble--enough with binary, loyalty-at-the-cost-of-reality thinking and speech. And, second, thanks for taking the time to present some verifiable facts.

Is there a bias? Yes. Lessig's thoughts are organized to tell a story. But the the thinking is considered, deep and informed by history and context. Thanks for that. It is a welcome addition to useful political discourse.