Suppose there are 8 numbers. I choose one, but you don’t know which number it is. Your task is to guess which number. You can ask yes or no questions. What is the minimum number of questions you would need to ask to guess the number?

1
2
3
4
5
6
7
8

All have an equal chance of occurring (P(any number) = 1/8).

What would be your approach for asking questions?

Start at the mid-way point and asking if the number is less than 5.  Suppose the answer is yes. Now you know the number must be

1
2
3
4

Do the same thing and ask if the number is less than 3. Suppose the answer is yes again. Now you know the number must be

1
2

Now you ask whether the number is 1, and if the answer is no, then you know the number is 2. You have guessed the number in 3 questions.

3 is the minimum number of bits you would need to use to encode the probability distribution of these 8 numbers. This algorithm is at the core of how we are all able to send emails, pictures, and videos so quickly now. Before the father of information theory Claude Shannon had invented this algorithm, it was unthinkable to send a picture — even a voice message, across the globe. There was simply no efficient way to send it. Nobody knew how many bytes it would take to encode the message, or even how to encode the message.

Next question: What if the numbers did not have an equal chance of occurring?

I’ve been doing a lot of signal processing lately (Python tutorial on analyzing .wav files to come) and came upon this nice metaphor for Fourier Transforms, which can help extract information from sound.

Random dude: I need to know how many 7’s there are in 49.

High school algebra student: Duh, just divide 49 by 7. There are seven 7’s in 49.

Random dude: I need to know how many x’s there are in y.

High school algebra student: Duh, just divide y by x. There are y/x x’s in y.

Random dude: I need to know how much of the frequency 23Hz there is in my signal x(t).

High school algebra student: Duh, just divide x(t) by 23Hz. There is x(t)/23Hz amount of 23Hz in your signal.

via dmorris at TechHouse

One of the most irritating things about learning a new framework or switching to a new computer is often not learning the different nuances in syntax or even debugging — is installing the darn thing. I can recount the numerous dependency and version management errors I was met with upon when forking my friend’s Django project or deciding to pick up Flask, both Python web frameworks.

When I discovered virtual environments, a lot of these headaches disappeared. Virtual environments allow you to compartmentalize your framework and package installations into the clean slate, separate from the rest of your computer.

Why a virtual environment?

When people type ‘sudo’ before attempting to install something, I fret at the amount of trouble they may be causing themselves later down the line. Installing something globally will inevitably lead to version management problems and other clashes down the line.

What to install globally

• Python (usually comes installed)
  • Easy_install (comes with Python)

• Pip (Python package manager)

What to NOT install globally

• Packages
  • OK: sudo easy_install pip
  • May lead to problems: sudo pip install django

How to install a virtualenv.

Assuming you already have Python installed:

1. Install Pip.
   sudo easy_install pip
2. Use Pip to install virtualenv
   sudo pip install virtualenv
3. Change directories into the folder where you want the virtual environment installed.
   cd ~/Desktop/projects/my_app
4. Create your virtual env
   virtualenv env
5. Activate your virtual environment
   source env/bin/activate

Now your command line will include the virtual environment:

And you can install frameworks like django and packages in this new environment as necessary.

Notes

• Your environment will only exist when you activate it. Notice that if you open a new terminal window, it disappears.
• You can deactivate the environment by typing
  exit
Or closing the terminal window.

Close your eyes and listen to the brain.

What does it sound like?

Scientists did just that — they used amplifiers to listen to how neurons in our brain sounded, so they could discover how our eyes form an image of the outside world.

Where would you start if you wanted to model this process?

One thing that is generally helpful is to first identify the different components of a system. We have:
(1) the image in front of us
(2) some representation of the neuron
(3) an output of that image we see

How should we put those pieces together?

But what is wrong with this? That it’s probably more like:

But the brain is made up of many neurons, how would we model just one?

Now what if we do this for every neuron? Then we would have a whole picture.

The next question that arises is: How does each neuron know which slice of the picture to look at?

Well, what are the different components of a picture? A picture is fundamentally made up of:
(1) colors
(2) edges

Edges seem like they would give us more information about a picture, so let’s go down that route. What is an edge and how can we break that concept down? A major component of edges is the orientation of an edge.

So maybe each neuron has a preferred orientation it favors. And all of these neurons work together to create one picture.

So we can take lots of bars with different orientations, run them through a gabor filter (a fancy term that refers to a function that is similar to those in our visual system), and see when the gabor filter produces the most excited response.

If we run these through a gabor filter, we end up with a curve looking like this.

Thus, this neuron’s preferred orientation is ~40 degrees. If we do this for every neuron, we’ll have a starting model of how our brain sees edges.

You just learned a few fundamentals about how your retina sends signals to your brain, creating a picture for the world in front of you.

To come: Code as to how you would model this.

Life is filled with instances – a conversation with a friend, a new concept learned, an experience with a new person. The mission is to find the Class that ties together a set of instances. And then maybe the super Class.

Above and below the layers of abstraction, everything is connected.