Nym Notes
From GhettoInstituteOfTechnology
What is entropy? http://en.wikipedia.org/wiki/Entropy
Thermodynamic entropy vs information entropy (uncertainty) The concepts are connected, but some say they're not really similar.
Physical Entropy:
As I shuffle this deck of cards, it's going to tend to a more entropic state (on a micro scale) The act of randomization is leading towards more entropy. Not easily reversible.
Entropy == uncertainty
If you already know the order of a deck of cards, it's at a very low state of entropy. 52 possibilities for the top card in a deck of cards.
Information is subjective.
Different entropic states
How many yes or no questions will it take to identify a number?
2 : 1 4 : 2 8 : 3 16 : 4 32 : 5 64 : 6 64 = 2^6 log^2 64 - everybody loves log!
log^2 63 log^2 62 ... log^2 1
We cannot transmit an average bit
log^2 4 + log^2 3 + log^2 2 + log^2 1
log^2(4*3*2*1) which is ( = log^2(4!) )
In the setup we have an understanding of what information we expect.
Why do we bother with redundant words?
Vectors for data - dna, cards, voltage, light pulses, kitten purs, smoke signals, ...
The one thing that ties all these mediums is that you can ask yes no questions to identify the state
ASCII: a-z, A-Z, 0-9
20!e?
log^2 128 = 7
7 is a shitty number for base 2 purposes, so for ASCII purposes, we use 8 bits
ASCII - http://en.wikipedia.org/wiki/ASCII
There's lots of cruft in ASCII because of historial reasons (carriage returns vs newlines)
Americans are stupid though, which is where unicode came in. Instead of using the wingdings font for representing Greek characters, you just transmit more information.
Notice all the most frequently used words in the english language are short (ie: "two dollar words").
Huffman Coding - http://en.wikipedia.org/wiki/Huffman_coding
Binary Tree - http://en.wikipedia.org/wiki/Binary_tree
