Entropy for Information Systems

Posted on 30th August 2010 by ideonexus in Geeking Out,Ionian Enchantment

Entropy is a fairly easy concept to define, the measure of disorder in a closed system, and a rather difficult concept to grasp, but one that furnishes us with wonderful insights into the way the world around us operates. The amount of entropy in the Universe is ever-increasing, the energy concentrated in our sun is constantly radiating away in light and heat, dissipating into an unusable state, absolute undifferentiation.

Sunflower
Sunflower
Credit: riandreu

Living things form “pockets of resistance” to the force of entropy. They do this through syntropy, or negentropy, which is the entropy we export to reduce our internal entropy; in other words, it’s the waste energy we generate to keep our soma in an organized working state. We collect the sun’s waste energy and use it to organize ourselves through syntropy.

How Much Information Entropy?
How Much Information Entropy?
Credit: Moi

In Information Systems, entropy, known as Shannon entropy for Claude Shannon, is the measure of uncertainty in a random variable. A coin toss has one bit of entropy for the 50/50 chance of it turning up heads or tails, 0 or 1. A six-sided dice carries three bits of entropy for the possible outcomes it may produce with each roll (1 (000), 2 (001), 3 (010), 4 (011), 5 (100), 6 (101)). The weather has an amount of entropy difficult to quantify, but it varies from location to location. The weather in New York has more entropy than the weather in Southern California because Southern California has a more consistent climate. Similarly, in our first example, if we were dealing with a rigged coin, one that turned up heads more often than tails, then there would be less than one bit of entropy in each coin toss because we would expect heads more frequently than tails.


On the face of it, the only thing in common with the thermodynamic and information theory definitions is that entropy is a measure of disorder, but the two are analogous in other ways. In our thermodynamic Universe things move toward a state of increasing entropy, and a similar tendency towards a state of total uniformity occurs in an information system, only in reverse.

A living organism in an information system starts out in a world of absolute entropy, nothing is known. As that life interacts with its Universe, the amount of entropy in the Universe decreases for that being, and the amount of its internal entropy increases as what it knows becomes more of a variable to its peers. As the beings living in an information system decrease the entropy of their universe, it tends toward a state of absolute syntropy, absolute predictability.

We exist in a thermodynamic system, and it powers the information systems in our brains, the information systems we construct, and the information system these combine to form in our civilization. The increase of syntropy in an information system comes at the cost of an increase of entropy in the thermodynamic system powering it. Our thermodynamic system is winding down, a bad thing for us, but our information system is becoming increasingly more sophisticated, more syntropic. This brings a deeper insight to H.G. Wells’ prescient observation: “Human history becomes more and more a race between education and catastrophe.”


Notes:

  • With this understanding of Information Entropy, apply this deeper understanding to the Monty Hall Problem (and the interactive demo of information entropy in effect.
  • Playing with a deck of cards is also a fun way to think about information entropy. What’s the measure of entropy in a 52-card deck? What’s the entropy of just the suits?
  • Simulations and bioinformatics are giving us increasing syntropic power over previously chaotic (read “highly entropic”) systems. Chaos theory could just as well be “Information Entropy Theory.”
  • Three Methods for Providing a Print View of a Web Page

    Posted on 11th August 2010 by ideonexus in Geeking Out - Tags:

    Web developers put content online to be consumed, but we have little control over the mediums consuming it. Web pages are rendered on mobile phones, printers, televisions, and their semantic content consumed by a variety of bots, each with their own requirements and best-practices for layout. Here I’ll outline three commons solutions to providing a print version of web page content.

    Defining Printer-Friendly Design

    First we need to define the differences between design for the screen and design for print. Computer monitors and mobile phones are very different media from paper, even when displaying static content. Screens are backlit, while paper is reflective. Web development for the screen must take into account screen resolutions, while presentation on paper must consider page dimensions and toner consumption.

    Jennifer Kyrnin has a definitive list of print-friendly attributes, changes that should be made to web content before outputting it to the printed document. Here are the three I find most important, but the list can get much longer depending on your needs:

    1. Eliminate dark backgrounds and darken font colors presented over them.
    2. Do something with links, either display full URLs, emphasize the text, or remove them altogether.
    3. Remove headers, footers, side menus, and advertising.

    Server-Side Solutions

    One strategy for handling both print and screen layouts of a web page is to have two methods for building the page on the server. For example, w3schools’ “PRINT” button at the bottom of each page opens a new window/tab accessing the same page, but with an “output=print” variable added to the end of the URL, which the VBScript accesses via a GET method. When the VBScript detects the “print” flag, it builds the page without all the extraneous content.

    w3schools Regular and Print View
    w3schools Regular and Print View

    Pros:

    1. Provides a perma-link for users to reference the main content of your page without extraneous information.
    2. Provides a primitive type of Print Preview functionality for the user.
    3. Setting up your application to display print-friendly pages in this way sets it up to display other cool things, like XML versions and REST services.

    Cons:

    1. Print view must be turned on manually, doesn’t automatically render a print-version of the page for printers.
    2. We are making a return trip to the server, and probably a database hit too, in order to get information we are already displaying on the screen.
    3. Provides a perma-link for users to bypass your advertising and menus when referencing your content.

    Strictly CSS Solutions

    Another technique involves applying cascading style sheets to targeted mediums. In the example below, the first CSS file describes the page display, while the second css file describes the page display for printers specifically:


    <link rel="stylesheet" type="text/css" href="screen.css" />

    <link rel="stylesheet" type="text/css" href="print.css" media="print" />

    Alternatively, you can use the @media Rule with an inline style like so:


    <style type="text/css">
    @media screen
    {
    body {font-family:verdana,sans-serif;font-size:12px;}
    }
    @media print
    {
    body {font-family:times,serif;font-size:10px;}
    }
    @media screen,print
    {
    body {font-weight:bold;}
    }
    </style>

    Using this strategy involves identifying all of the items on the page we don’t want printed and setting their styles to “display: none;” so that they appear on our screen, but don’t appear in our print outs.

    Pros:

    1. The same URL automatically renders the page in screen- or print-appropriate layouts depending on the device rendering it.
    2. There is no “print” button or link cluttering up the page, the functionality is abstracted cleanly away from the user.

    Cons:

    1. Having a print.css file with media=”print” is a violation of the DRY principle. It leads to maintaining duplicate code. It’s better to use one CSS file and bracket print and view-specific design choices in @media tags.
    2. Means having to put ID attributes around all the content you don’t want to display in order to hide it from the print view.
    3. Takes some control away from the user, makes it inconvenient for people who want to save your page electronically. I don’t print web pages. I feel it defeats the purpose of using computers. So I like to pull up the print-version of a page in my web browser and save just the article. @media=”print” means I have to File>Print and then save electronically in some BS proprietary format instead of html.

    JavaScript Solutions

    With some JavaScript-driven DOM manipulation, you can come up with a function that will provide a pop-up window “print preview” that displays just the content on the page that you want printable. The following openPrintView() function accepts the id of the html element you want to display in a printable view. This function then uses the printFormat() function to strip the links out of the text provided it.


    <script type="text/javascript">
    /**
    * Strips HREF tags from an html string.
    * Can be modified to strip other tags as well.
    */
    function printFormat(html)
    {
    var container = document.createElement("div");
    container.innerHTML = html;
    var newHTML = container.innerHTML.toString();
    var anchors = container.getElementsByTagName("a");
    for (var i = 0; i < anchors.length; i++)
    {
    var hrefcontainer = document.createElement("div");
    hrefcontainer.appendChild(
    anchors[i].cloneNode(true)
    );
    var href = hrefcontainer.innerHTML.toString();
    var text = anchors[i].innerHTML.toString();
    newHTML = newHTML.replace(href,text);
    }
    return newHTML;
    }
    /**
    * @param html: The block of html to be displayed in the print window.
    */
    function openPrintView(content)
    {
    // specify window parameters
    printWin = window.open
    (
    "",
    "print",
    "width=600,height=450,status,scrollbars"
    + ",resizable,screenX=20,screenY=40"
    + ",left=20,top=40"
    );
    // write content to window
    printWin.document.write('<html><head>');
    printWin.document.write('<link rel="Stylesheet" type="text/css" href="style.css">');
    printWin.document.write('</head><body>');
    printWin.document.write('<span id="printcontrols" style="float:right;">');
    printWin.document.write('<a href="javascript:window.print();">print</a>');
    printWin.document.write(' | <a href="javascript:window.close();">close</a>');
    printWin.document.write('</span>');
    printWin.document.write
    (
    printFormat(
    document.getElementById(content).innerHTML)
    );
    printWin.document.write('</body></html>');
    printWin.document.close();
    printWin.focus();
    }
    </script>

    You can then apply a style rule to hide just the print controls in the new window:


    <style type="text/css">
    @media print
    {
    #printcontrols
    {
    display:none;
    }
    }
    </style>

    Pros:

    1. Uses an inclusion rather than exclusion strategy. You can grab other elements off the page using parent.getElementById() methods and writing them into the window. This way, when new elements are added to the page, there is no need to modify the print CSS to exclude them.
    2. Can be used to strike a balance between presentation control and user control. The JavaScript references just the block of html you want to print, so on a page displaying multiple articles or reports, the user can pop-up a print-view of just the section they want.
    3. Can serve as a “Print Preview” function for slicker web browsers, like Google Chrome, that don’t have that feature.

    Cons:

    1. No permalink with this method (A permalink could be provided with JavaScript that reformats the page onload, but that’s klugey.)
    2. Like the server-side solution, this must be triggered.

    While this last method is my preference because I’m a JavaScript geek, each of these methods is valid. Depending on your skill set and aesthetic, any of these, or a combination of two can serve your visitors well.

    Google Aquires the Semantic Web, or Why Metaweb Matters

    Posted on 20th July 2010 by ideonexus in Geeking Out - Tags: ,
    Isaac Asimov Entry on Freebase
    Isaac Asimov Entry on Freebase

    With Google’s aquisition of Metaweb the searchopolis takes a stake in the seemingly-forever-emerging Semantic Web, a concept with endlessly verbose standards and few demonstrable applications for all it promises. I yawned when I read of Google’s move, remembering a few years ago when I explored Freebase, Metaweb’s semantic database. I even tried downloading and playing with their semantically-rich database version of Wikipedia… and was really unimpressed. It was 3GB of schemas, xml, and ontology, seeming to add up to little of practical value.

    Was I ever wrong. I returned to the database this last weekend and found a community of several hundred users maintaining a browsable schema, with some of these volunteer ontologists having contributed millions of facts to the database the same way people devote thousands of hours to maintaining Wikipedia. On its surface, the database just seems like a bunch of web pages, you click through the associations as you would on Wikipedia, with articles and entries leading into each other in standard hypertext fashion.

    Where the value of semantic associations comes into play is when you experiment with the Freebase Query Editor, where you can search for data in explicit detail, tailoring it to your specific needs. For example, with the following query I can get a list of computer scientists born before 1950:


    [{
    "b:type": "/computer/computer_scientist",
    "date_of_birth": null,
    "date_of_birth<=": "1950",
    "education": [{
    "institution": null,
    "id": null
    }],
    "name": null,
    "id": null,
    "type": "/people/person"
    }]

    A slight modification, and we can restrict the list to just those scientists affiliated with Harvard:


    [{
    "b:type": "/computer/computer_scientist",
    "date_of_birth": null,
    "date_of_birth<=": "1950",
    "education": [{
    "institution": "Harvard University",
    "id": null
    }],
    "name": null,
    "id": null,
    "type": "/people/person"
    }]

    This is a person-instigated search, but Freebase offers many ways for other applications to take advantage of its data. With REST services provided by Freebase, we can run JSON queries to inform our web applications like so:

    http://api.freebase.com/api/service/mqlread?query={%22query%22:{%22type%22:%20%22/people/person%22,%22
    b:type%22:%22/computer/computer_scientist%22,%22name%22:[],%22
    date_of_birth%22:[]}}

    Or, if you prefer XML, try Freebase’s RDF Browser, where I queried for Isaac Asimov in RDF and got back a huge amount of semantic data. Currently the API only allows a measly 100kb of data downloading 100k of HTTP requests a day, but with Google’s backing, this database could start providing semantically-rich data to applications all over the World Wide Web.

    I know Google will keep it free, the company’s business model is built entirely on the universe of free stuff online, and since most of the content in the database is fueled by Wikipedia, it’s in the commons. I trust Google will increase Freebase’s accessibility, making it so everyone can play mashup games with the data, just as they did with Google Maps. And, of course, Google will reap incredible benefits to the services they already provide with this acquisition, meaning it’s a good time to invest in GOOG.


  • Freebase Devlopers Wiki
  • Freebase was inspired by Daniel Hillis’ Aristotle essay in Edge
  • Also check out Google Advanced Search, which I didn’t know about until today.
  • Diversifying Technofauna

    Posted on 12th July 2010 by ideonexus in Geeking Out - Tags: ,
    Pager, Tamagotchi and Psion Organiser
    Pager, Tamagotchi and Psion Organiser
    Pager, Tamagotchi and Psion Organiser
    Credit: steve greer

    Every time a new device comes around, the self-proclaimed techsperts start declaring the death of older devices. It’s boring to keep hearing critics say things like the Smartphone is the “death of laptops,” the Kindle is the “death of books,” and the iPad is the “death of PCs” and the “death of e-Readers.”

    Eken M001 Android Tablet
    Eken M001 Android Tablet
    Credit: mrbill

    I can see the iPad being nice for lying on the couch, flipping through websites and comic books, or playing relaxing video games that aren’t too intense. As appealing a toy as it is, I don’t find it very hard to resist buying the Android tablet equivalent. I get whatever it could provide me between my existing devices, my laptop, e-reader, and Smartphone. Like a lot of other technologies on display at Best Buy, the tablet pc doesn’t fit any particular need in my life.

    IBM Portable Personal Computer
    IBM Portable Personal Computer
    Credit: br1dotcom

    The idea tablet-pc will replace the personal computer is just silly. For being productive, a sit-up, lean-forward device like the workstation, with a keyboard and a mouse for faster input, is the best medium. I suggest anyone who thinks the iPad will replace the laptop try coding for eight hours on one.

    Flashbacks Arcade
    Flashbacks Arcade
    Credit: goodrob13

    Likewise, the tablet-pc won’t replace the Xbox, Nintendo, or Playstation either. Those are also lean-forward media. People don’t lay back for a nice relaxing afternoon of first-person shooters, they lean forward to kick-ass. Hardcore gaming requires a gaming console, with a ergonomic controller, big-screen TV, and sound-system. That’s why PCs are no longer the best medium for gaming either; you want to lean forward on your couch, not upright at a desk.

    Microfische
    Microfische
    Credit: Mosman Library

    The Kindle and Sony e-Reader aren’t going anywhere either. Tablet PCs are not quite relaxing enough and lacks sufficient battery life to completely fill this niche. The liquid ink display is a much more relaxing medium than a backlit display, making the e-reader more conducive to the lengthy, meditative attention novels demand. Plus I can take my e-Reader on a week-long camping trip and catch up on my summer-reading without having to worry about recharging.

    Prehistoric Life, The Definitive Visual History of Life on Earth
    Prehistoric Life, The Definitive Visual History of Life on Earth
    DK Publishing

    The e-reader won’t replace all the books either. There’s no tablet or e-reader with a screen large enough to replicate Albertus Seba’s Cabinet of Curiosities or Prehistoric. I can’t imagine an electronic device that would replace the coffee table book, unless it were a screen built into the surface of a coffee table or an absurdly large and cumbersome tablet.

    Telefunken U-1465 Radio
    Telefunken U-1465 Radio
    Credit: GonchoA

    We live in a world of increasingly specialized techofauna. As our options for digital solutions increases, the market share each new device can claim grows smaller. The long tail is getting ever longer. The tablet pc, e-book reader, and Smartphone are all niche items, rather than serve all our needs as they are advertised, they serve increasingly specialized needs. Any of these devices can serve as our phone, web browser, game console, mp3 player, video player, personal planner, etc, etc, but each of them provides varying degrees of comfort in our consumption of those services.

    None of these are the death of the PC, put they do make the PC into more of a niche item, serving a specific, lean-forward purpose for productivity, just like my light-weight, thumb-sized MP3 player accompanies me on my morning jog, my FM radio accompanies me on my ride to work, my workstation keeps me focused all day at work, while video displays keep me entertained on the cardio machines at the gym afterwards, and my Smartphone serves as a niche-item for keeping me entertained while I stand in line at the grocery store. : )

    Creative Commons License