Mashing a Map

This post is somewhat disjointed as I needed a bit of background to get to the point.
First off, Nelson Minar is great. He is also a foodie who I have been vicariously living through as he's been traveling in Paris over the last few years. I trust his recommendations and have sent them to family and friends going to Paris quite a few times.
Secondly, I've been getting really into mapping and it's possibilities. One of the recent toys I've been playing with is the excellent Google My Maps (go to maps.google.com and click on the "My Maps" tab). It quickly and easily allows you to mark up a map. Then the Google maps team added an easy way to embed a map.
All of which led me to want to take Nelson's Paris food recommendations and map them. That way I could easily share them with my friends and family and they'd be able to see whether any of his recommendations were near a place they were going. Also, it'd be a nice thing to do for Nelson -- or at least I hoped it would. Interestingly, I didn't have to guess or be a friend of Nelson's to know whether he would be OK with me doing this. He has licensed the content of his blog with a Creative Commons attribution non-commercial license. So I know that as long as what I am doing is non-commercial and that I do proper attribution, Nelson will be happy. Sometimes non-commercial is a hard judgment to make, but not here with my personal blog.
The only two things missing are an ability to allow others to add to the map and a croissant icon for Boulangerie Malineau (though you can create your own icons).
The resulting mashup is below.



View Larger Map -- reviews from Nelson Minar, licensed Creative Commons by-nc-2.0

ShiftCasting not PodCasting

Seems to me that podcasting got off to a bad start with its name. The name is too narrow on two fronts. First, we don't all have iPods. At least one of us doesn't: me. Second, 'casting - which is really short for broadcasting - isn't only about sound, it's also about video and text and images. PodCasting sounds like broadcasting sound to others with iPods. If that was all this "revolution" was about I wouldn't be interested. So, I think we need a new name. How about: "shiftcasting." Narrowcasting is also not a bad one, and often used, but narrowcasting sounds more like broadcasting (real-time consumption) to a smaller set of people.
So, that's the name rant, but the real purpose of this post is to outline some of the key tech steps / alternatives / issues to making that broader concept work in practice. Mostly I am doing this now so that I'll have to write these components and get it working for myself on my decidedly non-pod setup -- the iRiver H340. Hopefully its also useful for others.
Steps, alternatives and issues in shiftcasting:

1. Get a recommendation of content
   
   • an RSS/Atom Feed
     
   • a list of recomendations, that may not be unique resources
     
   • recurring searches?
     
   • news outlets?
     
   • what someone else looks at or shifts
     
   • what groups do
     
   
   
2. Get the content itself
   
   • http and other straight get
     
   • bittorrent or p2p
     
   • tv / radio
     
   • netflix? or other purchase?
     
   • searching to figure out what ambiguous stuff is and how to get it
     
   • making sure you're not geting the same stuff twice
     
   • arbitrage, get the stuff the best way
     
   
   
3. Format the content for the device
   
   
   
   • downsampling
     
   • resizing
     
   • transcoding (both change in format and change in media -- mpeg2 to mpeg4 and text to speech)
     
   • summarizing and excerpting
     
   • removing cruft
     
   
   
4. Put the content on the device
   
   
   
   • be intelligent about what is on there
     
   • may need to queue for distribution later
     
   • may need to generate some sort of "format" for writing to the player (folders? playlists?)
     
   • would be great to also know what was looked at or "played" -- maybe ratings too
     
   
   

SliMP3 on MythTV

MythTV's built in music player is fine but not nearly as good as the Slim Devices system. Like MythTV, Slim uses a server/client(s) architecture [look to the graphic at the bottom of that page]. Slim's server is an open source marvel that has just graduated to version six and has a bunch of great plugins that do everything from reading RSS to listening to the Internet Archive's great collection of concert and other free recordings. It's clients range from slim's own fabulous hardware versions to the open source software version called SoftSqueeze (because slim's new hardware players have been called Squeezeboxes). Can't say enough good things about slim devices and wanted to use SoftSqueeze on my KnoppMyth-based MythTV. Here I'll explain how to get that done.

1. Get slimserver. Set it up following the instructions in the download. I use the tar file.
   
   
   
   1. Download the tar.gz (otherwise known as the Perl Source Code)
      
   2. Untar it with:
      tar -xvzf FILENAME.tar.gz
      
   3. Change to the slimserver directory with:
      cd DIRECTORY_NAME
      
   4. Start the slimserver with:
      ./slimserver.pl --daemon
      (this puts it in daemon mode, be sure that your music directory is accessible for the user running slimserver)
      
   5. Set the options with your favorite browser, for me:
      firefox http://SLIMSERVER.IP.ADDRESS:9000/
      (most importantly, change the server settings -> music folder to the appropriate folder on your system)
      
   
   
2. Get java. This is actually the hardest step. If you don't have java installed, it is a bit of a pain to get it set up. Follow the directions here.
   
3. Get SoftSqueeze. You can get it by opening your favorite browser to the following URL on your slimserver http://SLIMSERVER.IP.ADDRESS:9000/html/softsqueeze/softsqueeze.jnlp but the most recent version will be at SourceForge, also you'll need the jar to incorporate it into the mythtv menus. Unzip it into its own directory somewhere that your mythtv user can access. For me:
   mkdir /home/mythtv/SoftSqueeze
   cd /home/mythtv/SoftSqueeze
   wget SOFTSQUEEZE_DOWNLOAD_URL
   unzip SOFTSQUEEZE_FILENAME
   
4. Make sure softsqueeze works by running it from the command line.
   java -jar SoftSqueeze.jar
   Edit the basic and networking settings to suit your purposes. For me that means setting the SlimServer hostname to the IP of my slimserver and turning on SSH tunnelling. Your mileage may vary. If you have the server on the same box as the client, you can use localhost as the server name and don't need tunnelling. When you are done, click OK, and you should see a SoftSqueeze player that you can navigate with the arrow keys. Once you have verified that, press 'Q' to quit. Of course, there is a handy guide to the SoftSqueeze keys.
   
5. Change your mythtv menu structure to accomodate SoftSqueeze. For me, I added the following to the file /usr/share/mythtv/mainmenu.xml:
   <button>
    <type>PLAY_MUSIC</type>
    <text>Play Music</text>
    <action>EXEC /usr/bin/java -jar /home/mythtv/SoftSqueeze/SoftSqueeze.jar</action>
   </button>
   
6. Enjoy.
   

The one extra feature I would love to have implemented in SoftSqueeze is the ability to specify keys differently from the defaults. Then I could match them up with the ones I use in mythtv already. As it is, I have to use my keyboard and remember another set of keys, which is less than ideal.

Wendy and White Canvases

Seems like every day people (such as Verisign and some at the FCC) forget the importance of a stupid network. As always, Wendy Seltzer is eloquent on the subject:

Painters buy white canvases for a reason. The Internet has succeeded as a platform for innovation because its architecture does not preempt its uses; instead, the stupid network offers a neutral background for line drawing, oil painting, and collage. Sure a grid on the blank canvas would help those making mechanical drawings at the right scale, but it's just noise to the rest, who now need to paint an extra layer to cover it up. Complexity built into the network (such as a search engine that responds to every nonexistent domain name query) may enable a few uses, but it slows or breaks many more, and impedes the development of alternatives.

Painting / Hacking

Lisper extraordinaire, Paul Graham, just posted his essay Hackers and Painters -- apparently originally a speech at Harvard.

The essay explains his choice to follow up CS grad school by studying painting. In so doing he talks about different approaches to programming:

For example, I was taught in college that one ought to figure out a program completely on paper before even going near a computer. I found that I did not program this way. I found that I liked to program sitting in front of a computer, not a piece of paper. Worse still, instead of patiently writing out a complete program and assuring myself it was correct, I tended to just spew out code that was hopelessly broken, and gradually beat it into shape. Debugging, I was taught, was a kind of final pass where you caught typos and oversights. The way I worked, it seemed like programming consisted of debugging.

the dearth of great programming gigs:

All makers face this problem. Prices are determined by supply and demand, and there is just not as much demand for things that are fun to work on as there is for things that solve the mundane problems of individual customers. ... When I say that the answer is for hackers to have day jobs, and work on beautiful software on the side, I'm not proposing this as a new idea. This is what open-source hacking is all about.

and some hopeful thoughts about the novelty of programming as a medium of expression:

Over and over we see the same pattern. A new medium appears, and people are so excited about it that they explore most of its possibilities in the first couple generations. Hacking seems to be in this phase now.

I could quote the entire essay in pieces here. But I won't because you should read the whole thing

Log Retention

Jeff Ubois's notes prior to the Log Retention workshop are worth a read, and Cryptome's got them.

Berkman Courseware is Out!

H2O, the newest iteration of the Berkman Center's courseware has finally been released to the world. Jonathan Zittrain's announcement includes a description of some features. Having logged many hours coding earlier versions of this software (before the professionals came) and helped teach a handful of classes with it, I am very glad to have a version out in the wide open. Congratulations to Berkman!

cMusings Continues the Conversation

Derek Slater at cMusings (now at its spiffy new blogs.law.harvard.edu location) takes issue with my suggestion of basing what artists earn on what their audience actually does with what they create. He writes:

... Alex argues that we should "base pay on what we do with music: listen, mix, and derive new music" rather than copying.

Basing pay on precisely how something is consumed has some drawbacks, though. Monitoring use could be seen as an invasion of privacy. Someone might not want their MP3 player sending statistics back to a server telling it exactly how many times s/he listened to a particular song, for example.

While this is true, I am not sure it is the end of the story for a few reasons. First, the privacy problem is here anyway. As the Verizon case makes clear, your privacy in sharing music is by no means currently guaranteed (actually, since most of the current file sharing products allow users to see what other users are sharing, the only reason I need a subpoena is to find out who you are -- and I may be able to figure that out without a subpoena depending on how much I know about you and how good of a sleuth I am.) Second, the sampling of copying proposals are no less privacy intrusive, they just shift the information that becomes "known": copying v. use. Third, there is no reason to have the data be personally identifiable, or even to need to sample everyone: just as the Neilsen's do television, so too could we have a sample set of music consumers who did not mind the privacy problem. As I and others have suggested elsewhere, this voluntary monitoring may work.

Still, Derek has a point in that monitoring use is a step in controlling use. The more fine grained a monitoring system, the more fine grained the potential for use management. Lack of privacy and control are intimately linked. (This is the personal autonomy type of privacy that runs throughout the constitutional jurisprudence of privacy -- including abortion-rights).

He also makes the good point that I had read too much of my own views into Fisher's proposal. Derek reports that Fisher's proposes sampling downloading, not sampling use.

Later: Derek responded. Good points, raises the problem of being paid for derivative works.

DBD::Google

DBD::Google is excellent! (why open standards based API's kick so much butt) [from Raelity Bytes].

MusicBrainz

MusicBrainz is a great start. The idea is to uniquely mark (I would use the term "fingerprint" but ...) individual recordings. Let that sink in and think about what you could do if you could unambiguously say that you had just listened to x, or that you liked x. First off, it makes a project like MoodMusic easier and better (especially the collaborative filtering part), but also it makes Fisher's plan easier to implement. And, a lot more. [via Archipelago].

Lincoff's Music Plan

As Copyfighter also writes, Bennett Lincoff's music licensing plan is stuck in the past. Copyfighter points out that Lincoff still believes in the old distribution model (from trusted -- read "big" -- sources) and I would add that Lincoff is also still trying to remunerate musicians for copying, rather than listening and using. Lincoff writes: "Royalty distribution would be based on a full census of licensed transmissions." Why transmissions? I get no value from "copying" music or music "transmissions," so why should artists be remunerated on that basis. , Why not base pay on what we do with music: listen, mix, and derive new music.

If we are reforming copyright and looking for a fair way to recompense artists, then there is no reason to dogmatically stick to the old rules.

National Geographic Background

I am posting the following as a bit of celebration after reading the decision discussed below.

National Geographic posts some incredible photographs for use as backgrounds on its Photo of the Day page along with instructions (for Windows and Mac only) on how to make the image your desktop background. The following script downloads the daily background to automate that task. If you need to convert the image, I suggest ImageMagick's wonderful collection of utilities.

#always use strict

use strict;

#for the getting of the page and image

use LWP::Simple;

#get the entry page and look for the link to the big background picture

(get("http://lava.nationalgeographic.com/cgi-bin/pod/PhotoOfTheDay.cgi") =~m!/pod/pictures/sm_wallpaper/(.+?jpg)!);

#store that big picture

getstore("http://lava.nationalgeographic.com/pod/pictures/lg_wallpaper/$1","back.jpg");

I've been using the script for a few months now and seeing a new national geographic picture every day has made me want to resubscribe to the magazine.

More User Configuration Variables

Newsget user configuration of the number of entries shown per screen (Wendy's request) and easy blog entry (my request) are both now operational. There is no way of editing the configuration values yet, so you will have to e-mail me to set yours.

New Door

A great new tool in expanding your reading list was discussed and made available (including source) by Mark Pilgrim today.

Awesome stuff. Definitely a feature I will try to plug into the NewsGet engine. Way to go Mark!

NewsGet Release 0.3

This is the first alpha release. I now also have some testers, let me know if you would like to participate as well.

You can look at the source of NewNews.pm a perl module to be run under Apache. With the next release, I will provide installation instructions.

Aggregator Etiquette

Aggregator etiquette is important. Mark Pilgrim dives into why and how.

GrepMoodMusic Screenshots

Here are some screenshots of GrepMoodMusic that will let me talk a little about functionality before I get more actual documents up (such as source code).

First off, the object of GrepMoodMusic is to manage a large music collection and enable playlists to be automatically generated based on a person's mood, how much the person likes different recordings and whether the person has recently listened to particular recordings. The other goal of the system is to do all of this with as little human rating of recordings as possible.

This is the main interface to GrepMoodMusic. GrepMoodMusic is a combination of several layers of functionality. The database layer (not pictured) stores all the information relating to the recordings, ratings and listenings. The database access layer (not pictured) performs the appropriate queries on the database and returns objects. The interface layer and the layout/format layer (pictured here) displays the information for a user and allows for streaming of the music.
The database and database access layer are the base of the system which can be used with many different interfaces, such as SliMP3, web, winamp, shoutcast or portable device, such as the diamond rio.

Storing information in a normalized relational database with indexes makes retrieving the data easy. There are indexes for genres, artists (pictured here), albums and songs. Each index can be used to look up recordings, stream recordings or shuffle and stream recordings. (Yes, I know the color scheme is horrible).

This shot shows the session specification. The listening user is "macgill", the sample rate is "medium" and the mood is "Wake Up". Everything in GrepMoodMusic is keyed to individual users. It is currently a personal jukebox though I would like to add the ability to have small groups listen and their ratings be combined. It can resample (the code was borrowed from Apache::MP3 as were many of the ideas and the inspiration for this program). Finally, and most importantly, the mood is "Wake Up". Moods can be defined by the user and can be anything the user chooses. Mine are "Wake Up," "Work," "Party," "Mellow" and "Snooze."

This is the Recent Tunes box, which shows the recent tunes played (the last three were part of the randomly generated "Wake Up" setlist that GrepMoodMusic created on the fly based on my preferences. Each of those were rated highly in that mood. This is not always the case. GrepMoodMusic makes educated guesses based on what it knows about the ratings of songs on the same album, by the same artist or of the same genre (it also discounts songs that were recently listened to often). I would like to add tempo to the mix as well but haven't found a good tempo calculator that works in batch mode.

My current to dos are:

1. Squash the remaining little bugs


2. Clean up the code


3. Make an installer


4. Release it


5. Write some external docs


6. Make improvements including:




• Multiple concurent users
  
• Other interfaces
  
• Editing MP3 Tags
  
• XML feeds and interactions
  

What to Code?

I program mostly on the side, for fun. However, from time to time coding projects are directly useful to doing my job.

For the moment, my principal coding projects (links to come) are:

1. GrepMoodMusic: The idea here is to be able to have playlists generated on the fly based on a listener's mood and listening history.
   
2. NewsGet: An RSS feed reader on steriods. It remembers what you read and can accept any feed defined by perl regexes and a uri.
   
3. Fun with the patent databases:The USPTO database contains lots of juicy information that ought to be extracted.
   

I'd also like to learn python and lisp. I still maintain Public.law and HLSNet. And, I am a tester for the h2o Project and the Creative Commons license chooser.