iLike

I love music stats. Recently I had this post about iTunes stats. I also like music recommendations. I wrote about MOG and last.fm here, and Pandora here.

Last.fm is what I have been using the most, and my last.fm profile is here. It combines stats, with a recommendations / social networking aspect.

There is now a new one called iLike on the scene, and so far I really like it. One thing that was missing from last.fm is that it didn’t analyze your listening habbits prior to installing it, so it only showed stats going forward. iLike actually takes the time to analyze your entire library, looking at which artists you’ve played the most and which songs you’ve played the most. It doesn’t do an analysis of your ratings, like iTunes Registry, but that one is a bit of a pain as you have to upload your iTunes xml file manually, whereas iLike does everything once you install it, and continues to update your profile as long as you keep listening.

Here is a snapshot from my profile showing recently played songs, top artists, and top songs.

Here is a link to my iLike profile.

Another thing that iLike is missing is the “weekly top artists” that last.fm keeps, but last.fm only keeps the last weeks information. I’d love to be able to go back to any day, week, or month in time, to see what i was listening to way back when.

iLike also let’s you hook up with friends that use iTunes and iLike, and it let’s you listen to what they listen to. I don’t have any iLike friends yet, so I haven’t been able to use this feature. I just installed it a couple of days ago, but if you are reading this, and you use iTunes, give iLike a try and let me know, so we can hook up.

As far as the recommendations, it does a couple of things. First, it shows other users that have similar listening habits (even if you don’t know them) and what they listen to. Second, it shows songs you may like, and let’s you listen to them. Some of them are even free MP3 downloads, so that is pretty cool.

So far, it does seem to slow things down a bit in iTunes, but I haven’t really spent that much time figuring out if that is the case — it could be that I recently got iTunes 7.0.1 to actually work. But that is my only complaint.

iTunes Stats

iTunes has an XML file that it uses to track all sorts of neat things, like ratings, play counts, date added, last played, etc. Some of these stats are useful within iTunes, to sort by date added, or play count, or ratings. Or to make smart playlists, etc.

For example, I have a smart playlist called NeverPlayed, which lets me see all the music that I’ve never played since importing it to itunes. Since I spent a fair amount of time a couple years ago re-ripping all of my CD’s, I have quite a few songs that I’ve never played. In fact, it is over 3000, but I’ve been letting the NeverPlayed playlist play a couple hours every day, so I’ll get that down eventually. Though I have to admit there are some songs that I probably don’t ever want to play! 😉

(BTW, I re-ripped to standardize. I started ripping CD’s sometime in 2000, and over time ended up with various formats, like mp3, ogg-vorbis, aac, etc., and different encoding rates. I really wanted to standardize to 192K variable mp3, so that is what I did.)

But iTunes doens’t provide much more insight into all the stats then what I’ve just listed. I do use the last.fm plug in, so I’ve got stats from when I added that a few months ago, but even that is limited (and I wanted stats for all-time, not just when I added the last.fm plugin), so I searched around. I found one called SuperAnalyzer that runs on Mac and Windows. (It’s a java applet.) It provided a little information, but not much. Mainly that I have too many tracks that don’t have all the information filled out. :-/

I also found iTunes Registry, which let’s you upload your XML file and then does an analysis on it. One thing that is neat about this site is that it combines all users’ data that is uploaded to it, so you can see some “global trends.”

I uploaded my XML file, and here’s a link to that page. I did find some interesting info — like I’ve listened to Dar Williams the most. And it gives some insight into how you rate artists, etc.

Anyway, there are probably a lot more cool things that could be done. I’d really like to know percentages like how much a given song, artist, or album, comprise my total listening, or total listening in the last year, or a given month, by song/artist/album/genre, or what percentage of songs are responsible for X percent play time (i.e. how long is my song tail?) Now not all of that would be availble, as the XML file doesn’t keep everything. But some of it would be possible and would be interesting.

Carbonite

Carbonite is the last online storage/backup site I’ve tried, and so far I’m sticking with it on my PC. It is $5/mo for any amount of storage! As my last post stated, with most home broadband, anything over 40 or 50 GB is probably not worth it, though if you have a few PCs, you could quickly get over that amount and recovery on just one would be reasonable.

One review I read thought Carbonite had a simplistic interface, but I actually like it. You can use windows explorer and right click on a drive, folder, or file, and tell Carbonite to back it up, or not back it up, or back up this file type, or not back up this file type. It seems to use less bandwidth while I’m doing other things per my mrtg graphs. It is still going, but I realized that when I clicked on my “data” drive, that there were lots of things in there that did not need to be backed up. For instance, I back up my internet server to my data drive, and while I may back up some of that in more than one place, my users Maildir’s don’t need to be backed up againa and again!

Carbonite has a 15 day free trial, which is what I’m in now, but I imagine I will keep using them past that. The do not let you use your own encryption key yet, but say they will in the future.

There is no Mac support for Carbonite, though they say the will have it sometime this year. I saw that for Mozy as well, BTW.

Mozy

Mozy is the 2nd storage/backup site I tried. It offers 2 GB free, or 30GB for $4.95 per month, or 60GB for $9.95 per month. For home use, where you are probably stuck with upload speeds of 380k – 1 Mbps tops, going much past 40 or 50 GB for backup is probably not worth it. It will take days to upload, and days to download during a recovery. Once you have uploaded everything and are just synching, it doesn’t much matter. But the recovery would not be fun if you had too much data! (Then again, your download speed is probably more like 1.5 – 3 Mbps, or even 6 Mbps in some cases, and then maybe it wouldn’t be so bad.)

I set up Mozy on a friends computer that I was working on. (Friends and family often ask me to look at their PC’s due to performance problems. The most common problems are lack of memory, disk fragmentation, spyware/adware, and tons of temporary files… ) Anyway, this PC just had 1.3 GB of data on it so I set up mozy to upload it. Mozy did it in just about 8 hours. My mandwidth use at that time was my max of 520Kbps upload, but I should note I was also running a different service on a different computer at the time, so it’s hard to say if Mozy would always max it out…

One of the nicest features of Mozy is that you can keep your own encryption key. Some services, such as streamload, don’t even encrypt your data. Other services encrypt it, but they keep the private key. Mozy will either keep the key for you, or let you keep your own. This is probably not an issue for lots of folks, and can be very dangerous. If you lose your key, there is nothing anyone can do to recover your encrypted data!

Mozy had a little app that you download, where you can tell it file types, or just give it the files/folders you want to save. Overall, I think Mozy is a nice little service. I plan on using the free service with some docs for now as I try both Carbonite and this out.

If you want to try Mozy, click here to signup so I get a little more free space:

Thanks!
Sean

Streamload

Streamload is one of the 1st online backup and storage sites I tried. With up to 25GB free, it was the site that offered the most free storage that I could find. Granted there were limits on the downloads per month, but since I only wanted it for backup, that was fine.

They offer a few ways to upload content. First you can use your browser and standard buttons/forms. But that is far too tedious for any number of files. Second, you can use a java applet. That is better, but if you are backing up a lot of stuff, that still seems cumbersome. Finally, they offer a downloadable application that lets you tag files/folders, to keep up to date. That would be the best method, but the app kept locking up on me.

I finally gave up and moved on to a couple of other services…

(One other note in case Streamload fixes the app… They do not encrypt your data, so you’d have to do that yourself if you want it secure. Granted they do offer “sharing” of data, so it wouldn’t make sense to encrypt that. But I’m not looking to share, just looking for stroage/backup.)

Storage/Backup Solutions

I’ve been working a lot on storage and backup recently. Maybe I’ve gone a bit overkill, but I thought I’d document some of it here.

First, I had written of the Linksys NSLU2 (aka “”Slug”) before, here. That has been up and running really well. I have all the PC’s and MACs in the house rsync to it nightly, and then the Slug has a 2nd drive that synch’s nightly as well. And then I have a 3rd drive off site that I bring in once a month or so to keep up to date.

One issue with this solution is that my drives are all 160 GB and I’m now 70% full. The slug will handle any size drive(s) so that is not an issue, but it would be a bit of a pain and hard work to set up again. But at some point I will do that. I may just buy a new slug and 3 new drives… I haven’t decided yet!

Another issue is that the Mac’s HD was full. I set up an “archive” section on the slug so that we could off-load files manually to the archive and remove them from the mac. That’s fine, but see the point above about my slug drives being too small. :-/

We also archived a bunch of stuff off to CD’s and DVD’s, and keep two copies, one on-site and one off-site. But with many of the recent reviews saying expect 2 – 3 years out of these, that’s a bit scary. Granted, that is with the lower end products, but even if you buy the higher end, name brand CDs/DVDs, how much longer can you really expect? It may be better to just have two large HD’s, one on-site and one off-site, than keep all the CDs/DVDs…

In addition to all that, I’ve been looking at some of the on-line storage and backup solutions. I would never keep my only copy of valuable data at these places, so it’s only one piece of the puzzle. I’ve played with 3 such services and will write mini-reviews in my next few posts.

SPAM

Recently there was a post to an email list I’m on for Chatham county that I responded to, and I thought I’d include it here too:

> Hey all y’all savvy folk.
> I have been getting hit by unwanted, unsolicited, and
> apparently, untraceable emails from accounts that appear to
> be randomly generated.

Multiple layers of defense are needed. I run my own server so I have an advantage over using an ISP or free mail account in that I can tweak things in many ways, but I’ll run through my list anyway and say what ISPs/free mail folks are likely also doing. If you don’t have your own server, look at the last item to get the most bang for your buck. For those that run your own servers, I’ll include my postfix set up for spam at the bottom.

The latest big trick is sending emails with just 1 image, but altering the size, color, etc, so that it is not easy to detect. See #9 below for one way to handle these and other spam.

1. On the server, set up the mail system to reject mail from bad from/reply to addresses, ill-formed domains, etc. Most mail providers also do this.

2. Subscribe to free “real time blacklist” services. These list IP’s known to be sending spam, and the server rejects mail from those IP’s right way. Most mail providers do something like this.

3. On the server, set up greylisting. These means you reject all incoming mail the 1st time you see the sender. Real mail systems will attempt to re-send within a minute or two, and once that happens, the sender is validated. This catches a huge amount of current spam since much of it comes from compromised hosts (bot nets) that just send mail but never listen for incoming messages (such as rejects). Some mail providers do this.

4. On the server, set up a spam analyzing program that uses baseyan filters, such as spam assassin. Have it put headers in the mail before it puts them in the inbox that identify it as spam. Many mail providers do this, and they take the additional step of putting it in a spam folder. But then take the next step of having your mail client, which you said was thunderbird, “listen to spam headers from your server.” This is under tools -> junk mail controls -> trust mail from spam assassin.

5. As a final trick on the server side, I run postfix, which lets me use addresses like “sean+business@” That way whenever I sign up for something or order something from “business” I use that address. If I ever get spam to that address, I know which business gave out my email, and I can stop doing busines with them.

—- From here on you can do things like this on your mail client, this is specific to thunderbird since that is what I use for personal email, but I have similar items in place for outlook at work. —-

6. Set up thunderbird’s adaptive junk mail detection. Tools -> junk mail controls -> adaptive filter. And then train it! Every time a junk mail shows up in your inbox, tag it as junk.

7. I also like to have thunderbird just delete things when i mark them as junk. tools -> junk mail controls -> handling -> when I manually mark messages as junk -> delete them.

8. For mail that thunderbird tags as junk, I have it place it in the junk mail folder and automatically delete it after 3 days. Tools -> junk mail controls -> handling -> move incoming messages determined to be junk -> other -> junk mail -> delete after 3 days.

9. Configure your mail client to not display html emails, but to let you have it show html once you open the email only if you want to. And then tell it not to load images unless you want to.

—- postfix settings —-

For anyone that runs your own server and postfix, here are my settings. Some of these can just be turned on with no set up, while some do need other apps to be installed and running.

smtpd_recipient_restrictions =
permit_mynetworks,
check_client_access hash:/etc/postfix/db/pop-smtp,
check_recipient_access hash:/etc/postfix/db/access,
reject_unauth_destination
reject_non_fqdn_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unauth_destination,
reject_unauth_pipelining,
reject_invalid_hostname
reject_rbl_client relays.ordb.org,
reject_rbl_client sbl.spamhaus.org,
reject_rbl_client cbl.abuseat.org,
reject_rbl_client dul.dnsbl.sorbs.net,
reject_rbl_client opm.blitzed.org,
reject_rbl_client list.dsbl.org,
reject_rbl_client bl.spamcop.net,
reject_rbl_client sbl-xbl.spamhaus.org
check_policy_service inet:127.0.0.1:xxxxx