Storage/Backup Solutions

I’ve been working a lot on storage and backup recently. Maybe I’ve gone a bit overkill, but I thought I’d document some of it here.

First, I had written of the Linksys NSLU2 (aka “”Slug”) before, here. That has been up and running really well. I have all the PC’s and MACs in the house rsync to it nightly, and then the Slug has a 2nd drive that synch’s nightly as well. And then I have a 3rd drive off site that I bring in once a month or so to keep up to date.

One issue with this solution is that my drives are all 160 GB and I’m now 70% full. The slug will handle any size drive(s) so that is not an issue, but it would be a bit of a pain and hard work to set up again. But at some point I will do that. I may just buy a new slug and 3 new drives… I haven’t decided yet!

Another issue is that the Mac’s HD was full. I set up an “archive” section on the slug so that we could off-load files manually to the archive and remove them from the mac. That’s fine, but see the point above about my slug drives being too small. :-/

We also archived a bunch of stuff off to CD’s and DVD’s, and keep two copies, one on-site and one off-site. But with many of the recent reviews saying expect 2 – 3 years out of these, that’s a bit scary. Granted, that is with the lower end products, but even if you buy the higher end, name brand CDs/DVDs, how much longer can you really expect? It may be better to just have two large HD’s, one on-site and one off-site, than keep all the CDs/DVDs…

In addition to all that, I’ve been looking at some of the on-line storage and backup solutions. I would never keep my only copy of valuable data at these places, so it’s only one piece of the puzzle. I’ve played with 3 such services and will write mini-reviews in my next few posts.

SPAM

Recently there was a post to an email list I’m on for Chatham county that I responded to, and I thought I’d include it here too:

> Hey all y’all savvy folk.
> I have been getting hit by unwanted, unsolicited, and
> apparently, untraceable emails from accounts that appear to
> be randomly generated.

Multiple layers of defense are needed. I run my own server so I have an advantage over using an ISP or free mail account in that I can tweak things in many ways, but I’ll run through my list anyway and say what ISPs/free mail folks are likely also doing. If you don’t have your own server, look at the last item to get the most bang for your buck. For those that run your own servers, I’ll include my postfix set up for spam at the bottom.

The latest big trick is sending emails with just 1 image, but altering the size, color, etc, so that it is not easy to detect. See #9 below for one way to handle these and other spam.

1. On the server, set up the mail system to reject mail from bad from/reply to addresses, ill-formed domains, etc. Most mail providers also do this.

2. Subscribe to free “real time blacklist” services. These list IP’s known to be sending spam, and the server rejects mail from those IP’s right way. Most mail providers do something like this.

3. On the server, set up greylisting. These means you reject all incoming mail the 1st time you see the sender. Real mail systems will attempt to re-send within a minute or two, and once that happens, the sender is validated. This catches a huge amount of current spam since much of it comes from compromised hosts (bot nets) that just send mail but never listen for incoming messages (such as rejects). Some mail providers do this.

4. On the server, set up a spam analyzing program that uses baseyan filters, such as spam assassin. Have it put headers in the mail before it puts them in the inbox that identify it as spam. Many mail providers do this, and they take the additional step of putting it in a spam folder. But then take the next step of having your mail client, which you said was thunderbird, “listen to spam headers from your server.” This is under tools -> junk mail controls -> trust mail from spam assassin.

5. As a final trick on the server side, I run postfix, which lets me use addresses like “sean+business@” That way whenever I sign up for something or order something from “business” I use that address. If I ever get spam to that address, I know which business gave out my email, and I can stop doing busines with them.

—- From here on you can do things like this on your mail client, this is specific to thunderbird since that is what I use for personal email, but I have similar items in place for outlook at work. —-

6. Set up thunderbird’s adaptive junk mail detection. Tools -> junk mail controls -> adaptive filter. And then train it! Every time a junk mail shows up in your inbox, tag it as junk.

7. I also like to have thunderbird just delete things when i mark them as junk. tools -> junk mail controls -> handling -> when I manually mark messages as junk -> delete them.

8. For mail that thunderbird tags as junk, I have it place it in the junk mail folder and automatically delete it after 3 days. Tools -> junk mail controls -> handling -> move incoming messages determined to be junk -> other -> junk mail -> delete after 3 days.

9. Configure your mail client to not display html emails, but to let you have it show html once you open the email only if you want to. And then tell it not to load images unless you want to.

—- postfix settings —-

For anyone that runs your own server and postfix, here are my settings. Some of these can just be turned on with no set up, while some do need other apps to be installed and running.

smtpd_recipient_restrictions =
permit_mynetworks,
check_client_access hash:/etc/postfix/db/pop-smtp,
check_recipient_access hash:/etc/postfix/db/access,
reject_unauth_destination
reject_non_fqdn_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unauth_destination,
reject_unauth_pipelining,
reject_invalid_hostname
reject_rbl_client relays.ordb.org,
reject_rbl_client sbl.spamhaus.org,
reject_rbl_client cbl.abuseat.org,
reject_rbl_client dul.dnsbl.sorbs.net,
reject_rbl_client opm.blitzed.org,
reject_rbl_client list.dsbl.org,
reject_rbl_client bl.spamcop.net,
reject_rbl_client sbl-xbl.spamhaus.org
check_policy_service inet:127.0.0.1:xxxxx

MOG and last.fm

I’ve been playing with MOG and last.fm a little bit the last few days. For some reason, I’ve always been drawn to my listening habits as tracked by iTunes, so both of these seem interesting as they offer a way to track and “publish” that information.

Here is my MOG page.

Here is my last.fm page.

Both pages don’t have a whole lot of info yet, since I’ve only been using the services a copule of days. You can also click on last.fm under “pages” in the upper right to see local versions of my charts (more on this below).

Both are similar in that you download an application that then monitors and uploads information about what you are listening to to a web page you publish. MOG’s download is more of an agent, that runs in the background. last.fm is a plugin, at least for iTunes.

I thought they might upload historical data, but so far it only seems like data from the time you install the app going forward is uploaded.

MOG seems to be more of a social networking music site like myspace, where as last.fm is a bit more like pandora in that it helps you find music you may like based on what you listen to, and has a player. (MOG claims other MOG users will help you find music, not a “computer.”)

So far, I like last.fm a bit more. Being able to have charts on my personal home page is kind of cool. I’ve actually put them in my blogs pages section. I’ll keep playing with each for the next few days before I can definitively say which I’ll probably continue to use.

eMusic

I’ve been using the iTunes music store pretty much since it came out, and have never had any real issues with the DRM. It’s simple enough to burn CD’s, put the music on your iPod, etc. I would love for Apple to offer a subscription service, but so far they are reluctant to and until one of the subscription services, such as Yahoo, Napster, etc. become a threat, I don’t think they will.

All that being said, sometimes 99 cents per track, or 9.99 for most albums, can be steep if you just want to try some music out. The 30 seconds snippets they provide really aren’t enough in some cases. Of course, when I want to keep the music forever, 99 cents is fine. (I won’t get into the debate over whether I actually own or am just licensing the music here).

So when I saw some ads for eMusic, and their non-DRM’d MP3’s, and the chance for a no obligation trial of 40 songs, I jumped at it. And I’ve decided to stay a member on their basic plan which is $9.99 per month for 40 songs, or about 25 cents a song. You can add bonus packs too if in any given month that is not enough. The other monthly plans are 14.99 for 65 songs and 19.99 for 90 songs.

Their catalogue is not as extensive as iTMS but I’ve found a lot of music I like, including Johnny Cash, Joan Baez, Richard Shindell, Woody Guthrie, a tribute to the Beatles, etc. In other words, I have no problems getting 40 songs a month.

Pandora

Check out http://www.pandora.com. It’s a (somewhat) new web service that touts itself as a music discovery service. You can enter a few artists or songs that you like, and then it begins to stream music to you based on that information. You can then give thumbs up or thumbs down to the music to further tune your “station.”

They’ve undertaken what they call the “Music Genome” project, in which they’ve listened to and analyzed tons of songs, and categorized them (or more accurately categorized many characteristics of them). So based on the input you give it of what you like, they play you songs that have similar characteristics.

So far I’m fairly impressed, though I’m only in day 2. I’ve given it a few artists like Dar Williams, Holly Cole and Donna the Buffalo, that I thought might be too obscure, and they actually recognized them and have played a fair number of Dar songs. No Holly Cole or Donna the Buffalo yet, so my guess is that they don’t have licenses to play those two (yet).

Their web page says they have about 300,000 tracks now. While this doesn’t come near Apple’s 2-3 million, it’s a good start for something that is really only a few months old. Personally I’d like to be able to feed it my iTunes library, at least the play count number of the songs, and have it build my station that way.

At any rate, I wouldn’t be surprised to see one of the major Internet music services purchase Pandora at some point down the road. While I can’t see Apple doing it since they don’t have a subcription model (though Pandora does have an advertising agreement with them as well as Amazon)). Yahoo purchased MusicMatch, which had a similar “recommendation” feature though it was not done as a “genome” project. So I’m not so sure about them. Maybe Napster?

If you want to listen to my radio station that I’ve been working on the past 2 days, click here:

2sparrows radio