You are viewing joecarnahan

Joe's Journal
browse
my journal
links
June 2012
 

Joe Carnahan
Date: 2012-06-02 23:06
Subject: How to back up LiveJournal Scrapbook pictures
Security: Public
Mood:geekygeeky
Edit: It was not long after I posted this that LJ's image hosting implementation changed again. Unfortunately, I've been very busy with work and a new baby in the house, and so I haven't had time to update this. I won't delete this post, but I do have to warn everyone that it doesn't work right now and has not yet been corrected. Sorry!

As LiveJournal has moved to a new implementation of their image hosting service, the venerable fotoup.pl no longer seems to work for backing up pictures. krellis has been trying to keep fotoup.pl up to date here, but it still didn't know that it was supposed to use the new ic.pics.livejournal.com server.

I didn't relish the thought of trying to update the fotoup.pl script myself, but fortunately I found an alternate set of backup instructions using wget that were easier to modify.

Here are the steps I followed. Note that I use a bunch of Unix commands, which I did using Cygwin but which you could obviously do with Linux or at any other Unix prompt.
  1. In Chrome, install the cookies.txt export extension. If you're using Firefox, you could use a similar extension such as Export Cookies.
  2. Log in to LiveJournal.
  3. Export the cookies for livejournal.com using your handy browser extension and save them in a file called "cookies.txt".
  4. Replacing "USERNAME" with your LiveJournal username, run this command to download all of the web pages under your new Scrapbook "catalog" directory:
     wget --load-cookies cookies.txt -nc -np -r -o crawl_log.txt http://USERNAME.livejournal.com/pics/catalog
    
  5. From all of that HTML that you just downloaded, you want to extract all of the URLs that link to your original images, all of which end with original.jpg. This command should do it (again, replacing USERNAME with your username):
      grep -r original.jpg joecarnahan.livejournal.com | grep _blank | cut -d '"' -f 6 | sort | uniq > original_urls.txt
    
  6. Just to make sure it's all right, take a look at the first few lines of the file you created:
      head original_urls.txt
    
    It should be a list of links that look like this, with USERNAME in place of your username and with different numbers:
      http://ic.pics.livejournal.com/USERNAME/8675309/24601/original.jpg
    
  7. Now that you have the list of images to download, it's time to download them:
      wget --load-cookies cookies.txt -i orignial_urls.txt -np -o dl_log.txt -x
    
    This should create a ic.pics.livejournal.com subdirectory and download all your images under it.
  8. Finally, there's the problem where all of your images have been downloaded with the same name "original.jpg", each in a different directory whose name is a different number. In order to put all the pictures into a single directory, I went into the parent directory of the pictures' directories (something like "ic.pics.livejournal.com/USERNAME/8675309") and ran the following shell script there:
      for NUM in *; do mv $NUM/original.jpg $NUM.jpg; rmdir $NUM; done
    
    (I actually made a backup of all of my ic.pics.livejournal.com directory before doing this, just so that I wouldn't need to re-download things if anything went wrong with the script.)
At this point, I now had a directory containing all the pictures that were previously in my LiveJournal Scrapbook.

Of course, I can think of a couple ways to improve on this. In particular, it would be cool if I had kept the original album structure. This should be feasible, because the original album structure is captured in all of that HTML that I downloaded with the first wget command. So, either by processing each album separately or by looking back at the downloaded HTML after downloading the images, it should be possible to figure out which pictures were in which album.

I haven't gotten that far yet, though. In the meantime, I figured I should share what I have.
I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2012-06-02 11:17
Subject: Redisovering my old pictures from ScrapBook
Security: Public
Mood:excitedexcited
(cross-posted to Google+)

Once upon a time, back before people used the phrase "social network", LiveJournal was my primary social network. I even used ScrapBook, their image hosting service for paid members, as the repository for all of my photos. This included all of my pictures from events in grad school, particularly my first two trips to Sweden.

At some point, I moved on from LiveJournal and stopped paying for my account. At that point, I believed that my pictures were deleted. Of course, I didn't mind because I had the same photos backed up on several different computers.

Over time, I stopped keeping so many computers around the house. (This is a common side effect of getting married or of moving into a small apartment, both of which happened to me in 2008.) Then, at some point, I was running out of space on my main computer, and I deleted a backup directory that I thought was redundant with my Photos directory on my computer.

Except it wasn't. I had deleted my last copy of all of my pictures from grad school.

Fast forward another year or two, bringing us to this week. I received an email from LiveJournal telling me that they had migrated "over 1400" pictures of mine to their new photo hosting service. I followed the link, and sure enough, there they were! I immediately paid them for a short membership to try to keep them from deleting the pictures that I had thought were already deleted, and I went to work figuring out how to download the pictures.

It took a little bit of creativity, as all of the existing scripts out there for backing up one's LiveJournal pictures only worked with their old photo hosting service, but I did successfully download over a thousand pictures, most from my 2007 trip to Sweden for ando and AnnaSara's wedding. It'll take me a while to sort them all out, but I am already backing them up again to various places and will probably share some of them here on Google+.

I'll make a separate post to share the exact steps of how I did it, hoping that these instructions might be useful to someone else. In the meantime, though I figured I should share the good news about rediscovering these pictures!
1 thing heard | I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2012-04-16 09:28
Subject: Google UX research participants
Security: Public
(Yeah, I know, I haven't posted here in a million years, and I doubt anyone's reading. For that very reason, though, it shouldn't hurt for me to post this here as well.)

Like complaining about Google products? (In particular, are you still mad about what they did to GMail?) Want to see prototypes of Google products before they're released? Want to maybe earn gift cards or swag?

Google's User Experience team is looking for participants in all types of user experience research studies. Check it out: http://goo.gl/hxpAV
3 things heard | I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-07-14 23:02
Subject: (no subject)
Security: Public

A coworker calls to ask me a question, then says, "Wait, you're not the Joseph I'm looking for." I reply, "So, I can go about my business?"

I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-07-13 23:02
Subject: (no subject)
Security: Public

Got caught in a summer shower on my way home tonight. It's quite pleasant - The only trick is making sure that my phone doesn't get wet.

I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-07-12 23:02
Subject: (no subject)
Security: Public

From this morning: I'm riding the Acela to NYC a day of meetings. Between free wi-fi and the coffee that I carried on, this is actually pretty pleasant.

From this evening: OK, now I'm on the train back to Boston. Had a couple good meetings and got to meet lots of folks whom I only knew via email and phone until today.

I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-07-01 23:04
Subject: (no subject)
Security: Public
All this buzz about "Eclipse" is confusing me. I mean, sure, I like it better than NetBeans or IDEA, but why the big fuss? ;-) Besides, the "2010 Eclipse release weekend" that I care about happened last week: www.eclipse.org/helios/



Thanks to the Internet, I can easily find the name, samples, and full lyrics to a song that I remember dancing to once in Germany in 1994. Weird.
I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-06-27 23:04
Subject: (no subject)
Security: Public

Finally got around to seeing A-Team. It was self-consciously over-the-top, but enjoyably so. Quote: "Overkill is underrated."

I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-06-24 23:00
Subject: (no subject)
Security: Public

How have I been a professional Java Swing developer for years and not seen this? madbean.com/anim/totallygridbag/

I'm listening | Add to Memories | Share | Link



Joe Carnahan
Date: 2010-06-23 23:01
Subject: (no subject)
Security: Public

I just Skyped my brother and parents in Sweden this morning. I gotta say, technology can be pretty awesome.

I'm listening | Add to Memories | Share | Link