fandom Meta: archiving stories directly from LJ

[arrow00]

So, this is all (once again) cesperanza's fault. It all started with a posting she made in which...well, I'll let you read it for yourself if you're interested, but what came to me while reading it was we need better archiving. Not that existing archives aren't beautiful and whatnot, but there is no tie in with LJ, and the LJ community is thriving, and many people (such as me) are posting directly to LJ and not bothering to archive (for a variety of reasons.)

The problem with the LJ community is: stuff gets lost. I discovered that myself when digging up stories for my reclist. There are just too many wonderful gems disappearing into the mists of time. The folks at places like ds_weekly and others do a fantastic job of keeping us in touch with each other on a weekly basis, but they aren't searchable...and there is just no guarantee that LJ won't disappear like the wind in a desert.


The Proposed Solution

What I'm proposing is this: I'm willing to write an archiving script that will essentially pull LJ story postings and all the accompanying meta information and stuff them into a searchable database.

It would only do so for willing writerly participants. All that would be required of them is that they friend the archiving LJ users (ds_archivist or ts_archivist to start with) and follow a particular template when posting wherever they usually do, something like this:

Title: Guess Who's Coming to Dinner
Author: arrow00 (must be lj username)
Pairing: F/V
Rating: PG
Summary: Ma Vecchio and Dead!Bob find an uneasy truce.
etc. (TBD)

(Hm. I think I might actually write this story.)

The participants can post to whatever communities they like; I will simply make the archivist users join all the communities involved and check those communities and users daily for new stories.

What do you guys think? Will this fly? Are people willing to have their stories added to a centralized, searchable archive automatically?

The beauty of this system is a user can change their story and have it rearchived if they make their edits and then change the date. Sure, it will result in stories showing up more often (maybe they can preface the story with [posting updated] or something so people won't get mad.)

Obviously, there is more to think about and plan, but I find myself massively excited by this idea: it might bring the power and features of LJ to join with the missing features of an archive.

Heck, we can even (I think) include a comment box on the bottom of the story so folks can send their comments right back to the original LJ post.

Oh, and the best part about this: LJ will have already formatted the story html all pretty. I'll just grab everything between the cut tags.

Comments

[sallymn.livejournal.com]

I only have a few stories, but any archive not on OTW I'd be happy to be in...

[vsee.livejournal.com]

Hiya, arrow,

I am the "some woman" (eeep!) cesperanza was referring to in that post, and I was sort of quoted out of context anyway. I am very much in favor of more widespread archiving. See my follow up post over here.

Your system is something like what I was picturing. I'd like to see if it is possible to only slightly alter existing habits (something like inserting tags or whatnot) and using the existing power of the resources we use.

[karieflybabe.livejournal.com]

This is why I have a separate journal for my stories, to control confusion. It may be simple to others but I like it just fine. I use a variation of your proposed template.

Title:
Author: KarieAuthoress
email: karierauthoress@gmail.com
Permission to archive: Yes, but please tell me where.
Fandom: The Sentinel
Genre:
Pairing:
Summary:
Warnings:
Notes:

For all my stories, I do this.

For TS writers though, I think there is an LJ community who is at least attempting to help us keep track of some of our stories.

I'll keep an eye on this for later...

[spuffyduds.livejournal.com]

That sounds awesome and painless. (Well, not painless for YOU, but for the people getting authomatically archived...)

[laurie-ky.livejournal.com]

I'd do it.
Laurie

[pir8fancier.livejournal.com]

Thumbs WAY up for me!

[jesse_the_k]

Archiving is greatness.

The rest of this post is blither.

I've just arrived at dS fandom, and I would love to have an LJ-oriented archive. I try to keep current with new fic by following the weekly communities, picking up new names from the discussion and then using that info to search on DSA, but I know I'm missing wonderful stuff.

Consider: this fandom is a decade old. Testimony to the crack that is dS, and the Properly Prepared writers who improve on canon daily. But did someone penning the very first CoTW fic, on 13 May 1998, think I'd want to read it in 2008?

[morgandawn.livejournal.com]

I like this idea - if I grok it correctly, your database will only archive the tag info - being able to read the actual story texts will still be dependent on whether the community is still online (or the writer).

As a side note to the community maintainers: I encourage them to use the current LJArchive tool and back up their communities every week. I wish there was an automated tool for that.

[aprilvalentine.livejournal.com]

There's been a long discussion on SENAD recently about just this issue. Would you be doing stories for just DS fandom or could there be TS stories too?

[true-brit.livejournal.com]

Brilliant concept! But you're still going to write, right?

*regards arrow sternly from beneath lowered brows*

[t-verano.livejournal.com]

My brain, she is not working well, but from the little I grasp of this idea, I say "Hey, cool!" (I think. But then, I don't even know what OTW is. However, anything that makes fic more findable and more safely / redundantly stored makes me very happy, so I'd be in.)

BTW -- I know I have a bunch of your stuff to catch up on. ::drools (attractively... well, or not):: RL has kept me from being on LJ much for six (freaking) months now, except for sporadic little microbursts here and there, and I have approximately six million three hundred and eighty-two fics and friends' posts to catch up on. I may require IV amphetamines (and RL is still not cooperating as fully as it might), but eventually I intend to get a little less behind than I currently am. (God, I hope so. It's driving me crazy, everything I haven't read. Or said... )