branchandroot | Incandescent Outrage, Film at 11

Entry tags:

Incandescent Outrage, Film at 11

You know what infuriates me most about AO3 (today)? The more I read the little things that wranglers anonymous and not are stepping up to tell us, the clearer it becomes that it could work. I don't mean that in a fuzzy procedural way, either, I mean the actual structure of the archive is completely compatible with changing the archive over to canonical, navigable tags and usable fandom hierarchy navigation and not making the wranglers do every damn thing. IT COULD BE DONE RIGHT NOW. Most of the structure is already in place, it's just completely invisible to the users!

It would not take any extra hand-work on the part of the wranglers. It would not require more downtime than any other commit, or break existing functions. The big change would not even be a very difficult bit of code to write! (Terrifying, perhaps, but not difficult.)

How, you ask? Let me tell you, because the top of my goddamn head is about to blow off with the force of my indignation over the pointless ideological stonewalling that's stopping the archive in its tracks!

1) The absolutely necessary first step, on which all else is predicated, is that the public voices of the Archive must speak up to say that Responsible Parties were, after all, mistaken and that the tagging system needs to change. Announce what the changes will be, and a loose timeline of when, so users can take whatever action seems wise to them (eg doing the canonization of their stories themselves so there are no mistakes). Apologize for taking users down a dead-end path for so long and explain the logic behind the upcoming changes (ie readers actually being able to find the authors' stories and wranglers not dying of overwork as the archive grows). Admit that fixing things will inevitably cause some new mistakes, and surface old mistakes. Refrain from any \o/ whatsoever. Do it all again in email.

2) Next step. Make a "request new canonical tag" page. It should be quite simple, as close to one-click as can be. One text-field for each tag type (Fandom, Character, Relationship, Genre/Flavor/Whatever). If Character or Relationship is filled in, a prompt comes up for what Fandom this should be a child of. If a Fandom is filled, a prompt should come up asking for any character/relationships the suggester can think of offhand to populate it with. I'm thinking it should only be visible to logged in users; anything else is spambait. On the wrangler end, this should be equally one-click, as they review requests. Prospective buttons: Approve (immediately creates canonical), Request Review (pops up a flag in whatever task-flow forum exists, asking another wrangler or maybe staff for another opinion), Approve-Needs Forming (creates canonical which the wrangler may alter the phrasing of and optionally pops up a flag asking people to help find synonyms among the non-canonicals to wrangle into the new canonical), Reject (pops up a "reason" form which sends an email to the suggester with the reason filled in; also adds suggestion to blacklist table, with reason; previously blacklisted suggestions do not go through, just pop up a page with the reason for rejection [malformed, malicious, etc.]; the email and this form should both have a link to Support, in case someone wants to argue or get clarification).

3) Next step. Edit the posting form so that Fandom, Character, and Relationship fields can only be populated from the canonicals (possibly re-using the code for selecting a collection name). Fandom(s) remains the only required tag and must be entered first, to create the pool of Character and Relationship options; a warning to this effect should come up if someone clicks into those fields without entering a Fandom. The only other new bit of code required would be a new field for Genre/Flavor/hold a user poll to decide what to call this one. The place where all the No Fandom canonicals will go, at any rate. The Additional tags field can remain as is, in all its freeform glory, perhaps with a note to the effect that Additional tags will not be wrangled, now. Add a prominent link to the "request new canonical" form, and links beside each tag field that will lead to a page of that tag-type for the fandom(s) entered, so people can check what's available instead of having to guess forever. One new note, one new field, four new links, three altered field types. That's it! Test it and send that puppy live. Now all new stories will be in the appropriate format, working off the already existing canonical tag structure and requiring no further wrangling.

4) Adjust import function to look for and set Fandom first, and then try to match any other tags discovered to the fandom's canonicals pool, so as to allow as much of the filling-in process as can be done. Results will probably be about the same as they are now.

5) Write (another) filter sidebar in which only the canonicals show. A fandom index might show Characters, Relationships, and Genre/Flavor canonicals. A character index might show Relationships that character appears in and Genre/Flavor canonicals. A relationship index might also show associated Genre/Flavor canonicals. None of them should show Additional tags at all; instead perhaps there can be a link to the Tags section and a link to the Search form. Revise the in-menu "or" function so that it usefully searches for "character X OR character Y" AND "flavor A OR flavor B" instead of just "anything with X or Y or A or B" which is useless. All of these will be very straightforward mysql queries, instead of monstrous, ass-end-to walker functions. Hold the execution of this until tags are retrofitted.

6) Prep work done! On to the nerve-wracking, if not really difficult, part. Write a query to replace every non-canonical tag id with its canonical version in the story table of the database. Depending on the database structure, this could be as simple as "story.tag_id = tags.canonical where story.tag_id = tags.tag_id and not tags.canonical = ''". Take a good drink to settle your nerves. Clone the story table. Run the query on the clone; this will probably take a few days to get through. Make a news post that The Time Has Come. Take another drink. Shut down the posting form, import any new entries since cloning, swap the names of the clone and the live tables, revive the posting form, now breathe. That should take maybe fifteen minutes, if people prepare beforehand, and that's allowing a margin for a mild case of hysterics or two. If you want to close posting during the replacement process, you won't even have to do the import step. Lo, the archive is running seamlessly on all canonicals! Take another drink.

7) Now it's time for clean-up. Write a query to delete all empty and/or non-canonical tags. Write another to determine all stories with character/relationship meta-tags, the authors' emails, and send them auto-emails informing them that they have meta-tags Y on Story X that should please be re-set to the appropriate canonicals in order for the story to be searchable. Delete these particular meta-tags as they empty. Send general emails to inform all authors that retrofitting is done and they should please check their stories. Warn the wranglers and support people before-hand, because there will invariably be some mistakes showing up and probably some irate users needing to vent. In fact, make a special news post for them to vent in, including any strategies people can think of for easy checking and clean-up, and link to it in the email. Chairs should be on-hand for their volunteers with tea/hard liquor/kleenex/adorable kitten pictures. Schedule this, people.

8) Once the fallout looks to be dealt with, announce completion and success. Now you can \o/. Send the new sidebar live!

9) It's now time to reform the navigation. The Fandoms lists can probably stay as are, at least for now, but every single index page, whether for a fandom, a character, a genre, or whatever else, every index page should show the branch of the hierarchy it is in, as a breadcrumb at the top of the page. For example, selecting "Bishoujo Senshi Sailor Moon" should show, in the breadcrumb "Sailor Moon - All Media Types >> Bishoujo Senshi Sailor Moon". Selecting "Tennou Haruka" from one of those stories should show "Sailor Moon - All Media Types >> Characters >> Tennou Haruka" (or whatever the parent structure is, in that example). Beside the breadcrumb should be a link to the landing page of the fandom. All of these functions and pages exist already, all that is required is to make them visible to the users as well as the volunteers, with a few conditionals to conceal the wrangling links. The public view of the fandom landing page should also have a link to the "request new canonical" form.

10) While you're thinking about it, fix the advanced search, also, so that it has sub-fields for different kinds of tags, and searches for discrete tags as opposed to doing breakage-prone all-field string comparisons.

Congratulations. You now have a working, navigable, professional looking goddamn archive, that can run fast and sleek with as many readers/users as want to come; be proud of yourselves!

Now. That involved only a small amount of new coding, all of it straightforward, and it will fix both server-load and worker-load. The majority of the fix is one query to canonize existing story tags, and a slightly edited form to select new ones, using canonicals and hierarchy that are already established in every case. The rest of it is simply showing users the navigation that's already there. This change-over would not break any existing archive function. It could be nearly seamless. It would even surface things that are currently mis-wrangled but don't readily show it on the front end as the tags stand. And the wranglers would have the far more manageable job of reviewing requests for new canonicals and maybe populating new fandoms instead of trying to make sense of every senseless tag with their hands tied behind their backs. Everything is in place already, to make this work!

It could be done so easily. It could be started right now. WHAT IS STOPPING YOU?

Flat | Top-Level Comments Only

The only real suggestion I have for this is re: relationships. I don't think they should be canonized in the way you suggested(if I'm using the term correctly). I think only the characters should be canonized, and whenever you want to tag with a relationship, you're given the option to select an N, and then input the N characters you have in mind, no matter what canon they're from. After all, 1) what about threesomes and moresomes and 2) there are crossover couples, and I'm sure a million fics spawned by pan-fandom RPs or whatever if nothing else. >.>

*works through that* I think we're meaning different things with "canonical". I mean: "there is a single, official version of this tag, and that is the canonical version". It has nothing to do with what fandom (which I think is what you mean by canon?) the characters come from. The one official version, not infinite multiple versions, is what it means in tagging terms. That's what tag wrangling is, really, linking all versions to the one canonical.

Ah, I think I read something you didn't write, and as it turns out the tag terminology is irrelevant to my point; what I meant to refer to was:

If Character or Relationship is filled in, a prompt comes up for what Fandom this should be a child of.

I don't think relationships should be children of fandoms, because they may very well not be children to single fandoms, you see?

Ah, I see!

Thing is, for navigation purposes, they kind of have to be. Otherwise, we're back to insane scripts that take up a ton of server load. They could easily be children of more than /one/ fandom, though. It would be good to have something in the form to address that. Maybe some kind of "click here to add another parent fandom" thing. In the existing tags, I'm pretty sure that cross-over Relationships already do this.

re: server load, OIC, my programming-fu is non-existent, but I think I see how my initial proposal would have borked things again. But if you're right about what you proposed here, then that would work, of course!

Randomly: I've read and enjoyed your parts in the Choice series you've written with lysapadin! Er, naturally I have not commented to that effect on ao3 or given kudos or anything. ._.

*nods* It isn't instantly obvious, that's for sure, but if the tags aren't all associated in the database, so that tag-37 is listed with a parent of fandom-24, then we can only start from the stories themselves. And going through /every/ story in a fandom to harvest the list of their tags, even only one field of tags... that costs a lot of processing power. But the hierarchy can be flexible! It really looks pretty well set up already, it's just... not /used/ for anything but the fandom-summary pages only the wranglers can see. Total waste.

(*makes happy sounds* Oh, I'm so glad to hear that! Fixing Xanxus is kind of a mutual hobby, and it's endless fun to find new ways to do it.)

As another point, do you think relationships would just have to be alphabetized? I think they should be, just because many fandoms don't care about order and even those that care may have members that are oblivious and don't tag according to convention, and it seems the consistency would be more important. (Of course, they can always specify order in the summary/additional tag field if they must....)

*nodnod* I think it would be wisest to keep relationships purely alphabetical, yeah. It's the established way of doing things on AO3, and, like you say, there's always the Additionals or the summary for saying who's on top, if the author feels it's important. Or user-tags, if some users feel it's important!

(Actually, in that case, couldn't relationships just be children to the characters in the relationship, which are themselves children to fandoms?)

*thinks* I could see some uses for that, yeah. Though I think, for the best outcome when getting results from the database on different kinds of index pages, relationships should also remain children of fandoms too.

That's how it works at the moment, actually. Once a wrangler sets the characters involved in a relationship tag, then the relationship tag is a child of those character tags (as well as all the fandoms those characters are in).

So... would it not work/be just the same as it is now, or can they query the parent fandoms for linkages rather than querying the works, or....?

It could be done so easily. It could be started right now. WHAT IS STOPPING YOU?

Of course, the reason why is:

1) The absolutely necessary first step, on which all else is predicated, is that the public voices of the Archive must speak up to say that Responsible Parties were, after all, mistaken and that the tagging system needs to change.

Though I figured you know that. XD

*glum* Oh yeah. It's not that I actually think it'll happen. Just... so frustrating!

This sounds really sensible. The reduction on the wranglers' time especially, and their frustration about not being able to fix obvious typos. Thanks for outlining how it might be done (and including step one, which is the most important).

*wry* My pleasure. Goodness knows I have to do /something/ with all the frustration.

*grin* If only all frustration could be channelled into cogent suggestions!

Yes, yes please to the 10-step program! Step 1 is crucial, but seeing that and how it could be done is encouraging. Despite being frustrated that the first sentence of step 1 has been roundly ignored for far too long.

Step one would undoubtedly be the hardest. The rest of it is just basic code; that one requires someone(s) pretty high up to admit they were wrong. Despite how really, really obviously true this is. *sighs*

Yes, that is the very frustrating part.

And by now, by this point it would require not just one or two people to admit they were wrong. But also some of the people heavily involved in wrangling who seem to have become thoroughly Stockholmed into the idea that the current system does/can/will work forever.

So unfortunately -- for them and for wranglers wanting change and for readers and for potential volunteers turned off by the (invisibility of) current tag wrangling policies -- they're perpetuating the delusion that the "purest of folksnomies ever! run entirely on manual fan-power! because we're just that awesome!" idea is not wrong. *also sighs*

"purest of folksnomies ever! run entirely on manual fan-power! because we're just that awesome!"

*makes little whimpering sounds* I just cringe every time I run across that. Users are not some kind of guinea pig in a sociology experiment! Wranglers are not anyone's propaganda posters! Worlds of argh.

But noooo, you just don't understand how aaaaawesome it is! And we're sure wranglers would looooove to be propaganda posters! Er, wait, is Stockholm Syndrome infectious?

*snerks* Sometimes I really think it is, yeah.

The thing that kills me, from purely a user standpoint, is that when I post things I WANT to make it easy on the tag wranglers and I CAN'T. Like, it's really a federal fucking secret what the phrasing you want to use for moirallegiance is? Really? So I make my best guess, from batshit-logic that chances between fandoms anyway and based on a structure I'm not allowed to know about. Which then has to have someone come along behind me and fix shit behind the scenes. I do not like enforced helplessness.

*Also--could this be done without step one? Cause that's the only one I really really can't see them doing at all. Like... maybe they could sell it as \o/we're awesome and \o/ \o/ we've figured out how to be more awesome!

Edited 2012-07-10 01:05 (UTC)

Oh god, so much this. I think Step 0 is probably "Tell your freaking users what's going on!". Concealing the tag tree from the users is just such a /bizarre/ decision. I'm deeply grateful that some of the wranglers are finally saying 'screw this' and posting answers anonymously and not.

*considers* There has to be /some/ announcement of the change, but if they want to spin it as "omg, we had this fantastic new idea!"... well, I'll be laughing until I wheeze, and posting about what bullshit it is, but what the fuck. It would leave a lot of the attitudes and patterns that got us into this mess in place, but that may be a much longer fight.

So much this, from the very beginning. I HATE not knowing how I can make things easier on the wranglers.

I really like the writing you've been doing about Ao3 and the OTW lately. You are the only one I have on my reading page that talks about it with any sort of regularity, and I like that you do so without all the \o/ you get with the party line.

*rueful* Which is kind of a sad thing, but I'm glad you're finding these useful, at least!

I do like the re-interpretation that's taking place over the \o/. Person throwing up their hands in exasperation. Person drowning and waving for help. I'm sure there will be more.

Ha, yeah, I know. I do read other places about it, but I don't know those other people, and while I've mostly been a lurker, I've been a reader of yours since damn near the beginning, if memory serves.

At work there is a group chat that I and a few others participate in due to our work, and I picked up the \o/ while in IRC for LJ Support back in 2006 or thereabouts. Every time I use it I remember all the possible interpretations *g*

*grins* From way back, yeah.

Me, I'm just waiting for someone to decide how to indicate that the little person is flipping someone off.

I just imagine that I'm looking at the \o/ from far away and just can't see the flipping off part :D

I offer you "throwing up my hands in despair."

lacey, have you seen

unofficialotwnews? I'm finding it useful reading.

Somehow I was reading it as a face, a mouth bracketed by hands on cheeks (kind of a Home Alone face, if I can date myself). What sort of sound the mouth would be making, tho...

I haven't, but am adding it to my reading list right now! Thank you for the rec!

I like it! Sounds like fresh air to me!

Just as one niggle, I would hope that users could go ahead and post a story even if all the canonicals for it aren't in place/approved (having been the first/only person to post in a fandom before), and I do think it would be good to keep waiting/back-and-forth/"retouch it once you hear from us" to a minimum for users where possible, maybe keep the "uncategorized fandoms" area available for stuff that's pending approval...?

But I'm just poking at your thing because it's so very shiny...

BTW, this whole business just called to mind a TED talk I saw some time ago: Sheena Iyengar on the nature of choice, and on Americans' sometimes muddled view that More Choice (or fewer tagging constraints) Is Always Better.

*nods* I was thinking about that, myself, while I was writing. Uncategorized does seem like a useful holding bin. And maybe some kind of in-line version of the request form, when you hit a "my tag doesn't exist!" button--a dialogue box that comes up on the post page and you fill it out and that triggers a different version of the post form, one that will accept non-canonical input and send up flags and puts the story in Uncategorized for now.

I think you'd definitely want the inline version/the 'my tag doesn't exist' button on the story page, I was thinking that as I was reading your post, and it was the only tiny nitpick I had with it: it does needs to be immediately obvious and accessible how to request new canonicals from the 'post new story' page, or people will get frustrated - an obvious in-line tag request set-up, and a working holding space for stories with unapproved tags would maintain the sense of flexibility and of the archive being open to new/unusual characters/pairings/etc. that the current system offers without the chaos of total tagging freedom.

*nodnod* I was thinking about writing up another post just on what the post form might look like. There would need to be some clear distinctions between "here is the required Fandom field" and "this other stuff is optional" and also "this set of tags will be used in menu navigation, select from a pool or request more" and "this other set will be linked, but not in any menu, totally freeform, knock yourselves out". And also a very clear "mine aren't in the pool, gimme more" that wouldn't interrupt the work-flow too much.

I would hope that users could go ahead and post a story even if all the canonicals for it aren't in place/approved (having been the first/only person to post in a fandom before)

This! One of my favorite features of the tag system is that I can post my obscure-fandom fic and it will have the tags up and searchable right away. Even if they merit editing later, they should be available before the review happens.

Also, from the OP:

Write a query to delete all empty and/or non-canonical tags.

That would be bad. Empty-but-canonical tags often exist because wranglers with some free time pre-emptively created them, so that a future writer posting the first fic with those tags will have the correct formats suggested to them. It's one of the places we can really be proactive and lay groundwork, and it would be silly to undo that when it would likely need to be redone in the future anyway.

Empty-and-noncanonical tags are already deleted automatically. I'm not a tech person and can't break down the details, but I can copy you what it says in the wiki:

The rake task "rake Tag:delete_unused" cleans out unused tags on the archive. Currently it runs automatically every 24 hours, at 7AM UTC.

The rake will remove any tags which are:

--Not canonical
--Not synonyms (merged) with another tag
--Not attached to any works, drafts, or bookmarks (0-use tags)
--Not used in any tag sets
--Not parents of any child tags

*thinks about the query logic* Yeah, you're right; if empty tags are marked as canonicals, they should stay. Presumably, then, the first rake after the canonizing would take care of all the now-empty non-canonicals without an extra task. Bonus.

I agree with you (having prepopulated tags myself), and in Branch's plan it seems like you'd really want to have canonicals prepopulated as much as possible. The only trouble I foresee is making the tag tree visible/navigable to users and having people go "Oh, I wanna read about that character! ::click their tag! 0 works found!::". I'm not a coder and don't know how complicated it would be, but I have to think there would be some possible way for empty canonicals to just not display for non-wranglers --- or worst case, it's small potatoes compared to the current problems.

*nodnod* From the look of the rake task, I think there must be a relatively straightforward way to tell if a tag is not associated with any works. If it runs daily, it can't be one of those crazy "start from the stories" functions; that would take way too long. So it should be easy to say, when producing the tags on a landing page "if there are no works, skip this one".

One of the most frequent things I do, in doing DW product management, is tell people, no, you cannot have an option.

There are times when options are good! There are times when options are great! ...those times are not 'always'.

I just want to say I love you in an entirely platonic way for these posts.

*laughs* My pleasure!

I have a very /pocogorn.gif reaction to most of this mess, but it's super-informative to see a technical breakdown of the trainwreck, too.

This sounds brilliant! Therefore, I doubt it will ever happen.

I mean, I try to put canonical tags on my stories whenever possible, but it's hard to stick all the appropriate ones on if you don't know (and can't find out!) what they are. I also use some extremely non-canonical tags sometimes, but that's because I occasionally want to provide information or commentary that doesn't really fit in a summary or author's note -- and/or I would like that information visible in the fandom works page rather than hidden until someone actively clicks on the story link -- and I would be just as happy having that kind of tag roped off as a sort of... hyperlinked additional author's note, maybe? Instead of jumbled in with all the normal searchable tags. Because that kind of tag is not intended to be searchable and it's stupid to treat them as if they are.

*reads the first sentence and bursts out laughing*

*nods* I think leaving /one/ field open to whatever meta-info the author wants to add is worthwhile. There are always possibilities that the folksonomy drift will come up with some useful things that way, things become frequent enough to be canonized. Like the "women being awesome" tag. And if not... well, it's not eating server time if it's not used in the menus, so if someone wants to post non-search metadata that way, go for it.

I have a question. Concerning tags that are a sort of... hyperlinked additional author's note... not intended to be searchable and it's stupid to treat them as if they are... can you give an example? Because I'm very confused about the purpose of an tag that is not meant to be searchable.

Not actual teal world examples, but many variations exist. "lol so drunk when I wrote this", "Ed has so many, many issues", "tumblr style tags ftw".

Their purpose is to convey the artistic vision of the fanwork as commentary, and I find them very useful to my fannish experience, since it's been quite helpful in avoiding particular works, as the author pov is one I will likely not identify with.

Tag wranglers currently deal with them by assigning to a fandom if applicable and not canonising or marking unwrangable if applicable. (so they don't clog sidebar /drop down)

if a tag is not a character, fandom or relationship, there should be multiple uses before canonising - so for example if a group of people started tagging with "get xxx laid challenge" in order to group works for a particular challenge - that could get canonised.

*enthusiastic nodding* That's exactly what I was thinking. And, as I said to Anatsuno, below, those tags really do have a purpose! Just... not a navigational one. They should totally stay, as meta-comments by the author on zir fic, just not be inflicted on a wrangler until and unless they reach critical mass in the wild.

Some that I, personally, have used include things like "they're pretty crude illustrations though since I am not really an artist" (in reference to an illustrated story, because false advertising in re: artistic quality is nobody's friend!), or "once you create the rest of the world it has to count as much as your special country Mr. Lewis" (in reference to a Chronicles of Narnia story set entirely in Calormen from a Calormene viewpoint), or "minor background creepiness on account of Gamzee" (because there isn't a tag for that, but when the point of the story is fluffy cuddles and snuggles, it's nice to point out a potential mood killer), or "there is not actually any sex in this story" (to qualify a canonical "Threesome" tag, because again, false advertising is nobody's friend).

In other words they're little bits of additional information to convey details about the story -- mood, background assumptions, attitude toward canon, etc. -- that help people decide whether any given work matches what they want to read.

Thank you for clarifying, I appreciate it.

That's information that I would add in the author's notes, or in the summary if I felt it was necessary for the reader to know before clicking through. Is there a reason that you would not do the same thing, given that probably you are the only person who will use these tags, and only in one instance? (I have a feeling we come from different internet cultures re tagging, and it's interesting to me to question my assumptions of Things that Should/n't Be Tags. I mostly picked tagging up from Delicious, where there was a sort of a developing fan consensus over tag formats to allow people to find stuff, i.e., people coming to use the common format "fandom:Homestuck" so that all the bookmarks could be located in one search. That is, a user-need based system, not a creator-need one.)

As I said to Branch, it's stuff that would either be awkward in an author's note - requiring way more words to introduce, or tangential to the rest of the note - or is stuff I want visible on the page(s) before a potential reader opens the story itself.

And, hey, I just realized I have more to say.

There are times when I've been really grateful that existing tags weren't just auto-replaced with the canonical. One of my fandoms was really weirdly organized when I got there -- the part that was the biggest pet peeve for me is that two characters with similar names were wrangled together. If all the instances of "Character A" had been auto-converted to "Character A+", there would have been no way to fix it short of going through all the fics and telling all the authors "hey, I see you were writing about Character A, please go back and change all your tags!" As it stands, all I had to do was unhook the existing "Character A" tag and make it canonical in its own right.

That sort of issue is why I'm still in the "give wranglers more latitude" camp, instead of wanting a full revision to a "run most fields on canonicals only" system.

(Although I really appreciate the thought that went into this plan, and the discussion it's generated. It actually gives people something to chew on, in contrast to most of the "our system is just fine as-is because wranglers are awesome, now stop asking for anything different \o/" official ones.)

*rueful* That's /exactly/ the kind of mistake that canonization would show, and would require hand-fixing by authors, yes. Which does, indeed, suck big hairy ones. But, while I'd be all for a period of tag-grooming and a concerted push to find those things before canonization, I still think that canonization is the direction we need to be moving in. The un-scalability of the "call it anything you like" system isn't going to get better, and the fact that old mistakes will be harder to fix isn't, I think, enough reason to not make the change. It's a really good reason to do all possible to find the mistakes beforehand, though! Even just asking people to start clicking on character and relationship links, in stories, and report any mistakes they find could give that a boost. And it would help the user-base feel like they could do something to help.

I /also/ think wranglers should have more latitude, of course! Letting wranglers contact authors about problems, at the very least, good grief.

I pretty much assumed you would approve of more wrangler latitude, yeah :)

A sidenote: you've got one of my comments on FFA linked in your post the other day as an example of "wranglers who don't feel free to speak in their own names." (The last one.) In fact I've made a couple of critical posts and plenty of logged-in comments on the subject -- and I don't see anything in the other comments to imply that they're anon out of fear, and not anon because, well, it's an anon meme. Things aren't that dire yet.

I did notice you on AW's recent post, yes! I'll add a note to my post tomorrow, then.

I have been contacted backchannel by *thinks* three volunteers in the last month, all of whom asked me not to use their names if I ever mentioned our discussions for fear of repercussions. That... really kind of alarms me. So when I see the lion's share of useful information only being passed over anonymously, I tend to start thinking "oh god".

Thanks for the note :)

Yikes. That's the thing to mention, then!

(Although I would be really curious to know just what kind of "repercussions" people are afraid of. Is this all hypothetical, or is there some kind of backchannel shunning or shaming already going on...?)

Were those volunteers in positions other than tag wrangling? I wouldn't be surprised if fear of repercussion were more likely to be a Thing for people in other volunteer positions that may involve more working directly with other people and groups within the organization. Wranglers, it would be really hard for anyone to effectively shun those of us at the grunt level, because once we're set up with access to everything we need, we can just putter along doing our jobs independently; it's a pleasant position for introverts because you don't ~have~ to chit-chat with other people in order to do your work.

One I know is a wrangler, one I know is in a different area, one I'm not sure about offhand. But yeah, the screenshots Fox has shown me look like the wrangling end is set up with a lot of nice automation!

Although if you are an introvert who's just nestled in doing your job independently without much chit-chat, the idea of being called on the carpet out of the blue can be quite scary. I don't really think it's going to happen to me and don't recall anything anyone's said to make me think it likely, but I understand the fear, especially when you're breaking with an apparent culture/unspoken code of opacity and rah-rah.

(BTW, Branch, LMK if you want any more screenshots.)

For the record, while none of my comments seem to have been directly linked, I'm also one of the wrangler-anons commenting in those threads. It's not due to any sort of fear of repercussion on my part; anonymity is simply the rule of the road for discussions in that comm. I've made similar comments elsewhere while logged in.

While I'm not quite at the public expression of frustration stage myself, I will admit that I have never understood -- speaking here as someone who's married to a person with a library science degree -- why the belief at the OTW seems to be that some kinds of academic work (e.g. media studies) have great value, but other kinds (e.g. library science, taxonomy) have none, and that there is nothing to be learned from the years and years of work that people have put in to learning to classify shit (particularly written-down shit), we should just reinvent classification from the ground up.

Goodness knows I know a lot of fannish librarians. Were I looking to design a data structure for sorting metadata about a set of objects, I would start by asking them for help.

But yes, I very much get the impression that this is a situation where commitment to an intellectual program is getting in the way of keeping the servers from falling down and going boom. And that you can't design a sensible automated metadata sorting system unless you're prepared to admit that that is the project you are undertaking.

So very much this. There are so many resources for this stuff! Use them!

*makes the sign of librarian-spouse solidarity*

I have never understood...why the belief at the OTW seems to be that some kinds of academic work (e.g. media studies) have great value, but other kinds (e.g. library science, taxonomy) have none, and that there is nothing to be learned from the years and years of work that people have put in to learning to classify shit (particularly written-down shit), we should just reinvent classification from the ground up.
As a sometime student of classification, THANK YOU FOR THIS. Centuries of debate and options and theory are available! And they are awesome!

Every argument I've seen about tagging at the OTW is, essentially, something that I've also seen covered just reading over C's shoulder the first year he was working on his MLIS. This business where it's somehow groundbreaking to have these taxonomy arguments all over again just mystifies me -- and I will be the first to admit that this isn't even my field!

But yes: there are a lot of really clever people out there who have spent the last few decades focusing their energies on how to write metadata for online objects. They have come up with at least a couple of good ideas....

Interesting reading and hopefully do-able....

This might also make it possible for people to be one-shot tag wranglers: when a new canon arrives, they could sign on to just add in the new canonicals (at least as far as fandom, character names, canon pairings), which would have the advantages of a) being a small but rewarding bit of work (for volunteers with small amounts of time) and b) setting up a framework for when the fic/art starts pouring in.

I really, really want to see the hierarchy and know what the canonical tags are. (For an example, I just wrote a story for a challenge and as there was no pop-up I just guessed as to what the tag should be. So now there's one challenge with at least three different name tags on the first page....)

I agree, it would be great to have that kind of prep work for a new fandom or pairing available for a fan to put in without a sustained volunteering commitment.

*vigorous nodding* I've been agitating for /years/ for users to have enough access to do some preliminary wrangling like that. I mean, sure, the work should be reviewed. Everyone's work should be reviewed! But populating a new fandom, or even a neglected fandom, would be exactly the kind of finite task with visible reward that gets people to contribute a few hours in passing. So let them!

I wish that the people who run the Archive (... NN, right? and how sad is it that I don't even know this?) would just follow this easy step-by-step process. Seriously.

(I presume the Genre/Flavor category would also include things such as kinks and certain fandom tropes (for instance, vampires, werewolves, etc), yes?)

(Buggered if I know who all is involved, but yeah NN has been pivotal the whole time.)

And yes, that's just what I meant by Genre/Flavor! There's already a threshold of how many times a tag (and synonyms, currently) must be used before it gets a canonical and is made searchable. Things over that threshold that aren't fandom-specific get attached to the parent "No Fandom". So the body of these tags already exists, and each has been given a searchable name. I think there's kind of a lot of them, actually.

No Fandom currently includes about 4380 canonical Character tags, 500 canonical Relationship tags, and 5220 canonical Additional tags.

/drive-by statistic fairy

*grins* Sweet.

...how do Character and Relationship canonicals wind up in No Fandom?

Generic names. Like the character "Jack" and the relationship "Jack/Sam". They're metatags for tags with the characters' full and/or disambiguated names (Jack Harkness, Jack Sparrow, Sam Carter/Jack O'Neill, Jack Harkness/Sam Winchester, etc), which are filed under the fandoms in question.

*facepalm* I knew that!

Isn't it also when "Jack/Sam" ends up in the "Character" field instead of "Pairing" too? Or do those go somewhere else? (I've been out a while.)

No, miscategorized tags aren't canonized at all. They may be auto-assigned the fandom (and, by extension, the media) of the story they were posted on, or they may not be assigned a fandom at all. Generally we tick a box that marks them unwrangleable, after which they show up as little as possible.

Thanks for clarifying; it's been a while!

The main thing is generic un-disambiguated names, e.g. "Harry," or descriptive tags for characters without names, like "Mom" or "a bartender," and relationships composed of such characters; I saw "Harry/Tom" come up as an example of this. Generally they're metatags with disambiguated versions for fandoms that particularly need them (e.g. "Harry Potter/Tom Riddle" or "Mom (Homestuck)") attached as subtags.

(Argh, crossposted...)

Edited 2012-07-10 18:12 (UTC)

I should be running out the door and I’m not - at all - skilled at this on the technical level like you are; I completely agree that the way things work NOW is unsustainable and it needs to change, and I haven’t read the comments (this is my whole disclaimer), but here’s where and why I respectfully disagree with you here - I hope I’ll be articulate enough.

I understand that in other archives since time immemorial the work the archive did, the infrastructure it provided, was to have / relied on having canonical silos. But the AO3 was created precisely with the philosophy that it would support and encourage diversity and not enforce formulations (much) in those so-called canonical things. this was explicitly asked for/debated/decided at the time, I remember it clearly because it was and remains crucial to me.

What I mean is, the fact that I can designate a work of mine as being LotRiPs (fandom) and not Lord of the Ring RPF (fandom) is not unimportant to me - it is one of the basic, fundamental reasons I archive my work at the AO3 when I have never archived it in a fandom-specific or pairing-specific archive before.

What you’re suggesting would totally make AO3 life easier but I can’t agree with your step #3, at all. I wish wrangling would change but I disagree strongly w/ making tags enforceable from the top. Still & forever. The promise of a lack of top-down imposed vocabulary / taxonomy was made, and I want AO3 to keep it.

(aside: that promise is not even implemented enough imo: I am disappointed that while I can - I do - indicate that my fic is a LotRiPS fic on the page itself and in the header, the title of the page in my browser, which also becomes the text of the bookmark if I bookmark the page on Pinboard, say, or the filename if I save the fic, remains the “canonical formulation” - that’s a top-down imposition I resent, not just because I would resent it no matter what, but precisely because it goes against the promise that was made)

The thing is, whether it ruffles coders and librarians’ feathers or not, different ways to format relationship and fandom tags have semantic meaning, sometimes a LOT of it, see many a fandom kerfuffle - so imo users must be able to tag using those formats if we mean to respect them as creators. The whole point of the tag system is to invisibly unify that diversity, not to erase it.

I’m fascinated with

sara’s comment about taxonomy having value as work that is looked down upon in the OTW, and I agree that is a problem - work is work, and librarians indeed have worked long and hard on these issues, we might not have to reinvent the wheel. But I think we’re also doing something that has never been done, precisely because we’re not trying to impose (too much of) a top-down decision on semantic differences that are meaningful to a very wide, very varied and very opinionated crowd. As a friend of mine (a coder, too) put it after reading your post: “So they’re proposing to remove the part of the tagging system that’s more sophisticated than any other site in or out of fandom? No.”

What would help more than enforcing CANONICAL TAGS ARE ALL THAT APPEAR is letting people designate synonyms themselves. So that if I tagged my story with a variant tag, it would say “Is this related to X”? Or, “is there a canon tag you want to hook to?” But I could still tag my stories Star Trek: Alternate Original Series instead of Star Trek (2011), damnit. Just let ME wrangle them as I submit my work; allow users to suggest merges like LibraryThing does, etc. But don’t force my hand.

It seems, from what I hear from within, that building infrastructure for interactive support is regularly backburnered - but it could totally address this.

So yeah, there is a lot to discuss, here. Also, I admit, I'm not just loath to see some of the changes you propose, I’m also irritated that the history of the AO3 seems forgotten; that most people seem to think it all was done wrong because omg some people are egotistical/stupid/inefficient and the Org is a slow dim-witted behemoth, and so on and so forth. There are underlying philosophies behind some historical decisions, if not behind all, and there are ethical discussions that were had at the origin of the archive that we all seem to have forgotten entirely, here.

So yeah, there are reasons why I can tag my fic so the header says it’s a LoTRiPS fic, or a Star Trek: Alternate Original Series fic. There are reasons why I need to be able to say “this is a domlijah podfic” and not a “Dominic Monaghan/Elijah Wood” podfic. And these reasons are why I (and a number of others) archive my work at the AO3. They make the computer work difficult, and organizing wrangling the way it was organized has also required + made human work difficult, so we both agree that some things need to change - but that doesn’t mean disappearing diversity is the answer. We can change in other ways.

At least I very much hope so.

Edited (typo) 2012-07-10 14:28 (UTC)

I hear you. And yeah, I was there for those debates, too. You're completely right that there's a lot of fine-shaded meaning in the variations. And god knows I've been howling, for over a year now, about how users need to be allowed/encouraged to synonym their own tags for any of this to work.

The problem I see right now is: that self-wrangling has never been done. Never been suggested or debated in public. Never appeared as any kind of goal, to the users. The archive is growing fast, and none of its incoming users have been habituated to that way of thinking or doing things. Instead, tagging has been presented as something no one needs to think about or know anything about, except the wranglers. That's a big problem for implementing the idea now. Maybe it could still be done, but unless the archive makes synonyming your own tags a mandatory step, which I can't see happening, it's going to be just as fragile and prone to failure as the current system.

Which brings us to this: The thing is, whether it ruffles coders and librarians’ feathers or not, different ways to format relationship and fandom tags have semantic meaning. It's not about feathers getting ruffled. It's about whether or not the archive is going to actually work for finding the fic that's posted there, and whether it can continue to work as it grows larger. What a lot of taxonomy experts are saying is that there are reasons why what the Archive is doing with tags isn't done elsewhere, and those reasons are not a lack of creativity. It requires the kind of additional labor that becomes prohibitive very quickly as size increases, and even if the labor can be distributed across users it's horribly breakage and accident prone, leaving a significant minority of fic un-findable.

The ideal of letting people use all the fine gradations of naming is a beautiful one. But I am not willing to consider (at a guess) five to ten percent of the archive's stories, which are not going to show up on searches or in tag indexes or sometimes even in fandom indexes, acceptable collateral damage in pursuit of that ideal. If the navigation doesn't work for everything, then it doesn't work.

I would also offer this thought. The underlying structure of the archive navigation is already exactly what I'm suggesting, here, and it shows. If someone clicks on a synonym tag, the title and url and name of the page they reach will not be that synonym; it will be the canonical. The canonicals were already what show up in the filter sidebar, and on the fandom index lists. The only place that alternative forms appear, even now, is on a story's own blurb, and what I am suggesting still leaves the Additional tag field open for exactly those kinds of tags. Those are not navigation tags, and I don't think we should try to force them to be. Those are, as you point out, for meta and commentary and nuance. By all means, let people continue to tag their fic with the forms that will declare their own approach. But I actually think it's a little more honest to the users to admit, and let the division of tags reflect the fact, that the site navigation is already running on lowest-common-denominator canonicals. Because it has to, to work.

I think you've nailed it here.

a) it has to work, and
b) the mechanism has to be sufficiently transparent to the user that they can participate in the sense-making of the archive.

Right now, the heavy lifting of navigation-related sense-making is relegated to the wranglers and made heavier by the (ignorance*) of the users.

There are systemic arrangements that leave, as you say, collateral damage, and those really need to be fixed. I appreciate the folksonomic variations (free-style tagging) that enrich a controlled vocabulary (canonical tagging), and I think that your proposal to use controlled vocab for some fields and have free-form tagging available elsewhere is an ideal solution.

* ignorance only in the sense that because the system is opaque to the users, they can not participate in the process of helping it work better.

You're right about the underlying structure - it's what I'm pointing the finger at when I say the promise hasn't been fulfilled the way it could/should have been (imo, ofc). You can see your offer to make a cleavage between 'personal flavor' and site navigation as more honest, but personally I would still see it as a let down. It's not the same to have an engine that knows that my tag lotrips is like someone else's tag Lord of the Rings RPF tag or to have an separate field for local color. It's not the same semantically and socially, not just computationally.

I understand your argument about not wanting a certain percentage of works to be unfindable, of course. I guess I'm unconvinced that a user-proposed synonymity system + user-powerered crowdsource merging capabilities would really drop that much fic.

(The importance of which is not negated by pointing out, though, that not everyone who archives at AO3 does it to be findable in that way. Some people are perfectly content to be findable by people who know the shibboleth of their fandom in precisely the right way, too. Fandom runs on shibboleths.)

I'm way more inclined to think a (different) solution is, as

troisroyaumes says, to split display name from canonical name. It's not the same as splitting the fields where the info lives into site-wide & user-specific, imo.

(Dropping in from

unofficialotwnews) I feel like there should be a way to modify

branchandroot's proposal so that you have the canonical dropdown that determines where your fic appears but also have another field to type in the display fandom/character/relationship name if you dislike the canonical form. If you don't specify a display name, it'll default to whatever canonical you chose. So canonicals get used for searching, browsing, etc. and serve as the fanwork metadata, but the additional fields are used for the actual information display on your fic. That would stay true to the Archive's philosophy of user choice/expression while actually fixing the search/navigation issues.

However, I think this would involve more coding work than

branchandroot's current proposal. I mean, I am not involved with AD&T in any way, so I don't actually know how their database is set up but I suspect it would require more database wrangling if you had to separate out display from metadata.

ETA: I want to add...I really like this proposal (with the above modification or not) and am hoping that AD&T staffers will take a look and give it some consideration.

Edited 2012-07-10 18:43 (UTC)

If I'm guessing correctly about the current database structure, that could be done fairly easily. It would just reverse who does the synonyming and when, placing the burden on the author who wants an alternate form rather than the wranglers, and only calling on the synonyms when generating that story blurb.

The limiter, there, would be the server load. The more things about any single story/blurb that need an extra query, the slower the page loads. There's an extent to which that kind of problem can be fixed by throwing money into more servers, but that has limits too. I'm not well-versed in really large-scale site optimization, but if the org really can get in touch with Mark and beg a consult, he could certainly tell you how manageable that kind of aliasing would be at different archive sizes.

I think it wouldn't increase server load since the Archive already has to retrieve the original tags used by the author for display purposes. Changing the system so that these are just display strings that don't relate to anything else in the database and don't need to be indexed might actually speed up performance. These would be extra columns in whatever table holds the summary, notes, etc. so it shouldn't increase the number of queries. But yes, I don't know about large scale site optimization either to know for sure.

According to

ira_gladkova's latest post, AD&T is already talking to

mark so that's a hopeful sign.

Aah, okay, if the display fields are added as columns to the story table, yeah, that wouldn't require an extra query. It would make the posting page pretty crowded, though. I wonder if it would be worth it to make showing the display fields a user-setting or an on-demand field...

Good point...an on-demand field sounds like a good idea to me; one could do the same for the "canonical request" field as well. A UX consultant could probably weigh in on how to include the functionality while keeping the form simple, and I know there are definitely UX design professionals in fandom...

Hmm, maybe I'll post about this proposal in the volunteer forums and see if it gets anywhere. Granted, not too many people use the volunteer forums at the moment, but I've seen the AD&T chair post there.

Out of curiosity, because I've always wondered: why is "LotRiPS" vs "Lord of the Rings RPF" a meaningful distinction? I mean, I've always noticed that it is usually called the former in most of the circles i've encountered it in, but not always, and I've always been curious :)

Thank god I'm not the only person who has this question.

hahaha, this is where I show myself to be completely ridiculous - I'm not aware that there is any difference in meaning, *except* that one is a shibboleth and the other isn't - which isn't to say that that I prefer it because it's a filtering device (that's not why I like it).. I prefer it because it's the name of the fandom as I knew it. I was never part of the Lord of the rings RPF fandom, I was (am) "a lotrips fan". Even though both things supposedly mean the same thing, for me, calling that fandom something other than lotrips is like having to change the name I call my aunt by - which I would be happy to do *if my aunt was choosing a new name*, but be horrified at if the government made that choice for her (and me) - do you see what I mean? Lotrips is what the fandom I was in called itself. Lord of the Rings RPF is what outsiders call us.

Ah, got it! Thanks for clearing that up, it's been bugging me for years :)

I feel compelled to add that because one might find my personal peeve re: the name of my fandom ridiculous, that doesn't negate the validity of my argument. There are other examples where the canonical names for a fandom pose much more serious diversity issues (reflect a colonial holdover mentality for example) and alienate people in different ways that aren't just 'anatsuno loves the silly insider name for her fandom too much'. I hope you won't dismiss the idea out of hand - it's a real outreach/diversity issue. It might not be *enough* of an issue for it to take precedence, what with all the problems to solve with the archive, but it's not a ridiculous issue either.

IDK, though, that happens anyway. I still grind my teeth when doing pairing tags for Weiss Kreuz - yes, I can tag the damn story however I want, and the character tag for Omi | Mamoru is now helpfully one tag with a pipe, but on any search and inside the system, my "Naoe Nagi/Takatori Mamoru" story is going to get hooked to "Naoe Nagi/Tsukiyono Omi", and my feelings about the differences in those parings are essentially irrelevant.

Oh, God, I want this.

Back when The Playboy Club was on, I thought two of the characters' last names were Beaseley rather than Beasley for some reason. I wrote fic and tagged it as such, and only months later did I realize I'd been reading and spelling the name wrong the entire time. I was mortified, especially as my stories were among the first posted to the Archive for the fandom and thus had probably been seen by pretty much everyone who went looking for fic.

Had your proposed system been in place, I never would have had this problem. I'm a writer, a reader, and a tag wrangler, and these changes would benefit me in all three spheres. Wouldn't it be nice to think they could occur?

*wry* Yeah, I can't help thinking that it would definitely have side-benefits just like that, in addition to all the intentional ones.

A little more curious poking, would you preserve any metatags for (I'm primarily thinking of) characters? I think we'd be better off without the generic names metatags if we could suss them out, but they're also good for groups/teams; for example I wrangle Fushigi Yuugi and have "Seiryuu Shichiseishi" as a meta-tag with all the Seiryuu Seishi who've been tagged for as subtags (oddly enough, "Suzaku Shichiseishi" is the only one of the four that no one's tagged for yet...). Incarnations/aliases of a character might also seem to make sense as subtags to that character. Showing the menu of canonicals in an outline structure using the metatag relationships sounds sparkly, but would it be practical? What do you think?

"Sherlock Holmes" and the like might be interesting but I think I'd want to know a lot more about how much load it would take before I cared enough to keep it (or not).

That's kind of where I am; I'm used to it and think it's kind of neat, but I would be willing to sacrifice it if it would save load/confusion.

I definitely think there needs to be some policy about limiting this. I'm inclined to think flatter is better unless the reincarnation in question really results in two different people and, from the other end, that amnesia etc. should result in only one character tag even if it resulted in very different personas. And that Sherlock is Sherlock is Sherlock, and let the fandom cue readers which version it is. It's always going to be a judgement call, but "flatter is better" is probably a decent guideline.

Aliases/incarnations are so particular in how they occur in the wild of their canons, I don't think any set of rules will ever feel fully satisfactory, whether the rules are to decide (under current system) "syn/sub/split?" or (under proposed) "deny/sub/split?" or "deny/split?" For me, it kind of feels like six of one, half dozen of the other; no option perfect, any option workable.

The specific example I have on that one is that I wrangle Castlevania, which is a series of video games with self-contained stories mostly overlain on an overall myth arc. In one of the specific sub-fandoms (Castlevania: Aria of Sorrow & Dawn of Sorrow), I have a) a myth arc character who's been reincarnated with no memory of his past life and a much different role and personality, and b) a myth arc character going under an alias that the canon tends to avoid calling him out on (although it implies who he is with a sledgehammer before ultimately confirming it in a bonus character mode) and that fans refer to him as in that context fairly often.

It's workable, though. Currently A is entirely split (on the feeling that if I clicked "Dracula (Castlevania)" I wouldn't really want or expect the Soma stuff to come up). I went back and forth on B for awhile before synning it and wouldn't cry if it was entirely collapsed, although if it was a more active fandom, making people type a common alias like that in every time they wanted to use it might feel like more of an imposition.

*wry* I think "none perfect, any workable" is kind of the motto of any controlled vocabulary system. It's always going to be fuzzy, and there are always going to be judgement calls. It sounds like the wranglers are already pretty well prepared for that, and maybe once the users can see the tree it'll make more sense and they can decide more sensibly whether or not to request a change.

I would want, deeply and passionately, to get rid of character and relationship too-vague meta-tags. And the more I think about it, the more I think a lot of them could be found automatically in the query. I mean, if the fandom is Stargate and nothing else, than it's pretty clear which Jack/Sam it is.

The group-tags... I think you're right. Those have a specific use, and they do allow for helpful subdivision of casts-of-thousands. Someone else was just saying that pull-down menus for Menu Tag fields would be helpful, and I think using the hierarchy like you suggest would allow for doing those multi-level, so the dropdown isn't five miles long. The pairing names would definitely need to go into Additionals, though!

Flat | Top-Level Comments Only

Incandescent Outrage, Film at 11

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject