The only thing I really have to say about AO3's latest contretemps is: I toldyou so~
When a platform is so fucked up that the only way to make it run is to take out the navigation, then it's time to think real hard about who's been driving development.
The tag filters pull data from the 1000 works themselves, which is what they were assuming the page load/cache issues were stemming from. (I am not in any way a dba, so have NO idea how much else might be part of the issue and wouldn't even attempt to venture a guess.)
Tags and tag wrangling themselves are unaffected. Meaning - if you click on a tag - everything connected to that tag will pull up.
Advanced search, however, doesn't work that way - if you search for 'Coulson' in the tag search, you'll only get works tagged with 'Coulson', but if you then click on 'Coulson', you'll get works tagged as all the variations attached to 'Coulson'. (Unless something's changed dramatically that I've not been told about.)
So it's not the 'tags' themselves causing the issue, it's the db walking through each of the works that pulls up in the 1000 results to populate the filter w/ the top-level/canonical tags. Which, if it's one or two people, not a problem, but when it becomes waaaaaay more than that - yeah, problem.
How to fix that? *shrug* Again - I am not a dba of any fashion. I wouldn't even know where to start.
I do know that there are plans to update the filtering system. I don't know what they are or what the ETA is - though it's possibly going to get bumped up considerably given the recent evidence, but that's not my domain.
As far as invites - I'm with ya'll there - just send 'em out and let people in. Though, I don't think an invite is needed just to read on the archive - only if one wants to comment or bookmark w/in the archive.
*hollowly* An individual query. For every tag. On every story. In the whole set.
*puts her head down and just cries*
Oh my /god/, who thought that was a good idea?! How did the fact that the query had to be capped not suggest instantly that the basic concept was flawed, and it would only get worse?
How they fix it is to only put the canonical character and relationship tags associated with a given fandom source in that sidebar. Those to be retrieved with a /single/ query to a new table which uses the fandom source id as the primary key to associate canonicals with the source(s). Faceted filtering by freeform needs to wait until they have got a competent project manager and admin, and probably be a totally separate page in the Tags area.
Oh my unholy fuck. I think I need to go get my smelling salts. No wonder it was such a hideous drain.
This would explain why the sidebar had turned into an unusable mess in fandoms that had had crossovers or drabble dump posts dumped into them. It's not actually that they'd associated all the crossover tags in the database, which always struck me as flat out legit insane and something that really made no sense even for their tag wrangling lunacy, it's that they're walking through all the 1,000 works and pulling the tags from them, so -- you get all the characters from the fandoms that are in no way actually associated the fandom in question.
Exactly. As soon as someone said "starting from the story tags" it all made sense. In a really dreadful way. It's exactly because none of the tags are associated with anything but individual stories.
That graphic. That graphic is what is getting me. They put out there in visual form for all to see how flamingly goddamn incompetent they are.
And they probably think that's a good thing because it's totally a sign that they're Improving Communication and Working On The Problems.
(No, what would be improving communication and working on the problems would be recognising that you have profound and fundamental design problems and at minimum hire (yes, HIRE, as in pay them an appropriate professional fee) a consultant to help you go through the code and identify the problems. And then telling us you're doing that, and why. Or hey, mark keeps offering a few hours FOR FREE. Take THAT up and let us know you're doing so.)
I did not know that Mark had offered some of his time... I may chat w/ our Volunteers chair and see if she can coordinate with him on some stuff. I can't guarantee anything at all, but I can point her in his direction. Thanx for pointing that out.
I cannot recall specific details but, yes, mark has indicated several times he'd be happy to consult, particularly since OTWarchive is the only other relatively known majority-female open source project out there and, well, solidarity.
To date, he's been brushed off as far as I am aware, and since he's not personally invested in the platform sufficient to fight to be able to assist them, it hasn't gone anywhere. Perhaps the word in the right ears, though! I hope something comes of it. :)
Oh jesus the fuck christ, the implications really just hit.
Even if they are caching the works listing, they can't be caching the sidebar if they're doing it like that. There's -- I think I need some damn smelling salts myself, now.
I am wistfully wishing they had been associating all the tags that ever get posted in association with a fandom in the database.
Because that would be an absolutely bizarre act of lunacy, sure, but that would be a case of what else is new hey guys the sky is blue and AO3 handles tags bizarrely.'
But... no.
I am all the fuck out of can. Also, even. I am straight up out of even. I CANNOT EVEN, BRANCH. I ACTUALLY CANNOT EVEN.
I did some testing a few weeks ago, and as of then, they were not caching works listings, or if there was any caching it was a matter of seconds. Every single time I clicked on a page in my "1000 works" list, it was doing a new db search. And every single time, it was regenerating the tags list for the filter. Every time I changed sorting. Every page. Every time I used the filter. Crushing the db again and again. It was jawdropping.
That's my take on how it was explained to me. Looking at the graph, I'm almost certain that's exactly what happens - which explains the load.
*does not code for anything, does a LOT of guessing based on logic*
Personally, I love the *idea* of the filters, but I do agree they are clunky as hell. (I'm assuming others felt the same, hence the redesign that is in progress.)
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
branchandroot
no subject
Date: 2012-06-14 04:24 am (UTC)Tags and tag wrangling themselves are unaffected. Meaning - if you click on a tag - everything connected to that tag will pull up.
Advanced search, however, doesn't work that way - if you search for 'Coulson' in the tag search, you'll only get works tagged with 'Coulson', but if you then click on 'Coulson', you'll get works tagged as all the variations attached to 'Coulson'. (Unless something's changed dramatically that I've not been told about.)
So it's not the 'tags' themselves causing the issue, it's the db walking through each of the works that pulls up in the 1000 results to populate the filter w/ the top-level/canonical tags. Which, if it's one or two people, not a problem, but when it becomes waaaaaay more than that - yeah, problem.
How to fix that? *shrug* Again - I am not a dba of any fashion. I wouldn't even know where to start.
I do know that there are plans to update the filtering system. I don't know what they are or what the ETA is - though it's possibly going to get bumped up considerably given the recent evidence, but that's not my domain.
As far as invites - I'm with ya'll there - just send 'em out and let people in. Though, I don't think an invite is needed just to read on the archive - only if one wants to comment or bookmark w/in the archive.
no subject
Date: 2012-06-14 04:04 pm (UTC)*puts her head down and just cries*
Oh my /god/, who thought that was a good idea?! How did the fact that the query had to be capped not suggest instantly that the basic concept was flawed, and it would only get worse?
How they fix it is to only put the canonical character and relationship tags associated with a given fandom source in that sidebar. Those to be retrieved with a /single/ query to a new table which uses the fandom source id as the primary key to associate canonicals with the source(s). Faceted filtering by freeform needs to wait until they have got a competent project manager and admin, and probably be a totally separate page in the Tags area.
Oh my unholy fuck. I think I need to go get my smelling salts. No wonder it was such a hideous drain.
no subject
Date: 2012-06-14 06:10 pm (UTC)no subject
Date: 2012-06-14 06:14 pm (UTC)no subject
Date: 2012-06-14 06:24 pm (UTC)That graphic. That graphic is what is getting me. They put out there in visual form for all to see how flamingly goddamn incompetent they are.
And they probably think that's a good thing because it's totally a sign that they're Improving Communication and Working On The Problems.
(No, what would be improving communication and working on the problems would be recognising that you have profound and fundamental design problems and at minimum hire (yes, HIRE, as in pay them an appropriate professional fee) a consultant to help you go through the code and identify the problems. And then telling us you're doing that, and why. Or hey,
no subject
Date: 2012-06-15 03:06 am (UTC)no subject
Date: 2012-06-15 09:53 am (UTC)To date, he's been brushed off as far as I am aware, and since he's not personally invested in the platform sufficient to fight to be able to assist them, it hasn't gone anywhere. Perhaps the word in the right ears, though! I hope something comes of it. :)
no subject
Date: 2012-06-14 06:39 pm (UTC)Even if they are caching the works listing, they can't be caching the sidebar if they're doing it like that. There's -- I think I need some damn smelling salts myself, now.
No wonder it's two-thirds of their load, god.
no subject
Date: 2012-06-14 07:41 pm (UTC)no subject
Date: 2012-06-14 08:29 pm (UTC)Because that would be an absolutely bizarre act of lunacy, sure, but that would be a case of what else is new hey guys the sky is blue and AO3 handles tags bizarrely.'
But... no.
I am all the fuck out of can. Also, even. I am straight up out of even. I CANNOT EVEN, BRANCH. I ACTUALLY CANNOT EVEN.
no subject
Date: 2012-06-17 10:37 pm (UTC)no subject
Date: 2012-06-15 03:13 am (UTC)*does not code for anything, does a LOT of guessing based on logic*
Personally, I love the *idea* of the filters, but I do agree they are clunky as hell. (I'm assuming others felt the same, hence the redesign that is in progress.)
no subject
Date: 2012-06-14 07:58 pm (UTC)