HomeSearchPrevious dayNext day

00:03:22Barto (Barto) joins
00:20:10<@EggplantN>JAA is it worth making the death watch into a basic table?
00:20:23<@EggplantN>I was wondering about this when me and Harry were going through it
00:20:27<@EggplantN>Would make my life easier
00:22:24<@JAA>EggplantN: Given what a mess the tables on the URLTeam page are, I'm not convinced. It would probably improve some things while significantly worsening others.
00:24:17Barto quits [Ping timeout: 244 seconds]
00:25:31<@JAA>I don't think there's anything fundamentally wrong with the current structure of Deathwatch. It's just a big page, and adding more stuff to it would make it much larger.
00:26:23<@EggplantN>Hrm
00:26:30<@EggplantN>Maybe if we put the things in bold that were done?
00:26:39<@EggplantN>Just a simple mark to let me know "ignore this"
00:26:53<thuban>EggplantN: would require a mass edit for style--old entries had the site name on bold
00:27:37<thuban>*in
00:28:18<thuban>JAA: i disagree, slightly--the 'pre-emptive alarmbells' section (especially "misc ideas") is a mess, and should go at the bottom or under deathwatch/misc or both
00:28:24<@JAA>Bold would rather indicate 'hey, pay attention to this' for me personally.
00:28:52<@JAA>Also, 'done' is not a simple binary thing on archival.
00:29:56Barto (Barto) joins
00:29:58<@JAA>thuban: Ok yeah, agreed on that section.
00:30:16<@EggplantN>It isn't but true. Kinda thinking just something so I can pick out sites todo
00:36:20<thuban>archivebot reference group + special styling for ab references in text + use of the job template in footnote (with template itself updated to whatever viewer JAA comes up with) + maybe adding a date field to the template (so that it's visible from the wiki page rather than requiring you to clock away to the viewer)?
00:36:55tzt quits [Ping timeout: 252 seconds]
00:37:36<thuban>i am also in favor of more small-job tracking on deathwatch; decent compromise between that and page volume?
00:37:54<thuban>*click
00:39:01<@JAA>Whatever we do, let's make it generic, not specific to AB. Lots of things are archived outside of AB, and AB crawls often (especially due to JS) aren't complete anyway.
00:40:50<@JAA>Perhaps we should just be more liberal with creating project pages even for smaller sites.
00:41:13<thuban>ah fair enough, i remember being concerned about socialscrape/chromebot jobs (but iirc your viewer will also handle those?)
00:41:20<@JAA>But ultimately, it's always going to be underdocumented because people seem to prefer to spend time archiving shit over spending time documenting archiving shit.
00:41:25<thuban>^ i think realistically no on eis joing to do that
00:41:27<thuban>yeah
00:42:07<thuban>*is going (christ i'm fucked up today)
00:42:08<@JAA>socialbot doesn't do anything on its own, so not directly. chromebot will be in there, yes.
00:43:00<@JAA>Also, AB job IDs in the new indexer will change to uniquely identify a unique run rather than ambiguously referencing a URL.
00:43:16<thuban>great!
00:43:23<@JAA>DPoS projects will eventually also be there.
00:44:00<thuban>oh, that's actually really cool.
00:44:00<@JAA>At least future ones, probably not past stuff because the relevant metadata to implement it generically is missing.
00:44:38<thuban>mhm
00:46:24<@JAA>First I'll need to finish the indexer anyway.
00:47:07<@JAA>Too many unfinished projects. :-(
00:47:27<thuban>using a generic reference group & template is probably trivial
00:47:29<thuban>i think i'm in favor of leaving off dpos references for projects that have their own pages, though (more context + backwards compatibility + reduce page size where redundant)
00:47:35<thuban>aren't there always
00:48:17<thuban>(that is, dpos references _on deathwatch_, adding them on the project pages seems like an obvious win)
00:49:47<@JAA>Agreed.
00:50:17<@JAA>I've been meaning to overhaul the project page infobox for ... uh, years.
00:50:39<thuban>what did you want to fix?
00:52:02<@JAA>Support for non-DPoS archival, hiding rows that aren't relevant, no default placeholder image, link to archives, page title as box title by default, and some other things that I don't remember right now.
00:52:18<@JAA>There's actually a 'sandbox' template somewhere on the wiki with some of that stuff.
00:53:13<@JAA>https://wiki.archiveteam.org/index.php/Template:Infobox_project_sandbox
00:53:23<@JAA>September 2018, eww.
00:54:28<@JAA>Another one that recently came up: decommissioning dead project IRC channels.
00:55:17<thuban>yeah, i remember that discussion
00:55:29<@JAA>That will require a mass edit of all project channels. Much fun will be had.
00:55:33<@JAA>project pages*
00:57:38<thuban>if someone patches the template and gives me a cutoff date (or other rule of thumb) for decommissioning channels, i will do it :I
01:03:04<@JAA>Idea: add an irc_dead parameter to Template:Infobox_project. When present, it triggers the display of '{{IRC|archiveteam-bs}} (<s>{{IRC|deadchannel|network=EFnet}})' (using the values of irc and irc_network, obviously).
01:03:18<@JAA>+ </s>
01:03:33<@JAA>Perhaps some {{Grey|...}} as well.
01:04:43<thuban>'formerly' might be more comprehensible than strikethrough
01:04:45<thuban>but yes
01:05:13<@JAA>Yeah, I was trying to keep it compact.
01:05:28<@JAA>But 'formerly' is fine as well.
01:05:37Church quits [Ping timeout: 265 seconds]
01:05:49<@JAA>Oh yeah, the other fun mass edit will be changing the default IRC network.
01:05:54<thuban>née :P
01:06:11<@JAA>:-)
01:12:19hooway quits [Client Quit]
01:18:35<Lord_Nightmare>JAA: we did an archive back in 2017-ish of every single size of every emote on twitch
01:19:10tzt joins
01:19:28<Lord_Nightmare>urls like https://static-cdn.jtvnw.net/emoticons/v1/1/1.0 https://static-cdn.jtvnw.net/emoticons/v1/1/2.0 https://static-cdn.jtvnw.net/emoticons/v1/1/3.0 for the three sizes
01:19:41<Lord_Nightmare>and the '1' is replaced by a number between 1 and ... i think its over 20 million now
01:20:46<Lord_Nightmare>no, it ends somewhere around 2 million, a bit above that
01:21:00<Lord_Nightmare>lots of holes from deleted and rejected emotes though
01:24:32<Lord_Nightmare>odd. i think the emotes started at 1 and end around 2048576 (possibly not accidental that this is a power of 2)... then start again at 300000000
01:24:49<Arcorann>2^20 = 1048576, 2^21 = 2097152
01:24:55<Lord_Nightmare>er yeah
01:24:59<Lord_Nightmare>2097152
01:25:09<Lord_Nightmare>it might be coincidence though
01:25:31<Lord_Nightmare>but newer emotes seem to start around https://static-cdn.jtvnw.net/emoticons/v1/300175000/2.0
01:26:33<Lord_Nightmare>there's also the 'emoticons v2' interface which is at https://static-cdn.jtvnw.net/emoticons/v2/28087/default/light/2.0 but the emote ids are the same, just url format is different
01:28:21<Lord_Nightmare>and theres a third, NEW interface i just saw today, where the emote ids are encrypted or encoded somehow, which looks like https://static-cdn.jtvnw.net/emoticons/v2/emotesv2_e0968475c5e643869b96df9bc9835326/default/light/1.0
01:29:36<Lord_Nightmare>interestingly emotes in this most recent 'new' format are not properly scraped by twitchemotes.com etc
01:29:52<Lord_Nightmare>i wonder if all new emotes for 2021 and onward are in the encrypted/non-linear id format
01:30:34benjinsmith is now known as benjins
01:32:51<Lord_Nightmare>oh I get it! the last type of emote is special, its a bit-badge-locked emote, unlockable using bits, not a normal channel emote
01:35:08BlueMaxima quits [Client Quit]
01:44:41Jens quits [Quit: Jens]
01:45:05Jens (JensRex) joins
01:52:34<Lord_Nightmare>305768198 is the highest 30xxxxxxx emote id i've seen so far
01:52:45<Lord_Nightmare>but there's probably newer ones too
01:53:30<@JAA>Sounds like a fun little qwarc job.
01:54:51<Lord_Nightmare>305768498 is newer; based on rekmak (the streamer in question)'s twitter i think that emote is from after feb 2nd 2021
01:55:02<Lord_Nightmare>so that's right around the bleeding edge of 'new emotes'
02:01:35<Lord_Nightmare>305811772 was added yesterday or the day before, i scanned twitter for people posting about 'new emotes' in the past 2 days or so
02:01:58<Lord_Nightmare>that means twitch is adding about 20,000 new emotes each day? is that right?
02:03:17<Lord_Nightmare>i suspect emote 'number slots' are being taken up by emotes which aren't even approved yet, many of which never will be
02:03:23<Lord_Nightmare>hence why the numbering has so manu holes
02:03:27<Lord_Nightmare>*many
02:24:38systwi_ is now known as wsystwi
02:24:49wsystwi is now known as wystwi
02:25:05wystwi is now known as systwi
02:29:02BlueMaxima joins
03:12:17aleph joins
03:28:05nepeat_ joins
03:28:05nepeat quits [Read error: Connection reset by peer]
03:34:48TheLovinator quits [Quit: Ping timeout (120 seconds)]
04:03:31qw3rty__ joins
04:07:21qw3rty_ quits [Ping timeout: 265 seconds]
04:17:04<thuban>archivebot jobs take 1-2 days to get uploaded/make it into the viewer, correct?
04:20:48<@JAA>When everything is running smoothly, yes. Can take longer at times.
04:23:25<thuban>thanks
04:25:10OrIdow6 quits [Ping timeout: 252 seconds]
04:34:41OrIdow6 (OrIdow6) joins
04:48:29DogsRNice quits [Read error: Connection reset by peer]
04:51:01OrIdow6 quits [Ping timeout: 252 seconds]
04:56:17aarchi (aarchi) joins
05:08:17etnguyen03 quits [Client Quit]
05:35:32OrIdow6 (OrIdow6) joins
05:41:34astrid (astrid) joins
05:58:12daalekz (daalekz) joins
06:44:51<atphoenix>thuban, I ran some KhanAcademy jobs for deprecated content in AB late last year. I think they ran okay with concurrency 1. I have a list of additional KA content that has not yet been run. I probably also posted that list the AB chat.
06:47:54daalekz quits [Client Quit]
07:19:19Mateon2 joins
07:21:14Mateon1 quits [Ping timeout: 244 seconds]
07:21:15Mateon2 is now known as Mateon1
07:49:16BlueMaxima quits [Client Quit]
07:54:10OrIdow6 quits [Ping timeout: 252 seconds]
07:57:10sec^nd quits [Remote host closed the connection]
07:57:34sec^nd (second) joins
08:01:43mutantmonkey quits [Remote host closed the connection]
08:02:00mutantmonkey (mutantmonkey) joins
08:09:11atphoenix quits [Quit: Leaving]
08:09:49atphoenix (atphoenix) joins
09:02:17OrIdow6 (OrIdow6) joins
09:25:31spirit quits [Client Quit]
09:43:00<purplebot>4chan edited by Wickedplayer494 (+129, /* List of Fuuka Archivers by board …) just now -- https://www.archiveteam.org/?diff=46295&oldid=45743
10:00:34hooway joins
10:20:00britm0b quits [Ping timeout: 265 seconds]
10:30:59godane joins
10:32:01Barto quits [Ping timeout: 252 seconds]
10:32:45Barto (Barto) joins
10:33:32godane1 quits [Ping timeout: 265 seconds]
10:41:42Barto quits [Ping timeout: 244 seconds]
10:42:13Barto (Barto) joins
11:32:51no112 quits [Ping timeout: 244 seconds]
11:45:04Barto quits [Ping timeout: 265 seconds]
11:47:11Barto (Barto) joins
11:55:01no112 joins
12:01:47OrIdow6 quits [Ping timeout: 244 seconds]
12:25:22OrIdow6 (OrIdow6) joins
14:00:08ragu__ joins
14:01:00<purplebot>Webzdarma edited by Sanqui (+25, /* ArchiveBot jobs */ webzdarma_mwlinkscrape_11) just now -- https://www.archiveteam.org/?diff=46296&oldid=46286
14:03:13ragu_ quits [Ping timeout: 252 seconds]
14:25:55etnguyen03 (etnguyen03) joins
14:26:30Arcorann quits [Ping timeout: 265 seconds]
15:15:19OrIdow6 quits [Ping timeout: 265 seconds]
15:18:02OrIdow6 (OrIdow6) joins
16:10:34britmob joins
16:11:55aleph quits [Ping timeout: 252 seconds]
16:29:32britmob2 joins
17:12:46nertzy quits [Ping timeout: 265 seconds]
17:12:50onetruth joins
17:14:47godane1 joins
17:15:54onetruth quits [Read error: Connection reset by peer]
17:17:22godane quits [Ping timeout: 252 seconds]
17:22:42onetruth joins
17:36:37godane1 quits [Ping timeout: 252 seconds]
17:40:26nertzy (nertzy) joins
18:08:08aleph joins
18:49:13OrIdow6 quits [Ping timeout: 252 seconds]
18:51:09OrIdow6 (OrIdow6) joins
18:51:12OrIdow6 quits [Remote host closed the connection]
18:51:41OrIdow6 (OrIdow6) joins
19:24:08<@EggplantN>billy549 #mediaonfire
19:24:09<@EggplantN>:D
19:43:53hooway quits [Client Quit]
19:45:19hooway joins
19:53:54Kenshin quits [Client Quit]
19:55:04Kenshin joins
20:17:28<@EggplantN>hey arkiver beyond webbed anything else coming up this week?
20:18:49ThreeHeadedMonkey quits [Ping timeout: 244 seconds]
20:20:47ThreeHeadedMonkey (ThreeHeadedMonkey) joins
20:48:03DogsRNice (Webuser299) joins
20:58:33<@JAA>I started a qwarc crawl of the inQuake forums since I'm not sure the AB job will manage in time. I'm grabbing the thread pages and afterwards the archive thread pages. ETA for the first part is a bit under 10 hours, the archives should be faster based on tests earlier.
20:58:37<@JAA>jodizzle, Fusl: ^
20:58:53<@Fusl>ty
20:58:55<@Fusl>lmk if you need help
20:59:13<@Fusl>happy to throw ips at this since EggplantN hasnt announced the /22 yet
20:59:42<@JAA>All good, it looks like their server can't handle all that much, I already pushed the avg response time from ~250 to ~1000 ms.
21:05:18<@EggplantN>did i not Fusl
21:05:19<@EggplantN>>_>
21:05:34<@EggplantN>ffs
21:05:37<@EggplantN>i didnt uncomment
21:05:38<@Fusl>nope, they're still all not announced
21:05:43<@EggplantN>well lemme know whenever I can again
21:05:43<@EggplantN>my bad
21:05:52<@Fusl>uh
21:05:53<@Fusl>now
21:06:05<@EggplantN>sec
21:16:04betamax quits [Ping timeout: 252 seconds]
21:33:38<jodizzle>JAA: That makes sense, thanks.
21:34:05<jodizzle>I guess if the qwarc job works out and there's still time, we can tune the AB job to just grab the user pages it's picked up.
21:53:36BlueMaxima joins
22:02:42dbgft joins
22:04:51<dbgft>Sorry, guess I have to go here.
22:04:52<dbgft>https://archive.org/download/birds-of-the-world-a-multimedia-guide
22:05:00<dbgft>Found this today at the thrift shop.
22:05:25<dbgft>Do I need to provide scans?
22:07:27<dbgft>Nevermind, someone else got there first.
22:07:36<dbgft>https://ia802808.us.archive.org/20/items/birds_202002/IMG_0003.jpg
22:09:37<dbgft>This isn't there yet: https://www.amazon.com/Vivendi-Universal-71295-Newtons-Quest/dp/B00005KG69
22:10:05<dbgft>I'm uploading this now.
22:25:02<Sanqui>i approve of birds.
22:35:02daalekz (daalekz) joins
22:59:30<thuban>does warcio have documentation? like... at all?
23:09:26<@JAA>thuban: Only the README as far as I know.
23:09:39<thuban>oof
23:10:04<@JAA>Yeah
23:12:03<thuban>unfortunately it looks like hkpl sent archivebot a page with a completely different javascript viewer than the one they send (a) my browser and (b) curl, so i can't do the token extraction from the warc data
23:32:00<purplebot>URLTeam edited by Aarchi (+265, /* Archives */) just now -- https://www.archiveteam.org/?diff=46298&oldid=46256
23:32:41hooway quits [Client Quit]
23:46:47Arcorann (Arcorann) joins

HomeSearchPrevious dayNext day