HomeSearchPrevious dayNext day

00:55:37AlsoHP_Archivist joins
00:55:54AlsoHP_Archivist quits [Remote host closed the connection]
00:57:02qwertyasdfuiopghjkl quits [Ping timeout: 244 seconds]
00:57:07AlsoHP_Archivist joins
00:57:24AlsoHP_Archivist quits [Remote host closed the connection]
00:58:37AlsoHP_Archivist joins
00:58:54AlsoHP_Archivist quits [Remote host closed the connection]
01:00:05dm4v quits [Client Quit]
01:00:07AlsoHP_Archivist joins
01:00:24AlsoHP_Archivist quits [Remote host closed the connection]
01:00:50AlsoHP_Archivist joins
01:01:29dm4v joins
01:01:31dm4v quits [Changing host]
01:01:31dm4v (dm4v) joins
01:03:28Gaelan quits [Quit: ZNC 1.8.2 - https://znc.in]
01:04:27Gaelan (Gaelan) joins
01:12:14Gaelan quits [Client Quit]
01:12:47Gaelan (Gaelan) joins
01:31:17driib798943 (driib) joins
01:33:46driib79894 quits [Ping timeout: 240 seconds]
01:33:46driib798943 is now known as driib79894
02:03:06dm4v quits [Ping timeout: 240 seconds]
02:03:45dm4v joins
02:03:48dm4v quits [Changing host]
02:03:48dm4v (dm4v) joins
02:09:06Church quits [Ping timeout: 240 seconds]
02:09:06tbc1887 (tbc1887) joins
02:14:51Church joins
02:21:19AlsoHP_Archivist quits [Client Quit]
02:22:23HP_Archivist (HP_Archivist) joins
02:23:51HP_Archivist quits [Client Quit]
02:24:55HP_Archivist (HP_Archivist) joins
03:04:49HP_Archivist quits [Ping timeout: 252 seconds]
03:19:29ThreeHM quits [Ping timeout: 252 seconds]
03:21:40ThreeHM (ThreeHeadedMonkey) joins
03:22:20<@JAA>https://twitter.com/time_kz/status/1478313119990624258 There was an attempt. lol
03:50:28wickedplayer494 quits [Ping timeout: 265 seconds]
04:14:22sonick quits [Client Quit]
04:31:48wickedplayer494 joins
04:53:36IDK_ quits [Quit: Ping timeout (120 seconds)]
04:53:52IDK_ joins
05:52:04kiska5 quits [Quit: Ping timeout (120 seconds)]
05:52:17kiska5 joins
06:47:15qwertyasdfuiopghjkl joins
11:31:53tbc1887 quits [Read error: Connection reset by peer]
11:41:41BlueMaxima quits [Read error: Connection reset by peer]
12:18:22<wizards>is the '#' character used in c, c++, or c# for anything other than preprocessor directives?
13:15:29Arcorann quits [Ping timeout: 265 seconds]
14:20:04HP_Archivist (HP_Archivist) joins
14:20:22HP_Archivist quits [Remote host closed the connection]
14:21:34HP_Archivist (HP_Archivist) joins
14:21:52HP_Archivist quits [Remote host closed the connection]
14:23:04HP_Archivist (HP_Archivist) joins
14:23:22HP_Archivist quits [Remote host closed the connection]
14:24:34HP_Archivist (HP_Archivist) joins
14:24:52HP_Archivist quits [Remote host closed the connection]
14:26:04HP_Archivist (HP_Archivist) joins
14:26:22HP_Archivist quits [Remote host closed the connection]
14:26:45HP_Archivist (HP_Archivist) joins
14:47:17<anarcat>JAA: did you see https://utcc.utoronto.ca/~cks/space/blog/web/WebScrapingItsNotJustLoad ?
14:55:46sonick (sonick) joins
15:35:52<@JAA>anarcat: I did not, thanks.
15:36:08<anarcat>JAA: apparently we're not "polite" enough
15:36:42<@JAA>¯\_(ツ)_/¯
15:36:51<anarcat>i wonder if EMTs are supposed to say "i'm sorry sir, you seem to be having a heart attack, would you mind us taking you to the nearest hospital so we can save your life?"
15:46:26<@JAA>jodizzle: ^^
15:47:15<@JAA>anarcat: And I suppose if someone's unconscious, well, tough.
15:47:38<anarcat>yep
15:47:50<anarcat>and when you do EMT long enough, you stop caring about distinctions like this
15:47:55<anarcat>say hi, then do your job
15:48:10<anarcat>which is basically what we do, start crawling, and if it hurts you, we slow down
15:48:18<anarcat>i think those people don't understand how on fire everything is
15:48:40<anarcat>i subscribed to this guy's rss feed
15:48:43<anarcat>it's a frigging firehose
15:48:46<anarcat>one post a day
15:48:54<anarcat>pretty interesting in general, but it's kind of hit and miss
15:48:56<@JAA>Sounds like we should archive it regularly. :-P
15:49:00<anarcat>haha
15:49:02<anarcat>lol
15:49:06<anarcat>we should so totally do that
15:49:25<anarcat>ooh the poor guy, he's going to get so upset
15:49:35<anarcat>lol i just can't stop laughing
15:49:38<anarcat>must be monday
15:51:11<hexa->I'm still baffled there are entities that have enough space to archive all of this madness
15:55:38<anarcat>it is kind of counter-intuitive
18:02:47fofofofofofofofo quits [Ping timeout: 252 seconds]
19:06:14<@rewby>That's like what... A request a second?
19:06:21<@rewby>If that
19:06:42<@rewby>If your site can't handle that without breaking a sweat I have questions
19:10:34<@rewby>Ah it's some python cgi monster site...
19:13:06<@rewby>That said, how would we "improve"? Just go slower?
19:14:05<@rewby>Also, did they consider talking to us before getting angry?
19:16:30<@JAA>At default settings, AB typically does a 2-4 req/s, but yeah.
19:17:04<@JAA>No attempt to contact us as far as I know.
19:18:32<@rewby>Also just telling us off for not being polite without defining polite
19:18:42<@rewby>That said, does ab honor robots.txt?
19:19:25<@JAA>Nope
19:19:58<@JAA>And not rel=nofollow either.
19:20:21<@rewby>Okay, we do have contact into in UA right?
19:20:43<@JAA>The thing is that it's often not reasonable to follow those hints because plenty of sites put content behind those to not get indexed by search engines.
19:21:07<@rewby>Yeah, but it could be an idea to have that as a toggle
19:21:13<@rewby>So it honors by defaukt
19:21:29<@rewby>But can be told to Just Do It ™
19:22:19<@JAA>No direct contact info in the UA, but it starts with 'ArchiveTeam ArchiveBot', so it's trivial to find us from that.
19:22:43<@rewby>Additionally we could get clever and automatically raise the crawl delay based on response latency
19:22:54<@JAA>The default UA is something like this: ArchiveTeam ArchiveBot/20190427.01 (wpull 2.0.3) and not Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36
19:22:55<@rewby>So slow sites get even less load
19:23:17<@rewby>That UA could probably do with a link to the wiki tbh
19:23:41<@rewby>Then again, who knows what UA detection that'd set off
19:23:55<@JAA>Yeah, we're already running into too much of that.
19:24:08<@JAA>Anyway, we're well into on-topic now. :-)
19:24:58<@rewby>Good thing this is -ot
19:25:22<@rewby>Oh *on*topic
19:25:27@rewby is blind
19:36:58DogsRNice (Webuser299) joins
19:49:15spirit joins
20:17:38G4te_Keep3r2 joins
20:18:05G4te_Keep3r quits [Ping timeout: 252 seconds]
20:18:05G4te_Keep3r2 is now known as G4te_Keep3r
20:56:05<anarcat>so this happened https://github.com/github/dmca/blob/master/2022/01/2022-01-04-readium.md
20:56:40<anarcat>aka https://github.com/noDRM/DeDRM_tools was pulled for a relatively bogus DRM claim, according to https://news.ycombinator.com/item?id=29870331
20:56:57<anarcat>questionable, but the thing is on IA and latest master downloadable as zip https://web.archive.org/web/20211231094826/https://github.com/noDRM/DeDRM_tools
20:59:11<anarcat>and of course that means "mirrors" are popping up https://github.com/cryptonek/DeDRM_tools
21:07:37<@rewby>I do believe this is one of the reasons we're bringing the GitHub project back
21:07:52<@rewby>Not this case specifically, but things like this
21:25:23spirit quits [Client Quit]
21:54:31Arcorann (Arcorann) joins
21:54:52Arcorann quits [Remote host closed the connection]
21:56:00Arcorann (Arcorann) joins
21:56:22Arcorann quits [Remote host closed the connection]
21:57:30Arcorann (Arcorann) joins
21:57:52Arcorann quits [Remote host closed the connection]
21:59:00Arcorann (Arcorann) joins
21:59:22Arcorann quits [Remote host closed the connection]
22:00:30Arcorann (Arcorann) joins
22:00:52Arcorann quits [Remote host closed the connection]
22:02:00Arcorann (Arcorann) joins
22:02:23Arcorann quits [Remote host closed the connection]
22:03:30Arcorann (Arcorann) joins
22:03:52Arcorann quits [Remote host closed the connection]
22:05:00Arcorann (Arcorann) joins
22:05:23Arcorann quits [Remote host closed the connection]
22:06:30Arcorann (Arcorann) joins
22:06:52Arcorann quits [Remote host closed the connection]
22:08:00Arcorann (Arcorann) joins
22:08:23Arcorann quits [Remote host closed the connection]
22:09:30Arcorann (Arcorann) joins
22:09:52Arcorann quits [Remote host closed the connection]
22:11:00Arcorann (Arcorann) joins
22:11:23Arcorann quits [Remote host closed the connection]
22:11:39qwertyasdfuiopghjkl quits [Ping timeout: 244 seconds]
22:12:30Arcorann (Arcorann) joins
22:12:52Arcorann quits [Remote host closed the connection]
22:14:00Arcorann (Arcorann) joins
22:14:23Arcorann quits [Remote host closed the connection]
22:15:30Arcorann (Arcorann) joins
22:15:52Arcorann quits [Remote host closed the connection]
22:17:00Arcorann (Arcorann) joins
22:17:23Arcorann quits [Remote host closed the connection]
22:18:09BlueMaxima joins
22:18:30Arcorann (Arcorann) joins
22:18:52Arcorann quits [Remote host closed the connection]
22:20:00Arcorann (Arcorann) joins
22:20:10<@JAA>Software Heritage has it as well: https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/noDRM/DeDRM_tools
22:20:23Arcorann quits [Remote host closed the connection]
22:20:43<@JAA>Ah wait, the repo is up anyway.
22:24:46<@JAA>The forks are still dead though.
23:04:22<@rewby>Jake: I'm hyped for the monkeyball run on agdq in a few minutes
23:04:30<@rewby>Monkeyball is always good
23:04:38<Jake>yesss
23:11:45<Jake>haha you can be a .... sega dreamcast? wow
23:12:24<@rewby>Hilarious
23:12:48<@rewby>Also the handheld master system thing whose name I forget

HomeSearchPrevious dayNext day