HomeSearchPrevious dayNext day

00:01:52Earendil (Cobalt17) joins
00:02:01Earendil quits [Read error: Connection reset by peer]
00:08:58Earendil (Cobalt17) joins
00:21:48wizards_ quits [Ping timeout: 265 seconds]
00:23:11wizards_ joins
00:29:00Earendil quits [Read error: Connection reset by peer]
00:32:12bonga quits [Read error: Connection reset by peer]
00:32:47bonga joins
00:36:51Earendil (Cobalt17) joins
00:37:54fuzzy8021 quits [Read error: Connection reset by peer]
00:38:21fuzzy8021 (fuzzy8021) joins
00:39:03Earendil quits [Read error: Connection reset by peer]
00:50:46Earendil (Cobalt17) joins
00:52:55Earendil quits [Read error: Connection reset by peer]
01:00:03dm4v quits [Client Quit]
01:00:47chrismeller (chrismeller) joins
01:01:00chrismeller quits [Remote host closed the connection]
01:02:07chrismeller (chrismeller) joins
01:02:30chrismeller quits [Remote host closed the connection]
01:03:22HP_Archivist quits [Ping timeout: 265 seconds]
01:03:37chrismeller (chrismeller) joins
01:03:42dm4v joins
01:03:44dm4v quits [Changing host]
01:03:44dm4v (dm4v) joins
01:04:00chrismeller quits [Remote host closed the connection]
01:05:07chrismeller (chrismeller) joins
01:05:30chrismeller quits [Remote host closed the connection]
01:05:52chrismeller (chrismeller) joins
01:09:56Earendil (Cobalt17) joins
01:19:42onetruth quits [Read error: Connection reset by peer]
01:27:05Earendil quits [Read error: Connection reset by peer]
01:31:10DogsRNice (Webuser299) joins
01:33:07Earendil (Cobalt17) joins
01:41:39shufflertoxin joins
01:47:22Earendil quits [Read error: Connection reset by peer]
01:48:13Earendil (Cobalt17) joins
02:02:52DogsRNice_ (Webuser299) joins
02:05:06DogsRNice quits [Ping timeout: 240 seconds]
02:09:57igloo22225 quits [Quit: Ping timeout (120 seconds)]
02:10:12igloo22225 (igloo22225) joins
02:12:21HP_Archivist (HP_Archivist) joins
02:12:38HP_Archivist quits [Remote host closed the connection]
02:13:50HP_Archivist (HP_Archivist) joins
02:14:08HP_Archivist quits [Remote host closed the connection]
02:15:20HP_Archivist (HP_Archivist) joins
02:15:38HP_Archivist quits [Remote host closed the connection]
02:16:50HP_Archivist (HP_Archivist) joins
02:17:08HP_Archivist quits [Remote host closed the connection]
02:17:16bonga quits [Remote host closed the connection]
02:17:37HP_Archivist (HP_Archivist) joins
02:32:56AlsoHP_Archivist joins
02:34:43HP_Archivist quits [Ping timeout: 265 seconds]
02:57:22systwi quits [Read error: Connection reset by peer]
02:57:34fuzzy8021 quits [Read error: Connection reset by peer]
02:57:58ave quits [Quit: Ping timeout (120 seconds)]
02:57:58lun4 quits [Quit: Ping timeout (120 seconds)]
02:58:39fuzzy8021 (fuzzy8021) joins
02:58:49systwi (systwi) joins
02:59:00lun4 (lun4) joins
02:59:04ave (ave) joins
02:59:06@dxrt quits [Quit: ZNC - http://znc.sourceforge.net]
02:59:24dxrt joins
02:59:27dxrt quits [Changing host]
02:59:27dxrt (dxrt) joins
02:59:27@ChanServ sets mode: +o dxrt
03:01:09Stilett0 is now known as Stiletto
03:14:30AlsoHP_Archivist quits [Read error: Connection reset by peer]
03:14:58AlsoHP_Archivist joins
03:20:26AlsoHP_Archivist quits [Ping timeout: 240 seconds]
03:22:18HP_Archivist (HP_Archivist) joins
03:34:54Earendil quits [Read error: Connection reset by peer]
03:41:59Earendil (Cobalt17) joins
04:11:29Earendil quits [Read error: Connection reset by peer]
04:13:08shufflertoxin quits [Remote host closed the connection]
04:20:59Earendil (Cobalt17) joins
04:23:02Earendil quits [Read error: Connection reset by peer]
04:24:15Earendil (Cobalt17) joins
04:44:31godane (godane) joins
04:56:02Earendil quits [Read error: Connection reset by peer]
05:01:52Earendil (Cobalt17) joins
05:16:17onetruth joins
05:58:14Iki joins
06:00:26Iki1 quits [Ping timeout: 240 seconds]
06:09:03Earendil quits [Read error: Connection reset by peer]
06:09:58mav6771 joins
06:12:36Earendil (Cobalt17) joins
06:14:46Earendil quits [Read error: Connection reset by peer]
06:24:33Iki1 joins
06:26:35BlueMaxima quits [Read error: Connection reset by peer]
06:27:26Iki quits [Ping timeout: 240 seconds]
06:28:57mav6771 leaves
06:30:11Earendil (Cobalt17) joins
06:30:47Earendil quits [Read error: Connection reset by peer]
06:34:28DogsRNice_ quits [Read error: Connection reset by peer]
06:36:47Earendil (Cobalt17) joins
07:15:03HP_Archivist quits [Ping timeout: 265 seconds]
07:49:31Earendil quits [Read error: Connection reset by peer]
07:55:05Earendil (Cobalt17) joins
08:37:01Mouks joins
09:09:51Mouks quits [Ping timeout: 244 seconds]
09:16:37Mouks joins
09:18:38fiftysix_k_modem joins
10:41:39Earendil quits [Client Quit]
11:06:53HackMii quits [Ping timeout: 252 seconds]
11:08:11HackMii (hacktheplanet) joins
11:22:35Mouks quits [Remote host closed the connection]
11:51:22qwertyasdfuiopghjkl joins
12:12:32ayy_lmao joins
12:15:45ayy_lmao quits [Remote host closed the connection]
13:15:36spirit joins
13:33:26Arcorann quits [Ping timeout: 240 seconds]
14:47:37Matthww quits [Ping timeout: 252 seconds]
14:48:46Matthww joins
15:42:17HP_Archivist (HP_Archivist) joins
15:42:33HP_Archivist quits [Remote host closed the connection]
15:43:46HP_Archivist (HP_Archivist) joins
15:44:03HP_Archivist quits [Remote host closed the connection]
15:45:16HP_Archivist (HP_Archivist) joins
15:45:33HP_Archivist quits [Remote host closed the connection]
15:46:46HP_Archivist (HP_Archivist) joins
15:46:49Matthww quits [Client Quit]
15:47:03HP_Archivist quits [Remote host closed the connection]
15:47:06Matthww joins
15:48:19HP_Archivist (HP_Archivist) joins
15:48:33HP_Archivist quits [Remote host closed the connection]
15:49:46HP_Archivist (HP_Archivist) joins
15:50:03HP_Archivist quits [Remote host closed the connection]
15:51:16HP_Archivist (HP_Archivist) joins
15:51:33HP_Archivist quits [Remote host closed the connection]
15:52:46HP_Archivist (HP_Archivist) joins
15:53:03HP_Archivist quits [Remote host closed the connection]
15:53:29HP_Archivist (HP_Archivist) joins
16:08:36vukky quits [Client Quit]
16:10:26vukky (Vukky) joins
16:44:25spirit quits [Client Quit]
16:52:30ysdxyx joins
18:01:52apache2_ joins
18:01:57apache2 quits [Remote host closed the connection]
18:04:42sec^nd quits [Remote host closed the connection]
18:18:28<IDK>Roblox servers are getting kinda wacky recently
18:18:35<IDK>just a notice
18:27:45Hifihedgehog joins
18:28:12<Hifihedgehog>Hey JAA! Just reaching out since some users are worried that you might not have enough time to pull down the site content. Are you guys in good shape to get the content all pulled before the 31st and the plug is pulled? Referring to TechnologyGuide sites, that is.
18:28:54<@JAA>We don't know yet. It appears that they have quite ridiculous rate limits. Their robots.txt advises a 1 second delay between requests, but even that's not sufficient.
18:33:42<Hifihedgehog>Oh boy. I wish I would have reached out sooner. I sure hope they don't close any sooner than the 31st given this.
18:36:17<@OrIdow6>Are these blocks or capacity issues?
18:39:51<@JAA>Blocks with 403s
18:41:07maltejur joins
18:41:37<Hifihedgehog>That probably explains why when I tried to scrape the site (not at all like you guys do; just used a simple Node.JS script for scraping Xenforo forums) I would get blocked out.
18:44:06<@OrIdow6>Would something distributed be warranted in this case?
18:46:14<@JAA>NotebookReview: 650k threads, 9.35M posts
18:46:27<@JAA>TabletPCReview: 57k threads, 521k posts
18:46:39<@JAA>Brighthand: 274k threads, 1.88M posts
18:46:47<@JAA>DigitalCameraReview: 13k threads, 58k posts
18:47:49<@JAA>And that doesn't yet include the websites, only the forums.
18:52:35<@JAA>So we're looking at well over 2M requests (when including deleted thread IDs and pagination). The acceptable delay between requests is more than 3.75 seconds; that was the last attempt with AB and got banned.
19:03:07<@OrIdow6>arkiver: How do you feel about doing the above as a backfeed recursive crawl? A la #Y
19:07:54<@arkiver>deadline is 31st?
19:07:59<@arkiver>OrIdow6: may be good
19:09:34cousteau joins
19:10:02cousteau quits [Remote host closed the connection]
19:10:15<@JAA>Yes, 'will close Jan 31, 2022'.
19:10:32<@arkiver>actually
19:10:52<@arkiver>OrIdow6: JAA: i can probably rewrite urls-grab today to get something like that going
19:11:11<@arkiver>then we can use it for those sites and others like it in the future
19:11:17<@arkiver>eventually that project can turn into #Y
19:11:26<@arkiver>will ping back later
19:11:59<@arkiver>urls-grab has most of the required code already, a lot can be filtered out, and then we have what we need
19:16:56<@OrIdow6>arkiver: Alright
19:18:21<@OrIdow6>If it turns out you don't have time I can probably rig up something inflexible
19:42:06Megame (Megame) joins
19:45:46HP_Archivist quits [Ping timeout: 240 seconds]
19:51:46<Megame>Federated Networks Association shutting down February 1st
19:51:59<Megame>More info - https://old.reddit.com/r/Archiveteam/comments/s05wk0/federated_networks_association_ry_dissolving/
19:59:04<@OrIdow6>Shutdowns on 1 Feb and 1 Mar
19:59:28<@OrIdow6>Megame: Have you added to Deathwatch? If not I can
20:00:02ysdxyx quits [Remote host closed the connection]
20:02:19<Megame>Please add it OrIdow6
20:16:38<h2ibot>OrIdow6 edited Deathwatch (+735, /* 2022 */ Add Feneas stuff): https://wiki.archiveteam.org/?diff=48202&oldid=48197
20:19:39<h2ibot>OrIdow6 edited Deathwatch (+59, "On or soon after"): https://wiki.archiveteam.org/?diff=48203&oldid=48202
21:06:21BlueMaxima joins
21:36:31lennier1 quits [Quit: Going offline, see ya! (www.adiirc.com)]
21:37:03lennier1 (lennier1) joins
21:38:17chrismeller quits [Ping timeout: 265 seconds]
21:39:20Megame quits [Client Quit]
21:46:45fiftysix_k_modem quits [Remote host closed the connection]
21:57:25traverseda-guest joins
22:04:03Arcorann (Arcorann) joins
22:05:24maltejur quits [Client Quit]
22:23:07<h2ibot>TheTechRobo edited Exploring by the Seat of your Pants (+111, I'm going to yt-dlp it): https://wiki.archiveteam.org/?diff=48204&oldid=47636
22:25:08<h2ibot>Arkiver uploaded File:Fandom-icon.png: https://wiki.archiveteam.org/?title=File%3AFandom-icon.png
22:25:09<h2ibot>Arkiver uploaded File:Fandom-logo.png: https://wiki.archiveteam.org/?title=File%3AFandom-logo.png
22:57:08Hifihedgehog quits [Remote host closed the connection]
23:27:18HP_Archivist (HP_Archivist) joins
23:27:35HP_Archivist quits [Remote host closed the connection]
23:28:47HP_Archivist (HP_Archivist) joins
23:29:05HP_Archivist quits [Remote host closed the connection]
23:30:18HP_Archivist (HP_Archivist) joins
23:30:35HP_Archivist quits [Remote host closed the connection]
23:31:47HP_Archivist (HP_Archivist) joins
23:32:05HP_Archivist quits [Remote host closed the connection]
23:33:19HP_Archivist (HP_Archivist) joins
23:33:35HP_Archivist quits [Remote host closed the connection]
23:34:47HP_Archivist (HP_Archivist) joins
23:35:05HP_Archivist quits [Remote host closed the connection]
23:36:18HP_Archivist (HP_Archivist) joins
23:36:35HP_Archivist quits [Remote host closed the connection]
23:37:47HP_Archivist (HP_Archivist) joins
23:38:05HP_Archivist quits [Remote host closed the connection]
23:39:18HP_Archivist (HP_Archivist) joins
23:39:35HP_Archivist quits [Remote host closed the connection]
23:40:47HP_Archivist (HP_Archivist) joins
23:41:05HP_Archivist quits [Remote host closed the connection]
23:42:17HP_Archivist (HP_Archivist) joins
23:42:35HP_Archivist quits [Remote host closed the connection]
23:43:48HP_Archivist (HP_Archivist) joins
23:44:05HP_Archivist quits [Remote host closed the connection]
23:44:30HP_Archivist (HP_Archivist) joins
23:58:56HP_Archivist quits [Ping timeout: 265 seconds]

HomeSearchPrevious dayNext day