HomeSearchPrevious dayNext day

00:06:52Czechball quits [Ping timeout: 240 seconds]
00:24:14tzt quits [Remote host closed the connection]
00:24:32tzt (tzt) joins
01:00:25<h2ibot>JAABot edited CurrentWarriorProject (-4): https://wiki.archiveteam.org/?diff=49120&oldid=49108
01:17:39jacobk quits [Ping timeout: 268 seconds]
01:24:04qwertyasdfuiopghjkl joins
01:28:44BlueMaxima joins
01:33:31Czechball joins
01:38:16sonick quits [Client Quit]
01:42:11adamus1red quits [Quit: SigTerm]
01:44:46adamus1red (adamus1red) joins
02:04:31march_happy quits [Ping timeout: 268 seconds]
02:05:13march_happy (march_happy) joins
02:16:51michaelblob quits [Ping timeout: 268 seconds]
02:32:09sec^nd quits [Remote host closed the connection]
02:38:36sec^nd (second) joins
03:01:35BlueMaxima_ joins
03:01:37BlueMaxima quits [Remote host closed the connection]
03:02:11Czechball quits [Client Quit]
03:02:30Czechball joins
03:07:39eroc1990 quits [Client Quit]
03:08:01eroc1990 (eroc1990) joins
03:11:59atphoenix__ is now known as atphoenix
03:17:21jacobk joins
03:39:50atphoenix quits [Remote host closed the connection]
03:40:28atphoenix (atphoenix) joins
03:47:30march_happy quits [Ping timeout: 268 seconds]
03:47:47march_happy (march_happy) joins
04:15:00sec^nd quits [Ping timeout: 255 seconds]
04:17:06march_happy quits [Ping timeout: 268 seconds]
04:17:16march_happy (march_happy) joins
04:26:49sec^nd (second) joins
04:30:04datechnoman quits [Ping timeout: 240 seconds]
04:44:52fenugrec_ quits [Ping timeout: 240 seconds]
04:46:42march_happy quits [Ping timeout: 268 seconds]
04:47:25march_happy (march_happy) joins
04:48:21datechnoman (datechnoman) joins
04:52:28march_happy quits [Ping timeout: 265 seconds]
04:53:15march_happy (march_happy) joins
05:09:07BlueMaxima_ quits [Read error: Connection reset by peer]
05:33:27michaelblob (michaelblob) joins
06:19:24fenugrec_ joins
06:20:11Czechball quits [Client Quit]
06:20:35Czechball joins
06:53:32<@OrIdow6^2>I love reading backlog
06:53:38<@OrIdow6^2>This must be how arkiver feels
07:21:07<@rewby>Hah
07:23:00<@rewby>arkiver: What JA.A said, we have what we think is a complete list. Each blog is a site. The sites are all plain html. Usually not that big. Perfectly up AB's alley I think. We just wanna test a bigger one and see what happens.
07:23:44<@rewby>JAA: Should I go pick out a big blog and run it through AB?
07:31:50<@HCross>rewby: JAA is it worth slicing the list up
07:31:53<@HCross>and queueing a few AB jobs
07:41:53<@rewby>The plan is queueh2ibot
07:41:58<@rewby>Each site would be a job
08:04:52atphoenix quits [Remote host closed the connection]
08:07:03Czechball quits [Client Quit]
08:07:04atphoenix (atphoenix) joins
08:07:15Czechball joins
08:13:52<pabs>https://www.brickfanatics.com/lego-discontinuing-mindstorms-end-of-2022/
08:14:30Czechball quits [Ping timeout: 265 seconds]
08:15:34<pabs>ESR's site/blog seem a bit abandoned: http://www.catb.org/esr/ http://esr.ibiblio.org/
08:16:11Arcorann (Arcorann) joins
08:54:52le0n_ (le0n) joins
08:57:40le0n quits [Ping timeout: 240 seconds]
09:16:21march_happy quits [Remote host closed the connection]
09:18:08march_happy (march_happy) joins
09:18:45sec^nd quits [Remote host closed the connection]
09:21:47sec^nd (second) joins
09:29:41heisei joins
09:37:36heisei quits [Remote host closed the connection]
09:40:31<SketchCo1>I mean, good?
09:42:45SketchCo1 is now known as SketchCow
09:55:34LeanTo quits [Client Quit]
10:01:21sec^nd quits [Remote host closed the connection]
10:04:29sec^nd (second) joins
12:12:15katocala quits [Remote host closed the connection]
12:27:59sonick (sonick) joins
12:35:59programmerq quits [Ping timeout: 265 seconds]
12:36:12programmerq (programmerq) joins
12:44:37jacobk quits [Ping timeout: 268 seconds]
13:11:47dm4v_ joins
13:11:47dm4v quits [Client Quit]
13:11:47dm4v_ is now known as dm4v
13:14:28Arcorann quits [Ping timeout: 240 seconds]
14:17:45qwertyasdfuiopghjkl39 joins
14:20:23qwertyasdfuiopghjkl quits [Ping timeout: 265 seconds]
14:20:25qwertyasdfuiopghjkl39 is now known as qwertyasdfuiopghjkl
14:22:58Pingerfowder quits [Remote host closed the connection]
14:23:13Pingerfowder (Pingerfowder) joins
14:35:52sdss joins
14:36:22sdss quits [Remote host closed the connection]
15:13:42<IDK>JAA: it may be possible for Warrior as long as 403s get returned
15:15:05<IDK>Not sure if thats how it works
15:17:13<IDK>sorry got confused with the 2 topics
15:34:54jacobk joins
15:50:51eroc1990 quits [Ping timeout: 268 seconds]
16:05:04Hackerpcs quits [Client Quit]
16:07:15Hackerpcs (Hackerpcs) joins
16:10:13<@JAA>So Tweakers has silly rate limits. A three-URL curl already triggers them. They do return 429s at least.
16:15:50eroc1990 (eroc1990) joins
16:29:27<@JAA>Seems to be one request per ten seconds.
16:30:03<Ruk8>Has anyone put in queue the adobe thing I posted yesterday? If no, I can do an updated version with all the links I found until now.
16:34:25<@JAA>Ruk8: Don't think so, so yeah, an update list would be better I guess.
16:35:03<@JAA>Is this the same MM_TRIALS cookie thing from a couple months ago?
16:42:06<Ruk8>Yeah, I just found more links lol
16:45:19<@JAA>Even 9-12s delay on Tweakers still occasionally gets 429s. This is going to be a pain...
16:46:00<joepie91|m>Xesxen: ^ this is what I meant with "I wouldn't be so sure that we can easily make the deadline" :p
16:46:25<@JAA>arkiver: If you reach them and they're willing to work with us, an exception on the rate limiting for the AB UA maybe?
16:47:28<@JAA>Else we'll need a bunch of IPs I guess.
16:47:31<@JAA>Cc rewby HCross ^
16:48:17<@HCross>rewby also speaks the Dutch language if that's needed to negotiate with them
16:48:19<@JAA>The blog I'm running as a test does account for over 1% of all blog posts, so maybe it will still be fine even with these limits.
16:49:15<@JAA>But the pipeline handling stuff would be annoying. My scripts for queueh2ibot don't currently support that.
16:50:59<@rewby>Ah. I can do a ton of ips if need be.
16:51:07<@rewby>Actually, does it do V6...
16:51:23<@rewby>Oh it does
16:51:57<@rewby>JAA: Didn't you have an experiment for IPv6 ab pipelines?
16:51:59<Ruk8>Here's there's the updated archive: https://transfer.archivete.am
16:52:09<Ruk8>sorry, missclick lol
16:52:28<Ruk8>Here it is: https://transfer.archivete.am/Vr2t/archiveteam_adobe_2022-10-27.tar.gz
16:52:35<@JAA>rewby: I played around with it, yeah.
16:52:57<@rewby>That an idea to try and do?
16:54:07<@rewby>If we throw a /48 at it and just randomise the ips it should be fine as long as we don't go abusively fast
16:54:20<@JAA>Yeah, 'just' :-)
16:54:32<@rewby>Do I need to write some ldpreload magic?
16:54:39<@JAA>Will play with it later.
16:55:08<@rewby>Because I totally can do an ldpreload hack
16:55:08<@JAA>I wrote the necessary magic a while ago.
16:56:33<@rewby>Ah
16:56:33<@rewby>Let me know if you need anything
16:56:33<@JAA>It's not as simple as an LDPRELOAD hack, by the way. You also need to patch wpull.
16:56:33march_happy quits [Ping timeout: 296 seconds]
17:06:02<@JAA>On another note, confirmation that all reactions are rendered on a single page: https://kiswum.tweakblogs.net/blog/8392/asus-transformer-book-(tx300) has 300 of them.
17:39:24qwertyasdfuiopghjkl quits [Remote host closed the connection]
17:59:44jacobk quits [Ping timeout: 268 seconds]
18:16:17jacobk joins
18:43:20<@arkiver>JAA: no reply yet - but will propose that if they reply and want to help
18:43:49<@arkiver>JAA: any idea on what conditions they start limiting?
18:46:13<@JAA>arkiver: Seems to be purely based on request rate. The limiting affects pretty much all of tweakers.net and tweakblogs.net (including subdomains) as far as I could see.
18:46:25<@arkiver>alright
18:46:36<@arkiver>well we have time, will let you know asap when i have a response
18:46:44<@arkiver>going to send another message next week if nothing comes back
18:46:49<@JAA>Ack
18:50:19<Ryz>Any updates on archiving Crunchyroll legacy content?
18:50:51<Ryz>Otherwise I may have to start jerryrigging a text file and shove it into ArchiveBot with --no-offsite-links and a bunch of ignores
18:51:05<Ryz>The power of "!a <"
19:00:09sec^nd quits [Ping timeout: 255 seconds]
19:01:18<Ryz>Well crap, there's even forums in different languages...
19:01:42sec^nd (second) joins
19:18:26<Ryz>Archiving via ArchiveBot is not going hot right now S:
19:20:04<@JAA>They use Buttflare.
19:20:17<Ryz>Doom-i-nation imminent S:
19:27:04tech_exorcist (tech_exorcist) joins
19:45:44Minkafighter7 quits [Quit: The Lounge - https://thelounge.chat]
19:46:19Minkafighter7 joins
19:47:49Minkafighter7 quits [Client Quit]
19:48:34Minkafighter7 joins
19:52:34leo60228- (leo60228) joins
19:53:40leo60228 quits [Ping timeout: 240 seconds]
20:02:28leo60228- quits [Ping timeout: 240 seconds]
20:32:51HackMii quits [Ping timeout: 255 seconds]
20:35:38HackMii (hacktheplanet) joins
20:56:45tech_exorcist quits [Client Quit]
20:59:01BlueMaxima joins
21:23:13<IDK>is there a good way to archive tumblr images, tried save page now and non of the images actually load, there is a infinite loop of HTML stuff
21:27:40tzt quits [Ping timeout: 240 seconds]
21:28:05BlueMaxima quits [Read error: Connection reset by peer]
21:29:43JackThompson5 joins
21:30:52JackThompson quits [Ping timeout: 240 seconds]
21:30:53JackThompson5 is now known as JackThompson
21:31:00tzt (tzt) joins
21:50:18march_happy (march_happy) joins
21:54:01<thuban>IDK: archivebot with --user-agent-alias=curl should work
22:35:52katocala joins
22:56:51mutantm0nkey quits [Ping timeout: 255 seconds]
22:58:24HackMii_ (hacktheplanet) joins
22:59:06HackMii quits [Ping timeout: 255 seconds]
23:01:53mutantm0nkey (mutantmonkey) joins
23:05:31Arcorann (Arcorann) joins
23:20:35eroc1990 quits [Client Quit]
23:20:35JackThompson quits [Client Quit]
23:20:35Pingerfowder quits [Client Quit]
23:20:35adamus1red quits [Client Quit]
23:20:35dm4v quits [Client Quit]
23:20:36eroc19902 (eroc1990) joins
23:20:36JackThompson3 joins
23:20:36Pingerfo- (Pingerfowder) joins
23:20:36JackThompson3 is now known as JackThompson
23:20:42dm4v joins
23:20:49adamus1red (adamus1red) joins
23:25:30mutantm0nkey quits [Remote host closed the connection]
23:27:21mutantm0nkey (mutantmonkey) joins

HomeSearchPrevious dayNext day