HomeSearchPrevious dayNext day

00:09:05etnguyen03 quits [Ping timeout: 252 seconds]
00:20:16benjinsm joins
00:23:23benjins quits [Ping timeout: 252 seconds]
00:35:25<HP_Archivist>Hey JAA: Was going through some old files on my drive and saw an Existential Comic I had saved. Got me thinking if it was crawled, and sure enough it was. By you in 2020. Should the site be crawled recursively? I don't know how many there are to-date or any new ones since then
00:35:58<HP_Archivist>This item from April of this year has 1022 https://archive.org/details/existentialcomics
00:37:22<@JAA>Sounds like me alright.
00:42:43<@JAA>Oh yeah, you requested it, actually. Heh
00:42:58<@JAA>But archiving is off-offtopic. :-P
00:44:19<@JAA>I'll rerun it when there's space again.
00:46:10etnguyen03 (etnguyen03) joins
01:05:58<HP_Archivist>JAA: Ha, I guess I'm ahead of myself. Yeah, no worries on a recursive run rn.
01:17:50etnguyen03 quits [Ping timeout: 252 seconds]
01:21:52Shjosan quits [Ping timeout: 265 seconds]
01:22:20Shjosan (Shjosan) joins
01:31:58etnguyen03 (etnguyen03) joins
01:57:32Arcorann (Arcorann) joins
02:51:29icedice quits [Client Quit]
03:05:38etnguyen03 quits [Ping timeout: 252 seconds]
03:53:43etnguyen03 (etnguyen03) joins
03:57:14<nulldata>https://www.streamtvinsider.com/video/dish-sues-eight-companies-over-streaming-tech-patents
03:57:25<nulldata>Fuck software patents
04:07:24<fireonlive>agreed.
05:06:09etnguyen03 quits [Client Quit]
05:18:42DogsRNice quits [Read error: Connection reset by peer]
05:24:38<fireonlive>https://x.com/nsa_csdirector/status/1715811169699557604?s=12
05:24:39<eggdrop>nitter: https://nitter.net/nsa_csdirector/status/1715811169699557604
05:24:42<fireonlive>!
05:24:48<fireonlive>lmfao
05:42:54hitgrr8 joins
06:09:20CraftByte quits [Ping timeout: 252 seconds]
06:26:43<project10>daaaamn
07:18:39BlueMaxima quits [Client Quit]
07:52:36Naruyoko quits [Remote host closed the connection]
07:52:58Naruyoko joins
07:56:02Arcorann quits [Ping timeout: 252 seconds]
07:58:51Arcorann (Arcorann) joins
08:19:06<nulldata>https://news.ycombinator.com/item?id=37971038
08:19:09<nulldata>oof
08:30:48gfhh quits [Client Quit]
09:01:49<imer>oh come on. > built between 2015 and 2020.
09:11:52<flashfire42>https://server8.kiska.pw/uploads/bfd90535dd3825ed/image.png JAA what the fuck is wrong with your countrymen. you try and click on their youtube page and it takes you to a fucking vimeo thing
09:38:02Matthww11 quits [Quit: The Lounge - https://thelounge.chat]
10:04:08Matthww119 joins
11:22:45Exorcism|tor (exorcism) joins
11:44:19simon816 quits [Read error: Connection reset by peer]
11:44:50Arcorann quits [Ping timeout: 252 seconds]
11:46:46Arcorann (Arcorann) joins
11:52:50simon816 (simon816) joins
12:24:59Matthww119 quits [Ping timeout: 252 seconds]
12:33:37Matthww119 joins
12:42:35Arcorann quits [Ping timeout: 252 seconds]
13:41:40etnguyen03 (etnguyen03) joins
13:43:30<@JAA>flashfire42: ¯\_(ツ)_/¯
13:46:56<@JAA>The same people enter things like http://www.https://ManuelaWeicheltPicardZG into the Facebook URL field, so yeah...
14:00:16lunik173 quits [Quit: :x]
14:00:48lunik173 joins
14:55:48tzt quits [Ping timeout: 265 seconds]
14:56:55tzt (tzt) joins
15:24:18Hackerpcs quits [Quit: Hackerpcs]
15:26:17Hackerpcs (Hackerpcs) joins
15:57:20ymgve_ joins
16:01:03zhongfu_ quits [Ping timeout: 265 seconds]
16:46:42IDK quits [Quit: Connection closed for inactivity]
17:07:41Chris5010 quits [Ping timeout: 252 seconds]
17:13:11etnguyen03 quits [Ping timeout: 252 seconds]
17:27:29fuzzy8021 quits [Ping timeout: 252 seconds]
17:46:33etnguyen03 (etnguyen03) joins
18:07:33jess quits []
18:22:04<fireonlive>lmfao
18:25:05etnguyen03 quits [Ping timeout: 265 seconds]
18:58:26benjins joins
19:02:05benjinsm quits [Ping timeout: 252 seconds]
19:08:10jacksonchen666 is now known as RJHacker51027
19:08:14jacksonchen666 (jacksonchen666) joins
19:10:35benjinsm joins
19:10:51RJHacker51027 quits [Ping timeout: 245 seconds]
19:13:54benjins quits [Ping timeout: 265 seconds]
19:47:37nicolas17 joins
19:49:39<nicolas17>https://itunes.apple.com/us/lookup?id=1069511855 I was trying to scrape these and I started getting 403 errors... can't figure out the rate limit to avoid it
20:04:07etnguyen03 (etnguyen03) joins
20:15:14<nicolas17>I probably need a thousand IPs anyway
20:18:21<that_lurker>what delay did you use?
20:22:29<fireonlive>https://x.com/elonmusk/status/1716102436123783175?s=12
20:22:29<eggdrop>nitter: https://nitter.net/elonmusk/status/1716102436123783175
20:22:48<fireonlive>good news everyone, wikimedia’s funding problems to be solved
20:24:34<@JAA>Wikimedia has funding problems?
20:27:47<that_lurker>oh are they doing the shady we need funding thing again
20:29:23<that_lurker>https://unherd.com/thepost/the-next-time-wikipedia-asks-for-a-donation-ignore-it/
20:37:13<fireonlive>a personal appeal from dimmy wales
20:42:41<TheTechRobo>Also https://en.wikipedia.org/wiki/WP:CANCER
20:44:33<fireonlive>that_lurker: interesring so far
20:44:47<fireonlive>i wonder if there’s a ublock origin filter for their pleas lol
20:47:57fuzzy8021 (fuzzy8021) joins
20:50:19driib quits [Quit: The Lounge - https://thelounge.chat]
20:52:05driib (driib) joins
21:06:56Barto quits [Ping timeout: 252 seconds]
21:09:00BlueMaxima joins
21:16:08driib quits [Client Quit]
21:17:00dddddddd quits [Remote host closed the connection]
21:18:36Exorcism|tor quits [Client Quit]
21:28:20jacksonchen666 quits [Client Quit]
21:35:19driib (driib) joins
22:16:47etnguyen03 quits [Ping timeout: 252 seconds]
22:22:11<nicolas17>that_lurker: with no delay (as fast as my home network allows but with *one* thread), I did 10k IDs in 40 minutes and didn't get blocked, but if I do "concurrency 2" I get blocked
22:22:56<@JAA>They have IPv6...
22:23:29<nicolas17>also once I start getting 403s, it's sometimes 50% errors, sometimes 90% errors, and it's very inconsistent how much delay/backoff helps
22:24:22<nicolas17>JAA: if they're dumb enough to apply rate limits per /128 that would be golden
22:24:36<@JAA>:-)
22:25:22<nicolas17>but well, it's still one billion IDs
22:26:17Barto (Barto) joins
22:26:26<@JAA>Even if it's per /64, a /48 isn't hard to get.
22:41:43<fireonlive>tunnelbroker 😎
22:42:04Ruthalas59 quits [Client Quit]
22:45:15<nicolas17>warrior project (?)
22:47:30etnguyen03 (etnguyen03) joins
22:52:57myself quits [Quit: The Lounge - https://thelounge.chat]
22:56:32<nicolas17>yep with zero delay between requests and just my own network latency slowing things down, I seem to consistently get 10k/40min
22:58:29<nicolas17>I have a script that puts 10k IDs into a queue and starts multiple worker threads to work through them (currently set to 1), and if something returns 403 it's re-added at the end of the queue
22:59:25<nicolas17>and I tried different strategies for delays after a 403 and nothing seemed to work well enough
22:59:49<nicolas17>setting concurrency 1 avoids the 403s entirely, but well, it's slow
23:04:03<nicolas17>my VPS has lower latency and thus gets banned even with concurrency 1 without delays, I'll try 100ms delay per request there...
23:06:13myself (myself) joins
23:08:44<@JAA>So you want about one request every 240 ms.
23:09:28<nicolas17>the 100ms delay seems to be working so far
23:10:22<nicolas17>but request latency on the VPS is still not zero, so that's somewhat more than 100ms in practice
23:12:01<nicolas17>I was mainly interested in info about iOS apps, but so far I got 1856 songs, 343 ebooks, 35 apps >.>
23:30:30<nicolas17>done, 10k in 26m... let's try 75ms! :P
23:42:09<fireonlive>*banned 5ever*

HomeSearchPrevious dayNext day