HomeSearchPrevious dayNext day

00:01:18jtagcat quits [Quit: Bye!]
00:02:00jtagcat (jtagcat) joins
00:38:39kiryu (kiryu) joins
00:57:29Arcorann (Arcorann) joins
01:38:27<@JAA>So turns out that some files aren't downloadable from FOIAonline:
01:38:33<@JAA>> Message The request was rejected because the URL contained a potentially malicious String "%3B"
01:38:40<@JAA>The filename contains a semicolon...
01:39:07<@JAA>No, replacing %3B with a literal semicolon doesn't work either, nor does double-encoding it.
01:40:18<@JAA>This appears to be the problem for many of the items that failed.
01:42:18icedice quits [Client Quit]
01:56:57<project10>rogue WAF strikes again
01:57:58<fireonlive>………….
01:58:04<fireonlive>fucking christ
02:02:59<appledash>lmao
02:27:36<@JAA>Oh, it's even stupider than I thought.
02:27:54<@JAA>So those document download links contain the filename, presumably for the Content-Disposition header.
02:28:06<@JAA>But you can just change it (as long as you still send the right Referer etc.).
02:29:03<@JAA>Guess I'll add detection for this org.springframework.security.web.firewall.StrictHttpFirewall.rejectedBlacklistedUrls thing and just replace the filename with something generic for those.
02:29:57<@JAA>The bruteforcing still didn't happen, by the way. Machine was busy with the known existing requests and uploading.
02:41:36<@JAA>Since I need to retry these items yet again, not sure if it will happen I'm afraid.
02:42:38Exorcism quits [Remote host closed the connection]
02:43:03Exorcism (exorcism) joins
02:43:50<@JAA>I did briefly sample some of the 'missing' IDs, and those all seemed to fail (i.e. 403 on the API), so hopefully there isn't much missing.
02:55:50Exorcism quits [Remote host closed the connection]
02:56:14Exorcism (exorcism) joins
02:57:17nepeat quits [Max SendQ exceeded]
02:58:16BlueMaxima joins
02:58:41nepeat (nepeat) joins
03:03:19Exorcism quits [Remote host closed the connection]
03:03:20<@JAA>I'll also skip over files that can't be downloaded. Until now, it would fail the entire item (= request).
03:03:46Exorcism (exorcism) joins
03:12:19<@JAA>I also bumped the timeout since some API requests and some large file downloads ran into that.
03:12:26<@JAA>This is probably as good as it'll get.
03:25:06<@JAA>Oops, I played with a bruteforce sample and got myself banned it seems. Let's hope it doesn't last long.
03:26:57<@JAA>Actually, can't connect from elsewhere either. Uh oh...
03:28:23<@JAA>Ok, it's back.
03:30:22<@JAA>Needless to say that bruteforcing won't work if that's what happens.
03:30:29<@JAA>But also, not a single hit on that sample.
04:11:16kdqep__ joins
04:12:24DogsRNice quits [Read error: Connection reset by peer]
04:15:18parfait_ quits [Ping timeout: 265 seconds]
04:23:25Exorcism quits [Remote host closed the connection]
04:23:51Exorcism (exorcism) joins
04:46:50dumbgoy_ quits [Ping timeout: 252 seconds]
04:57:00etnguyen03 quits [Client Quit]
05:02:31mindstrut quits [Read error: Connection reset by peer]
05:02:32Unholy236131661808515997 (Unholy2361) joins
05:02:47mindstrut joins
05:06:05Unholy23613166180851599 quits [Ping timeout: 252 seconds]
05:06:05Unholy236131661808515997 is now known as Unholy23613166180851599
05:10:07Wohlstand quits [Client Quit]
05:24:30wyatt8750 quits [Quit: ZNC got killed or something else has gone wrong, probably.]
05:24:54wyatt8740 joins
05:33:13BearFortress_ joins
05:36:01BearFortress quits [Ping timeout: 265 seconds]
05:38:32lizardb0y joins
05:38:39lizardb0y quits [Client Quit]
05:43:53lizardb0y joins
06:19:36sec^nd quits [Ping timeout: 245 seconds]
06:19:40second (second) joins
06:20:07second is now known as sec^nd
06:55:51BigBrain quits [Ping timeout: 245 seconds]
07:05:02Unholy23613166180851599 quits [Remote host closed the connection]
07:06:36Unholy236131661808515997 (Unholy2361) joins
08:14:12treora quits [Remote host closed the connection]
08:14:16treora joins
08:18:25Peroniko quits [Ping timeout: 265 seconds]
09:06:55danwellby joins
09:28:37Exorcism quits [Remote host closed the connection]
09:29:02Exorcism (exorcism) joins
09:41:49BlueMaxima quits [Read error: Connection reset by peer]
09:50:01sec^nd quits [Remote host closed the connection]
09:50:29sec^nd (second) joins
10:00:01railen63 quits [Remote host closed the connection]
10:00:18railen63 joins
10:13:02cultpony quits [Quit: ZNC - https://znc.in]
10:13:54cultpony (cultpony) joins
10:36:29Peroniko (Peroniko) joins
11:12:56imer quits [Ping timeout: 252 seconds]
11:13:23imer (imer) joins
12:11:51parfait_ joins
12:15:44kdqep__ quits [Ping timeout: 265 seconds]
13:00:14magmaus3 quits [Quit: :3]
13:02:10magmaus3 (magmaus3) joins
13:57:41Exorcism8 (exorcism) joins
13:58:18Exorcism quits [Read error: Connection reset by peer]
13:58:18Exorcism8 is now known as Exorcism
13:59:53toss (toss) joins
14:34:27etnguyen03 (etnguyen03) joins
14:36:48<pabs>https://www.nytimes.com/2023/09/29/business/media/letterboxd-new-owner.html
14:51:50Arcorann quits [Ping timeout: 252 seconds]
14:57:02<Peroniko>While I love Letterboxd, it is ultimately a reskin of TMDB data with a few social media features. I would love it if they implemented better guidelines about what is a review and what is a comment. There is too many one liners on any decently popular film
14:57:58<Peroniko>Still better than Goodreads though. Amazon ruined that site
15:04:54AmAnd0A quits [Ping timeout: 265 seconds]
15:05:05AmAnd0A joins
15:12:55dumbgoy_ joins
15:14:20benjins joins
15:17:41benjinsm quits [Ping timeout: 252 seconds]
15:22:31railen63 quits [Remote host closed the connection]
15:22:48railen63 joins
15:29:36endrift quits [Quit: +++CARRIER LOST+++]
15:30:05endrift joins
15:30:19endrift quits [Remote host closed the connection]
15:31:08endrift joins
15:54:59AmAnd0A quits [Read error: Connection reset by peer]
15:56:11AmAnd0A joins
16:07:11etnguyen03 quits [Ping timeout: 252 seconds]
16:13:54Unholy236131661808515997 quits [Client Quit]
16:13:58Unholy2361316618085159973 (Unholy2361) joins
16:18:15DogsRNice joins
16:33:50icedice (icedice) joins
16:39:32h2ibot quits [Read error: Connection reset by peer]
16:41:50flashfire42 quits [Ping timeout: 252 seconds]
16:41:50Ryz263 quits [Ping timeout: 252 seconds]
16:41:50s-crypt2 quits [Ping timeout: 252 seconds]
16:42:03kiska quits [Ping timeout: 265 seconds]
16:42:05h2ibot (h2ibot) joins
16:44:02AnotherIki quits [Ping timeout: 252 seconds]
16:55:02etnguyen03 (etnguyen03) joins
16:56:31qwertyasdfuiopghjkl quits [Remote host closed the connection]
17:10:53qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
17:29:54etnguyen03 quits [Ping timeout: 265 seconds]
17:42:43treora quits [Remote host closed the connection]
17:42:44treora joins
17:43:57etnguyen03 (etnguyen03) joins
17:50:35FavoritoHJS (FavoritoHJS) joins
17:51:44<FavoritoHJS>fyi it appears discord is changing how cdn links work, this is likely to break the other half of the uploads that didn't get lit ablaze by dropbox dropping the box or imgur failing to image...
17:52:27<FavoritoHJS>considering how much important knowledge is in non-crawled, almost certainly non-backed-up guilds there, i wonder if a proper project would be worthwhile...
17:57:55<@Sanqui>#discard
18:08:35Peroniko quits [Client Quit]
18:08:54Peroniko (Peroniko) joins
18:08:54Peroniko quits [Max SendQ exceeded]
18:09:20Peroniko (Peroniko) joins
18:09:33Peroniko quits [Remote host closed the connection]
18:09:52Peroniko (Peroniko) joins
18:11:46Peroniko quits [Remote host closed the connection]
18:13:53Peroniko (Peroniko) joins
18:13:53Peroniko quits [Max SendQ exceeded]
18:14:21Peroniko (Peroniko) joins
18:14:25Peroniko quits [Remote host closed the connection]
18:15:00Peroniko (Peroniko) joins
18:16:26etnguyen03 quits [Ping timeout: 252 seconds]
18:17:22Peroniko quits [Remote host closed the connection]
18:17:40etnguyen03 (etnguyen03) joins
18:20:07icedice quits [Client Quit]
18:29:04<@JAA>Turns out that there are entries on FOIAonline which can't be found by the search (at least with how I used it), but they aren't in my bruteforce list either. Two examples: https://foiaonline.gov/foiaonline/action/public/submissionDetails?trackingNumber=DOI-FWS-2023-003849&type=Request https://foiaonline.gov/foiaonline/action/public/submissionDetails?trackingNumber=DOJ-2020-000763&type=Request
18:29:50<@JAA>Probably not much that can be done about that. :-/
18:29:51thunder_steak joins
18:31:01<@JAA>They don't even show up when you specifically search for those tracking numbers.
18:49:02<FavoritoHJS>about the discord cdn shenanigans... it appears it means all cdn links to discord will break outside of discord...
18:49:31<FavoritoHJS>and i have seen many MANY cases of a discord cdn link being used for a download that ought to be persistent...
18:50:51<@JAA>And this is still not the channel to discuss it.
18:51:26<imer>-> #discard
18:54:45Exorcism quits [Remote host closed the connection]
18:55:09Exorcism (exorcism) joins
18:55:40Exorcism quits [Remote host closed the connection]
18:56:16Exorcism (exorcism) joins
19:00:09<h2ibot>JustAnotherArchivist edited FOIAonline (+2477, Document site quirks): https://wiki.archiveteam.org/?diff=50912&oldid=50898
19:01:03thunder_steak quits [Remote host closed the connection]
19:03:11etnguyen03 quits [Ping timeout: 252 seconds]
19:04:40etnguyen03 (etnguyen03) joins
19:09:55itachi1706 quits [Quit: Bye :P]
19:13:43itachi1706 (itachi1706) joins
19:25:59mindstrut quits [Read error: Connection reset by peer]
19:26:16mindstrut joins
19:30:55FavoritoHJS quits [Client Quit]
19:38:07Exorcism1 (exorcism) joins
19:39:25Exorcism quits [Read error: Connection reset by peer]
19:39:26Exorcism1 is now known as Exorcism
20:07:26Peroniko (Peroniko) joins
20:07:26Peroniko quits [Max SendQ exceeded]
20:07:54Peroniko (Peroniko) joins
20:10:32Wohlstand (Wohlstand) joins
20:11:41Peroniko quits [Client Quit]
20:12:00Peroniko (Peroniko) joins
20:12:00Peroniko quits [Max SendQ exceeded]
20:12:27Peroniko (Peroniko) joins
20:13:02Wohlstand quits [Client Quit]
20:13:18Wohlstand (Wohlstand) joins
20:14:27Peroniko quits [Client Quit]
20:15:10Peroniko (Peroniko) joins
20:15:10Peroniko quits [Max SendQ exceeded]
20:15:37Peroniko (Peroniko) joins
20:17:01Peroniko quits [Remote host closed the connection]
20:17:31Peroniko (Peroniko) joins
20:17:31Peroniko quits [Max SendQ exceeded]
20:17:57Peroniko (Peroniko) joins
20:40:20etnguyen03 quits [Ping timeout: 265 seconds]
20:47:46<@JAA>Something broke at FOIAonline about 15 minutes ago. Getting a lot more errors now.
20:48:33icedice (icedice) joins
21:04:53BlueMaxima joins
21:06:11etnguyen03 (etnguyen03) joins
21:51:31thunder_steak joins
21:53:02thunder_steak quits [Remote host closed the connection]
21:53:08girst quits [Ping timeout: 252 seconds]
21:55:26Peroniko quits [Client Quit]
22:03:36<@JAA>FOIAonline is offline now, happened sometime in the past hour or so.
22:05:00dumbgoy__ joins
22:07:05toss_ (toss) joins
22:08:32dumbgoy_ quits [Ping timeout: 252 seconds]
22:09:58<@JAA>I was hoping it'd last a bit longer since they said that today would be the last day of access and it'd be inaccessible tomorrow, but oh well.
22:10:11<@JAA>I got the vast majority of discoverable content, I think.
22:11:17toss quits [Ping timeout: 252 seconds]
22:15:35<fireonlive>🪦 rip
22:15:49<fireonlive>thanks JAA
22:15:54<h2ibot>JustAnotherArchivist edited FOIAonline (-44, It's dead, Jim.): https://wiki.archiveteam.org/?diff=50913&oldid=50912
22:21:59<thuban>good work, JAA!
22:31:00<fireonlive>for sure
22:40:53toss_ quits [Client Quit]
22:42:31Exorcism6 (exorcism) joins
22:44:17Exorcism quits [Read error: Connection reset by peer]
22:44:18Exorcism6 is now known as Exorcism
22:46:00shinji257 quits [Ping timeout: 265 seconds]

HomeSearchPrevious dayNext day