HomeSearchPrevious dayNext day

00:00:35Arcorann (Arcorann) joins
00:06:14<fireonlive>-_-
00:13:27cascode quits [Read error: Connection reset by peer]
00:13:41cascode joins
00:27:31<@JAA>From another 1% sample, I got a size estimate of 2.6 TiB. Shows how uncertain that is due to the long tail of requests with many/large records.
00:27:33etnguyen03 (etnguyen03) joins
00:28:22<@JAA>s/another/a/ I guess, the previous one was 1‰, not 1%.
00:52:21<h2ibot>PaulWise edited IRC/Logs (+181, add more, done copyleft.org ones): https://wiki.archiveteam.org/?diff=50885&oldid=50312
00:56:22<h2ibot>PaulWise edited Mailman2 (+246, add more, done copyleft.org ones): https://wiki.archiveteam.org/?diff=50886&oldid=50882
00:59:11etnguyen03 quits [Ping timeout: 252 seconds]
01:10:54etnguyen03 (etnguyen03) joins
01:16:53mr_sarge quits [Read error: Connection reset by peer]
01:18:46jacksonchen666 quits [Ping timeout: 245 seconds]
01:27:32<h2ibot>PaulWise edited IRC/Logs (+48, AT IRC logs done :)): https://wiki.archiveteam.org/?diff=50887&oldid=50885
01:28:28jacksonchen666 (jacksonchen666) joins
01:30:10AmAnd0A quits [Read error: Connection reset by peer]
01:30:54AmAnd0A joins
01:54:46mr_sarge (sarge) joins
02:05:13etnguyen03 quits [Ping timeout: 265 seconds]
02:17:54etnguyen03 (etnguyen03) joins
02:20:57treora quits [Remote host closed the connection]
02:20:59treora joins
02:33:14etnguyen03 quits [Ping timeout: 252 seconds]
02:48:08<@JAA>My FOIAonline search bruteforcing got some duplicates. Wat.
02:49:23<@JAA>I listed the requests by receival date for each day from 2000-01-01, then going through the pagination of the results.
02:49:50<@JAA>These dupes are all over the place, oldest one is in 2016. Something is pretty broken there. lol
02:50:47<thuban>nice
02:51:32<@JAA>Yep, can confirm, CBP-2016-043045 on 2023-05-26 shows up on page 10 and 11.
02:51:37<@JAA>2016-05-26*
02:56:36icedice quits [Client Quit]
03:01:51<@JAA>And there's one duplicate that appears on two different dates. Are you fucking kidding me?
03:01:59<@JAA>DON-NAVY-2022-000829 appears on 2021-10-21 and 2022-03-01.
03:26:58etnguyen03 (etnguyen03) joins
04:02:14Wohlstand quits [Client Quit]
04:12:06Exorcism quits [Remote host closed the connection]
04:13:43DogsRNice quits [Read error: Connection reset by peer]
04:20:52Exorcism (exorcism) joins
04:25:43benjinsmi joins
04:26:39benjins2_ joins
04:26:50etnguyen03 quits [Ping timeout: 265 seconds]
04:28:33etnguyen03 (etnguyen03) joins
04:28:44benjinsm quits [Ping timeout: 252 seconds]
04:28:46benjins2 quits [Ping timeout: 265 seconds]
04:30:51jacksonchen666 quits [Ping timeout: 245 seconds]
04:46:58etnguyen03 quits [Client Quit]
04:59:32Exorcism quits [Read error: Connection reset by peer]
05:17:14hitgrr8 joins
05:21:08BlueMaxima quits [Read error: Connection reset by peer]
05:22:28Exorcism (exorcism) joins
05:23:11girst quits [Ping timeout: 252 seconds]
05:25:48magmaus3 quits [Ping timeout: 265 seconds]
05:41:09magmaus3 (magmaus3) joins
05:55:38magmaus3 quits [Ping timeout: 252 seconds]
05:58:48benjins joins
06:00:10benjins2__ joins
06:01:41benjinsmi quits [Ping timeout: 252 seconds]
06:02:32benjins2_ quits [Ping timeout: 265 seconds]
06:14:16magmaus3 (magmaus3) joins
06:19:17magmaus3 quits [Ping timeout: 252 seconds]
06:23:41dumbgoy quits [Ping timeout: 252 seconds]
06:41:24xarph joins
06:54:44cascode quits [Ping timeout: 265 seconds]
06:55:22cascode joins
07:09:20Unholy23613166180851599 quits [Ping timeout: 252 seconds]
07:25:41Exorcism|tor (exorcism) joins
07:26:40cascode quits [Read error: Connection reset by peer]
07:26:57cascode joins
07:27:54Exorcism|tor quits [Client Quit]
07:38:11Exorcism|tor (exorcism) joins
07:45:27Exorcism|tor quits [Client Quit]
08:04:36flashfire42 quits [Client Quit]
08:04:37kiska quits [Client Quit]
08:04:37Ryz263 quits [Client Quit]
08:04:37s-crypt2 quits [Client Quit]
08:07:04Ryz263 (Ryz) joins
08:07:05s-crypt2 (s-crypt) joins
08:07:12flashfire42 joins
08:08:45kiska (kiska) joins
08:13:43Exorcism|tor (exorcism) joins
08:16:25Exorcism|tor quits [Client Quit]
08:16:54jacksonchen666 (jacksonchen666) joins
08:17:57Island quits [Read error: Connection reset by peer]
08:34:39Exorcism|tor (exorcism) joins
08:39:07Exorcism|tor quits [Client Quit]
08:46:03<h2ibot>Exorcism uploaded File:WikiForge logo.png: https://wiki.archiveteam.org/?title=File%3AWikiForge%20logo.png
08:52:18icedice (icedice) joins
08:52:32icedice quits [Client Quit]
09:03:16magmaus3 (magmaus3) joins
09:04:23magmaus3 quits [Client Quit]
10:00:01railen63 quits [Remote host closed the connection]
10:00:19railen63 joins
10:02:35pabs quits [Ping timeout: 252 seconds]
10:07:46pabs (pabs) joins
10:10:59Peroniko joins
11:02:06icedice (icedice) joins
11:14:25Exorcism quits [Remote host closed the connection]
11:14:53Exorcism (exorcism) joins
11:30:43AmAnd0A quits [Remote host closed the connection]
11:30:56AmAnd0A joins
11:33:41jacksonchen666 quits [Remote host closed the connection]
11:33:53IDK (IDK) joins
11:34:38jacksonchen666 (jacksonchen666) joins
12:03:39RetiredTurtle joins
12:05:31Peroniko quits [Ping timeout: 265 seconds]
12:17:06jacksonchen666 quits [Ping timeout: 245 seconds]
12:20:59jacksonchen666 (jacksonchen666) joins
12:23:42Exorcism quits [Remote host closed the connection]
12:24:28Exorcism (exorcism) joins
12:29:14Peroniko joins
12:31:08RetiredTurtle quits [Ping timeout: 265 seconds]
12:36:04qwertyasdfuiopghjkl quits [Client Quit]
12:40:26igloo22225 quits [Ping timeout: 252 seconds]
12:46:02hitgrr8 quits [Client Quit]
12:48:35igloo22225 (igloo22225) joins
13:00:43parfait_ joins
13:04:58kdqep__ quits [Ping timeout: 265 seconds]
13:22:17qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
13:33:47Arcorann quits [Ping timeout: 252 seconds]
13:50:48icedice quits [Client Quit]
13:51:00balrog quits [Quit: Bye]
13:52:03balrog (balrog) joins
13:57:34Exorcism quits [Remote host closed the connection]
13:58:08Exorcism (exorcism) joins
13:58:20icedice (icedice) joins
14:01:18etnguyen03 (etnguyen03) joins
14:07:27dumbgoy joins
14:18:12fireonlive quits [Remote host closed the connection]
14:18:53fireonlive (fireonlive) joins
14:21:06PredatorIWD joins
14:33:14Exorcism quits [Remote host closed the connection]
14:33:42Exorcism (exorcism) joins
14:36:28Exorcism6 (exorcism) joins
14:37:41Exorcism quits [Read error: Connection reset by peer]
14:37:41Exorcism6 is now known as Exorcism
14:40:02<h2ibot>JacksonChen666 edited URLTeam (+613, add go.enderman.ch URL shortener): https://wiki.archiveteam.org/?diff=50889&oldid=50878
14:40:03<h2ibot>Contributor edited List of websites excluded from the Wayback Machine (-41, added Finnish-language 'perunamaa.net'; sorted…): https://wiki.archiveteam.org/?diff=50890&oldid=50766
14:42:48PredatorIWD quits [Read error: Connection reset by peer]
14:44:03<h2ibot>JustAnotherArchivist edited List of websites excluded from the Wayback Machine (+231, Add note about bot): https://wiki.archiveteam.org/?diff=50891&oldid=50890
14:45:30PredatorIWD joins
14:52:09<@JAA>https://foiaonline.gov/foiaonline/action/public/submissionDetails?trackingNumber=EPA-R5-2013-010234&type=Request has so many records that the web interface doesn't even display the number.
14:52:56<@JAA>The API returns `recordsTotal: 99999` but who knows whether that's true.
14:53:05<@JAA>Seems unlikely. :-P
14:55:49<@JAA>Or perhaps that's the limit of their system. Wouldn't even surprise me.
14:59:22Exorcism7 (exorcism) joins
15:00:06<h2ibot>JAABot edited List of websites excluded from the Wayback Machine (+0): https://wiki.archiveteam.org/?diff=50892&oldid=50891
15:00:11Exorcism quits [Killed (NickServ (GHOST command used by Exorcism7))]
15:00:18Exorcism7 is now known as Exorcism
15:01:07@JAA wonders whether we should keep a list of websites *formerly* excluded from the WBM.
15:02:21<@JAA>2023-09-24 14:58:54.582Z INFO request:EPA-R5-2013-010234:Request Got 1827 records
15:02:26<@JAA>It did not have 99999 records.
15:06:08<@JAA>lol, once you get near the end of the pagination, the API does return 1827.
15:06:28<@JAA>I'm glad I didn't rely on that number at all in my code.
15:16:10<jacksonchen666>.quit
15:16:13jacksonchen666 quits [Client Quit]
15:21:08Exorcism quits [Remote host closed the connection]
15:21:45Exorcism (exorcism) joins
16:03:19zhongfu quits [Ping timeout: 265 seconds]
16:07:45Exorcism quits [Remote host closed the connection]
16:08:12Exorcism (exorcism) joins
16:16:25<imer>added an advanced usage section with ipv6 instructions to the running projects with docker page on the wiki which needs approval please :)
16:21:58zhongfu (zhongfu) joins
16:27:31girst (girst) joins
16:33:38AmAnd0A quits [Ping timeout: 252 seconds]
16:34:05AmAnd0A joins
16:54:26Peroniko quits [Client Quit]
17:10:28icedice quits [Client Quit]
17:23:18icedice (icedice) joins
17:28:29<h2ibot>Imer edited Running Archive Team Projects with Docker (+8056, Added Advanced usage section w/ ipv6 guide): https://wiki.archiveteam.org/?diff=50893&oldid=50373
17:28:30<h2ibot>JustAnotherArchivist changed the user rights of User:Imer
17:28:39<imer>oh, thats me!
17:28:43<imer>thanks JAA
17:29:27<@JAA>*record scratch*
17:37:34etnguyen03 quits [Ping timeout: 265 seconds]
17:47:33<h2ibot>Imer edited Running Archive Team Projects with Docker (+2078, Advanced usage: Resource control via…): https://wiki.archiveteam.org/?diff=50894&oldid=50893
17:49:37<imer>thats all for now, if someone wants to give that a read over for bad wording and such that'd be great
17:54:18<imer>the URL template thingy links to www.webcitation.org which has an ssl error as the cert is only valid for non-www btw, not sure where to fix that
17:54:50Wohlstand (Wohlstand) joins
18:00:26etnguyen03 (etnguyen03) joins
18:01:31<@JAA>They used to have a cert valid for both. Ugh...
18:02:13<@JAA>So now lots of links across the web to their snapshots are broken.
18:02:25<@JAA><palpatine_ironic.png>
18:02:27Exorcism2 (exorcism) joins
18:02:35<fireonlive>sheesh lol
18:03:00<fireonlive>i get that www. is outdated but at least redirect it if you used it before ;)
18:03:35<h2ibot>JustAnotherArchivist edited Template:Url (-4, WebCite messed up their TLS certs in early 2022…): https://wiki.archiveteam.org/?diff=50895&oldid=50604
18:03:43<@JAA>I wonder wtf this cert is about: https://crt.sh/?id=9706125760
18:04:10<fireonlive>oh that reminds me i had a dream that crt.sh was shutting down and i was kinda sad
18:04:20Exorcism quits [Read error: Connection reset by peer]
18:04:20Exorcism2 is now known as Exorcism
18:04:22<fireonlive>@_@
18:04:58<fireonlive>hmm amazon issued… wonder if it’s one of their load balancer thingies
18:05:10<fireonlive>kinda like cloudflare and their shared san certs
18:05:53<@JAA>Fortunately, crt.sh is 'just' one mirror of the (aggregated) CT logs, but yeah.
18:07:13<fireonlive>mm, and it’s open source so i guess you could set up your own postgres server that outputs html if you really wanted to
18:07:38<@JAA>Yeah. I'm sure it requires a decent degree of masochism though.
18:08:01<fireonlive>haha for sure
18:20:53Exorcism9 (exorcism) joins
18:21:24<that_lurker>And if you check the cert for the domain medicine20.com its a wildcard cert for *.jmirx.org
18:21:47beario_ joins
18:22:48Exorcism quits [Read error: Connection reset by peer]
18:22:48Exorcism9 is now known as Exorcism
18:24:11beario quits [Ping timeout: 252 seconds]
18:24:44etnguyen03 quits [Ping timeout: 252 seconds]
18:26:02etnguyen03 (etnguyen03) joins
18:28:08<@JAA>I quickly checked what's in the WBM in terms of actual CT logs. There's a fair bit, and the ones I sampled come from https://archive.org/details/certificate-transparency , which I understood to be web crawls seeded by CT data, not CT logs themselves. The number of views on those items also make it unlikely that they're only CT logs. So I wonder how complete that is.
18:29:07<@JAA>Also, the key resource for actual CT logs is https://github.com/google/certificate-transparency-community-site/blob/d4b7663faa46b7c599d73e79e5efa10cba07cac8/docs/google/known-logs.md (and RFC 6962 for the API definition).
18:41:41<h2ibot>Exorcism edited WikiTeam (+0): https://wiki.archiveteam.org/?diff=50896&oldid=50638
18:41:42<h2ibot>Exorcism edited WikiTeam (+0): https://wiki.archiveteam.org/?diff=50897&oldid=50896
18:41:47etnguyen03 quits [Ping timeout: 252 seconds]
19:06:31etnguyen03 (etnguyen03) joins
19:11:12<that_lurker>Would streaming the cert transparency logs to a log with daily rotation + upload be the best way to archive them
19:12:20<Barto>fireonlive: there's an open pgsql db behind crt.sh, so if they ever shut down, it will be an easy job to dump maybe :-)
19:13:01IDK quits [Client Quit]
19:15:29<fireonlive>ah yes that too :3
19:15:34<fireonlive>:)
19:15:51<@JAA>As long as they don't shut down without announcement, but that seems unlikely given how popular the site is.
19:25:30graham joins
19:58:44DogsRNice joins
20:04:36IDK (IDK) joins
20:10:53cascode quits [Ping timeout: 252 seconds]
20:14:57cascode joins
20:31:05etnguyen03 quits [Ping timeout: 265 seconds]
20:32:32AmAnd0A quits [Ping timeout: 265 seconds]
20:32:36AmAnd0A joins
20:36:44AmAnd0A quits [Read error: Connection reset by peer]
20:37:20AmAnd0A joins
20:46:54<Barto>https://nitter.net/MinecraftWikiEN/status/1706004078206103965/
20:47:04etnguyen03 (etnguyen03) joins
20:50:29AmAnd0A quits [Ping timeout: 252 seconds]
20:50:58AmAnd0A joins
20:53:13<fireonlive>they linked to something called “getindie.wiki” which could be an interesting source: https://twitter.com/MinecraftWikiEN/status/1706004084497502439
20:53:13<eggdrop>nitter: https://nitter.net/MinecraftWikiEN/status/1706004084497502439
20:53:17Island joins
20:53:33<fireonlive>oops right, nitter share box copies a twitter url lol
20:56:07<Barto>:-)
21:02:08beario joins
21:04:04Peroniko joins
21:04:26beario_ quits [Ping timeout: 265 seconds]
21:45:18lukash94 joins
21:46:00lukash9 quits [Ping timeout: 265 seconds]
21:46:00lukash94 is now known as lukash9
21:47:40apache2 quits [Remote host closed the connection]
21:48:10decky joins
21:48:32coderobe quits [Quit: Ping timeout (120 seconds)]
21:48:34Arachnophine quits [Quit: Ping timeout (120 seconds)]
21:48:35CraftByte quits [Quit: Ping timeout (120 seconds)]
21:48:42apache2 joins
21:48:48Matthww11 quits [Quit: Ping timeout (120 seconds)]
21:48:49aGerman quits [Quit: Ping timeout (120 seconds)]
21:48:52mgrytbak quits [Quit: Ping timeout (120 seconds)]
21:48:54coderobe (coderobe) joins
21:49:01lea quits [Quit: quit.]
21:49:02mgrytbak joins
21:49:03Matthww11 joins
21:49:14lea (lea_) joins
21:49:49@OrIdow6 quits [Quit: ZNC 1.8.2+deb3.1 - https://znc.in]
21:49:49cultpony quits [Quit: ZNC - https://znc.in]
21:49:49Suika quits [Quit: Server is ded]
21:49:50@arkiver quits [Remote host closed the connection]
21:49:54HotSwap` quits [Quit: ZNC - http://znc.in]
21:49:55Hecz quits [Remote host closed the connection]
21:50:02HotSwap joins
21:50:05Suika joins
21:50:07cultpony (cultpony) joins
21:50:13ThetaDev quits [Quit: No Ping reply in 180 seconds.]
21:50:20JensRex quits [Remote host closed the connection]
21:50:21Sluggs_ joins
21:50:34arkiver (arkiver) joins
21:50:34@ChanServ sets mode: +o arkiver
21:51:01OrIdow6 (OrIdow6) joins
21:51:01@ChanServ sets mode: +o OrIdow6
21:51:01CraftByte (DragonSec|CraftByte) joins
21:51:03Arachnophine (Arachnophine) joins
21:51:04Hecz joins
21:51:11Sluggs quits [Read error: Connection reset by peer]
21:51:15aGerman (aGerman) joins
21:51:23Hecz quits [Changing host]
21:51:23Hecz (Hecz) joins
21:51:43ThetaDev joins
21:51:45JensRex (JensRex) joins
21:51:48decky_e quits [Ping timeout: 265 seconds]
21:53:57<@JAA>FOIAonline download is looking fine so far. 20% done. Projected size is only 1.8 TiB now, but that might change since I didn't randomise the list of requests and am therefore going through them in chronological order.
21:56:17<h2ibot>JustAnotherArchivist created FOIAonline (+555, Created page with "{{Infobox project | URL =…): https://wiki.archiveteam.org/?title=FOIAonline
21:56:18<h2ibot>JustAnotherArchivist edited Deathwatch (-221, /* 2023 */ Link to FOIAonline page): https://wiki.archiveteam.org/?diff=50899&oldid=50879
21:57:14<@JAA>That's after about 18 hours, so should easily finish in time.
21:57:39Miki57 joins
21:59:57AmAnd0A quits [Read error: Connection reset by peer]
22:00:13AmAnd0A joins
22:01:59Miki_57 quits [Ping timeout: 252 seconds]
22:06:06qwertyasdfuiopghjkl quits [Remote host closed the connection]
22:17:05magmaus3 (magmaus3) joins
22:19:46cascode quits [Read error: Connection reset by peer]
22:20:19cascode joins
22:35:32etnguyen03 quits [Ping timeout: 252 seconds]
22:58:38etnguyen03 (etnguyen03) joins
23:23:56etnguyen03 quits [Ping timeout: 252 seconds]
23:35:06flashfire42 quits [Client Quit]
23:35:07kiska quits [Client Quit]
23:35:07Ryz263 quits [Client Quit]
23:35:07s-crypt2 quits [Client Quit]
23:35:45RetiredTurtle joins
23:37:17Ryz263 (Ryz) joins
23:37:19s-crypt2 (s-crypt) joins
23:37:27flashfire42 joins
23:38:14Peroniko quits [Ping timeout: 252 seconds]
23:39:00kiska (kiska) joins
23:46:10Exorcism quits [Remote host closed the connection]
23:47:13Exorcism (exorcism) joins
23:49:14wessel1512 quits [Ping timeout: 252 seconds]

HomeSearchPrevious dayNext day