HomeSearchPrevious dayNext day

00:02:00@rewby quits [Ping timeout: 255 seconds]
00:05:03invadeuz quits [Quit: Ooops, wrong browser tab.]
00:07:51invadeuze quits [Ping timeout: 255 seconds]
00:10:49TheEnbyperor joins
00:10:56TheEnbyperor_ (TheEnbyperor) joins
00:12:10rewby (rewby) joins
00:12:10@ChanServ sets mode: +o rewby
00:16:19TheEnbyperor quits [Read error: Connection reset by peer]
00:16:30TheEnbyperor_ quits [Read error: Connection reset by peer]
00:16:51@rewby quits [Ping timeout: 255 seconds]
00:20:45TheEnbyperor joins
00:21:04TheEnbyperor_ (TheEnbyperor) joins
00:33:47rewby (rewby) joins
00:33:47@ChanServ sets mode: +o rewby
00:46:01etnguyen03 quits [Ping timeout: 252 seconds]
00:47:43etnguyen03 (etnguyen03) joins
00:58:52chunkynutz601816946658 joins
01:01:03chunkynutz60181694665 quits [Ping timeout: 252 seconds]
01:01:03chunkynutz601816946658 is now known as chunkynutz60181694665
01:03:22nexussfan quits [Remote host closed the connection]
01:34:32nexussfan (nexussfan) joins
01:39:33wickedplayer494 quits [Ping timeout: 252 seconds]
01:41:14wickedplayer494 (wickedplayer494) joins
01:42:36nexussfan quits [Client Quit]
01:44:35nexussfan (nexussfan) joins
01:44:49nexussfan quits [Client Quit]
02:09:21grill quits [Ping timeout: 255 seconds]
02:09:59grill (grill) joins
02:10:17etnguyen03 quits [Client Quit]
02:10:29etnguyen03 (etnguyen03) joins
02:19:50@rewby quits [Ping timeout: 268 seconds]
02:23:03rewby (rewby) joins
02:23:03@ChanServ sets mode: +o rewby
02:23:59thewinwin872 joins
02:27:51thewinwin87 quits [Ping timeout: 268 seconds]
02:27:57thewinwin872 is now known as thewinwin87
02:28:48etnguyen03 quits [Remote host closed the connection]
02:53:27<pabs>https://www.enricozini.org/blog/2026/migrating-away-from-org-net-com-domains/
03:22:17justauser quits [Read error: Connection reset by peer]
03:22:18justause- (justauser) joins
03:32:22PredatorIWD88 joins
03:34:27PredatorIWD8 quits [Ping timeout: 268 seconds]
03:34:28PredatorIWD88 is now known as PredatorIWD8
03:48:55DogsRNice quits [Read error: Connection reset by peer]
03:49:15hexagonwin quits [Ping timeout: 268 seconds]
03:53:15hexagonwin (hexagonwin) joins
04:01:02nicolas17 quits [Quit: Konversation terminated!]
04:22:24tertu quits [Read error: Connection reset by peer]
04:25:25tertu (tertu) joins
04:57:54nine quits [Quit: See ya!]
04:58:07nine joins
05:59:59TheEnbyperor quits [Read error: Connection reset by peer]
06:00:05TheEnbyperor_ quits [Read error: Connection reset by peer]
06:01:21@rewby quits [Ping timeout: 252 seconds]
06:07:59cascode joins
06:10:53TheEnbyperor_ joins
06:12:18rewby (rewby) joins
06:12:18@ChanServ sets mode: +o rewby
06:20:24TheEnbyperor_ quits [Read error: Connection reset by peer]
06:21:31@rewby quits [Ping timeout: 252 seconds]
06:21:40TheEnbyperor joins
06:23:14hexa quits [Quit: Disconnected]
06:24:08kl1n3 (kline) joins
06:24:21hexa- (hexa-) joins
06:24:39kline quits [Ping timeout: 268 seconds]
06:24:54rewby (rewby) joins
06:24:54@ChanServ sets mode: +o rewby
06:34:21ScenarioPlanet quits [Ping timeout: 252 seconds]
06:34:25nathang218438 quits [Quit: The Lounge - https://thelounge.chat]
06:35:08TheTechRobo quits [Ping timeout: 268 seconds]
06:35:09Pedrosso quits [Ping timeout: 268 seconds]
06:41:06nathang218438 joins
06:44:55TheTechRobo (TheTechRobo) joins
06:45:31ScenarioPlanet (ScenarioPlanet) joins
06:45:45Pedrosso joins
06:51:05nathang2184385 joins
06:51:30nathang218438 quits [Ping timeout: 255 seconds]
06:51:30nathang2184385 is now known as nathang218438
07:17:17LddPotato_ joins
07:20:09LddPotato quits [Ping timeout: 268 seconds]
07:20:18LddPotato_ is now known as LddPotato
07:56:22Webuser545493 joins
07:56:41Webuser545493 quits [Client Quit]
08:51:13TheEnbyperor quits [Read error: Connection reset by peer]
08:54:21@rewby quits [Ping timeout: 255 seconds]
08:59:46girst quits [Quit: ZNC 1.8.2 - https://znc.in]
09:00:06girst (girst) joins
09:00:23Bleo182600722719623455222011079 quits [Quit: The Lounge - https://thelounge.chat]
09:01:46TheEnbyperor_ joins
09:03:08Bleo182600722719623455222011079 joins
09:04:10rewby (rewby) joins
09:04:10@ChanServ sets mode: +o rewby
09:19:04cascode_ joins
09:19:05cascode quits [Read error: Connection reset by peer]
09:19:30cascode_ quits [Read error: Connection reset by peer]
09:19:42cascode joins
09:35:29NeonGlitch quits [Ping timeout: 252 seconds]
09:35:56atphoenix_ quits [Remote host closed the connection]
09:36:08NeonGlitch (NeonGlitch) joins
09:36:23atphoenix_ (atphoenix) joins
09:59:09Keijo joins
09:59:43<Keijo>Hi! Quick Wayback Machine question.
09:59:43<Keijo>I’m trying to find out whether Internet Archive has introduced any new site-owner removal/exclusion procedures, forms, ownership-verification methods, self-service tools, APIs, or other supported ways to exclude a URL, date range, multiple pages, or an entire domain from the public Wayback Machine.
09:59:43<Keijo>I’m also trying to find the best official Internet Archive channel to follow so I can learn about future changes as soon as possible.
09:59:43<Keijo>Does anyone here closely follow Wayback exclusions/removals, or know the best person/team/source to ask?
09:59:43<Keijo>Thanks!
10:00:29Keijo quits [Client Quit]
10:05:00nune quits [Remote host closed the connection]
10:05:01nune joins
10:06:06sirio joins
10:06:58Webuser647446 joins
10:07:18<Webuser647446>Hi! Quick Wayback Machine question. I’m trying to find out whether Internet Archive has introduced any new site-owner removal/exclusion procedures, forms, ownership-verification methods, self-service tools, APIs, or other supported ways to exclude a URL, date range, multiple pages, or an entire domain from the public Wayback Machine.
10:07:18<Webuser647446>I’m also trying to find the best official Internet Archive channel to follow so I can learn about future changes as soon as possible. Does anyone here closely follow Wayback exclusions/removals, or know the best person/team/source to ask? Thanks!
10:08:11Webuser647446 quits [Client Quit]
10:10:10pepsi2024 joins
10:42:35Umbire quits [Ping timeout: 252 seconds]
10:58:21oxtyped quits [Ping timeout: 252 seconds]
11:02:46Webuser014011 quits [Quit: Ooops, wrong browser tab.]
11:08:19APOLLO03 quits [Ping timeout: 268 seconds]
11:09:09oxtyped joins
11:53:46graham90 joins
12:25:58Matthww5 quits [Quit: The Lounge - https://thelounge.chat]
12:28:31Matthww5 joins
12:30:06Matthww5 quits [Client Quit]
12:32:13Matthww5 joins
12:41:39d02 joins
12:47:53Webuser541121 joins
12:50:33<anarcat>pokechu22: i think you were the one who handled the last noblogs crawl? i got the full list of blogs now
13:05:27@JAA quits [Ping timeout: 255 seconds]
13:06:41traxys quits [Ping timeout: 252 seconds]
13:08:03graham90 quits [Client Quit]
13:08:26Island quits [Read error: Connection reset by peer]
13:10:11graham90 joins
13:10:27thewinwin875 joins
13:11:02pepsi2024 quits [Ping timeout: 268 seconds]
13:11:42pepsi2024 joins
13:11:43Island joins
13:12:28JAA (JAA) joins
13:12:28@ChanServ sets mode: +o JAA
13:13:44DigitalDragons quits [Quit: Ping timeout (120 seconds)]
13:13:44Exorcism quits [Quit: Ping timeout (120 seconds)]
13:14:01thewinwin87 quits [Ping timeout: 252 seconds]
13:14:03thewinwin875 is now known as thewinwin87
13:14:16archessmn quits [Read error: Connection reset by peer]
13:14:16magmaus3 quits [Read error: Connection reset by peer]
13:14:22magmaus3 (magmaus3) joins
13:14:26archessmn (archessmn) joins
13:17:46@arkiver quits [Remote host closed the connection]
13:18:01arkiver (arkiver) joins
13:18:01@ChanServ sets mode: +o arkiver
13:18:35BornOn420 quits [Remote host closed the connection]
13:22:40BornOn420 (BornOn420) joins
13:34:07traxys (traxys) joins
13:40:44DigitalDragons (DigitalDragons) joins
13:40:57Exorcism (exorcism) joins
13:44:02graham90 quits [Client Quit]
13:44:48graham90 joins
13:45:27graham90 quits [Client Quit]
13:47:36Cupping128544 quits [Quit: Ping timeout (120 seconds)]
13:47:51Cupping128544 joins
13:48:15graham90 joins
13:54:30Cupping128544 quits [Client Quit]
13:54:45Cupping128544 joins
14:06:09sirio quits [Quit: Ooops, wrong browser tab.]
14:06:38Webuser673223 joins
14:07:35Webuser673223 quits [Client Quit]
14:17:08Webuser685901 joins
14:19:20Webuser685901 quits [Client Quit]
14:23:00Webuser639367 joins
14:23:24@Fusl quits [Quit: K-Lined]
14:23:41Fusl (Fusl) joins
14:23:41@ChanServ sets mode: +o Fusl
14:24:21<anarcat>also, i am wondering if we have a single bundle of WARC files that would be essentially a mirror of all noblogs sites
14:25:03<anarcat>could we do a --no-offsite crawl of all the domains, maybe?
14:27:31<klea>There's a bunch of separate bundles, but they are split across many IA items, and include outlinks, IIRC. The viewer is useful to get the list of files.
14:27:48<anarcat>yeah, i know
14:27:58<anarcat>i figured it might be good to make a single, final bundle before they completely shutdown
14:28:10<anarcat>i've asked if the blogs will be put readonly before full retirement, that would be best
14:28:53<klea>hmm, maybe, or if it's really important, we could ignore anything that's not noblogs on the last job (ask poke if you want to do that), since there has already been a few jobs very recently.
14:28:56d02 quits [Remote host closed the connection]
14:29:24<anarcat>i guess maybe that's just not possible, a !a < transfer.archiveteam.am just spwans new jobs, right?
14:29:40<klea>!a < link spawns a single new job, not multiple.
14:29:48d02 joins
14:29:50<klea>(can only be done by ops as it's an edge case)
14:29:51<anarcat>at least all the stuff i'm looking at in https://archive.fart.website/archivebot/viewer/domain/transfer.archivete.am are mostly empty
14:30:12entrox joins
14:30:13<anarcat>anyways, that's what i was thinking, to have a single job with a single set of WARC files, for all of noblogs with --no-offsite
14:30:16<klea>They get indexed as urls-transfer.archivete.am- something IIRC.
14:31:10<klea>(https://archive.fart.website/archivebot/viewer/domain/urls-transfer.archivete.am-topics.noblogs.org_json.txt)
14:37:41<anarcat>that one is also tiny
14:38:30<klea>yes.
14:47:00nine quits [Read error: Connection reset by peer]
14:47:13nine joins
14:59:51<h2ibot>TripleCamera edited Sina Cloud (+186, /* External link */): https://wiki.archiveteam.org/?diff=63867&oldid=63828
15:02:55d02 quits [Client Quit]
15:15:30earthnative quits [Ping timeout: 255 seconds]
15:37:58beastbg8 quits [Read error: Connection reset by peer]
15:38:07guicale- quits [Quit: ZNC 1.9.1+deb2+b3 - https://znc.in]
15:38:20beastbg8 joins
15:38:29guicale (guicale) joins
15:40:58BornOn420 quits [Remote host closed the connection]
15:42:08BornOn420 (BornOn420) joins
15:43:26kiska (kiska) joins
15:59:46<anarcat>https://www.inventati.org/ale/noblogs.jsonl.gz is a list of domain -> last update maps
16:04:44<anarcat>does archivebot retry 429s?
16:04:57<justause->It does, twice IIRC.
16:18:38justause- is now known as justauser
16:19:51unknownsrc quits [Ping timeout: 255 seconds]
16:20:59unknownsrc (unknownsrc) joins
16:24:45LddPotato quits [Read error: Connection reset by peer]
16:24:50fuzzy80211 quits [Read error: Connection reset by peer]
16:25:07fuzzy80211 (fuzzy80211) joins
16:25:36LddPotato joins
16:52:12earthnative (earthnative) joins
16:55:41<pokechu22>anarcat: the annoying thing is that a few of them loop, e.g.
16:55:43<pokechu22>https://fa86.noblogs.org/www.monde-libertaire.fr/venez-nombreux-lundi-6-janvier-2014-a-14h-accompagner-cerasela-a-la-prefecture/Les-parents-soutiennent-la-famille-russe-menacee-d-expulsion-1867616/Les-parents-soutiennent-la-famille-russe-menacee-d-expulsion-1867616/Les-parents-soutiennent-la-famille-russe-menacee-d-expulsion-1867616/?paged=105 (which is the same as
16:55:45<pokechu22>https://fa86.noblogs.org/?paged=105 ). There wasn't any clean ignore for those either; it was a lot of whacking extra ignores.
17:10:50lev (lev) joins
17:11:01lev quits [Client Quit]
17:11:35lev (lev) joins
17:16:28Mateon1 quits [Remote host closed the connection]
17:18:22lev quits [Ping timeout: 240 seconds]
17:37:26datechnoman quits [Ping timeout: 268 seconds]
17:38:08datechnoman (datechnoman) joins
17:49:05traxys quits [Quit: The Lounge - https://thelounge.chat]
17:49:52traxys (traxys) joins
17:54:32<justauser>cruller or anybody else with grab-site: Help needed, https://transfer.archivete.am/SUeQa/grab-site-request.txt .
17:54:33<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/SUeQa/grab-site-request.txt
18:05:51Webuser639367 quits [Quit: tv.disney.es]
18:10:41<justauser>^ Also http://www.tofuproxy.stargrave.org/ .
18:19:21<anarcat>i'm going to make a list of noblogs that changed since the last full crawl in 2025
18:26:01DogsRNice joins
18:30:00msfjarvis quits [Quit: Lurker 2.2.1 (the truth is out there) https://lurker.chat]
18:30:49msfjarvis joins
18:53:08<pokechu22>anarcat: note that invadeuz gave us another list of users, which included 8136 blogs that hadn't been seen before. https://transfer.archivete.am/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt
18:53:10<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt
18:53:54<anarcat>pokechu22: well i have a list from an authoritative source
18:54:00<anarcat>like the entire list
18:54:42<pokechu22>Yeah, the one invadeuz gave was also an entire list (which I deduped with previous lists) -- I guess I should compare the two
18:54:54<pokechu22>it was just usernames, without info about last updated though
18:55:47<anarcat>this is what i was given https://transfer.archivete.am/WrIsV/noblogs.jsonl.gz⏎
18:55:49<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/WrIsV/noblogs.jsonl.gz⏎
18:55:51<anarcat>bah
18:55:55<anarcat>this is what i was given https://transfer.archivete.am/WrIsV/noblogs.jsonl.gz
18:55:59<anarcat>without the weird suffix
18:56:45<anarcat>parsing and filtering by timestamp since the last crawl start gives me this list of 2829 updated blogs: https://transfer.archivete.am/14GLup/noblogs-post2025-09-17T21:21:51-04:00.txt
18:56:46<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/14GLup/noblogs-post2025-09-17T21:21:51-04:00.txt
18:57:00<anarcat>there are incorrect timestamps in the JSON files (all zeros), which i interpret as "needs a crawl"
18:57:56<anarcat>here's the full list of all the 22385 noblogs domain as a flat text file https://transfer.archivete.am/RUxZG/noblogs-domains.txt
18:57:56<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/RUxZG/noblogs-domains.txt
18:58:06<anarcat>i think we should start a crawl of https://transfer.archivete.am/14GLup/noblogs-post2025-09-17T21:21:51-04:00.txt immediately
18:58:24<anarcat>a no-offsite one, to speed things up, perhaps
18:58:39<anarcat>then, maybe in parallel of after, recrawl all the domains
18:58:58<anarcat>things will apparently shutdown in 10 days, according to https://cavallette.noblogs.org/2026/09/10099 (posted today)
19:03:03<pokechu22>~1200 of those are in https://transfer.archivete.am/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt meaning they were created relatively recently; the other ~1600 are older blogs with new posts it seems
19:03:04<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt
19:05:34<justauser>cruller: Sorry, looks like it was a false alarm.
19:07:22<anarcat>i am not sure i have the skills and time to nurse those crawls
19:07:23<pokechu22>I think maybe I'll do that 1600 in one job (with outlinks?) and then simultaneously another big job with every subdomain from any list (including those that went to wp-signup) without outlinks
19:07:25<anarcat>pokechu22: do you have itÉ
19:07:29<anarcat>pokechu22: do you have it? *
19:07:52<anarcat>https://transfer.archivete.am/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt was crawled?
19:07:52<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt
19:07:53<pokechu22>Yeah, I can set it up soon
19:08:11<anarcat>cool, thanks so much
19:08:13<pokechu22>That one is in progress but hasn't produced enough data to end up on archive.fart.website yet
19:08:17<anarcat>gotcha
19:08:21abirkill quits [Quit: Let us prepare to grapple with the ineffable itself, and see if we may not eff it after all.]
19:08:37<anarcat>is there going to be a single crawl with all the blogs in a single set of warc files we could mirror externally?
19:10:26abirkill (abirkill) joins
19:13:02<pokechu22>Yes, I'd like to do that
19:13:47<@arkiver>JAA: thank for the those with the hashcah challenge
19:13:55<pokechu22>that is the one where https://fa86.noblogs.org/ etc would cause problems, but if it's shutting down in 10 days we can probably just let it download as much as we can
19:14:50<pokechu22>... ugh, and https://fa86.noblogs.org/ doesn't have a sitemap. That actually would be a problem because the giant messy recursion actually is needed to discover some posts
19:16:14<pokechu22>That's one where we've definitely got everything in a previous job, just would be difficult to get everything again before the deadline
19:16:42<anarcat>for sure
19:16:51<anarcat>we'll do our best?
19:21:31<pokechu22>Started https://transfer.archivete.am/QPRFL/noblogs.org_old_subdomains_with_new_posts_sep_2026.txt which is deduplicated against https://transfer.archivete.am/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt. I'm going to also combine all of my lists into one big list next
19:21:31<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/QPRFL/noblogs.org_old_subdomains_with_new_posts_sep_2026.txt https://transfer.archivete.am/inline/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt.
19:22:09<anarcat>ack thanks
19:24:04<anarcat>pokechu22: i was told to "go full throttle", the worst that can happen is to hit 429 statuses
19:24:08<anarcat>pokechu22: so that could speed things up a bit
19:24:37<pokechu22>Hmm, previously it's gotten ip-banned for a week (from too many 404s I think?) but I guess if we have contact with them that's not an issue
19:24:50<pokechu22>but also if this is a list known to have posts that's less of a risk
19:25:14<anarcat>oh i don't know if it's a list known to have posts
19:25:20<pokechu22>Do you know if wp-json is restricted on all blogs now? It wasn't before
19:25:34<anarcat>relayed
19:25:35<anarcat>15:25:20 <pokechu22> Do you know if wp-json is restricted on all blogs now? It wasn't before
19:25:36<anarcat>er
20:02:41erebion (erebion) joins
20:03:20nicolas17 (nicolas17) joins
20:03:39<h2ibot>Ryz edited List of websites excluded from the Wayback Machine (+95, Added https://www.maxivista.com/ -…): https://wiki.archiveteam.org/?diff=63868&oldid=63862
20:03:55<erebion>I guess people here have heard about autistici.org shutting down, but I couldn't find it on Deathwatch. They have just announced shutdown in 10 days. https://keepitfree.ai/technical-updates/technical-howto-to-back-up-your-data-on-a/i-services/
20:04:24<erebion>Their domain autistici.org is no longer reachable due to US sanctions, but services can be reached by amending /etc/hosts:
20:04:25<erebion>88.99.75.198 git.autistici.org
20:04:30<erebion>93.190.126.19 autistici.org www.autistici.org conference.jabber.autistici.org live.autistici.org
20:04:35<erebion>I hope their stuff can be saved
20:04:59<pokechu22>Yes, we're aware and have been saving things - we got autistici.org using archivebot before the shutdown
20:05:03<erebion>I have managed to crawl their GitLab for several hours and find 440 public repos, in case anyone wants that list for mirroring :)
20:05:17<erebion>They have a lot of code
20:05:22<pokechu22>I think we crawled it in #codearchiver before the DNS was pulled but I'm not 100% sure
20:05:49<pokechu22>Yeah, looks like it was ran on the 26th
20:06:01<erebion>They had changes after that, even today
20:06:22<erebion>Is it continuous mirroring?
20:06:22<pokechu22>JAA: does it make sense to create more bundles with faked DNS?
20:06:50<pokechu22>I'm not sure if it's continuous (and if it is continuous, I'm not sure if it would continue once the domain was pulled)
20:08:03<@JAA>It was single-shot.
20:08:08<erebion>I've been trying to manually mirrors for every repo on my own Forgejo instance, but realised it's more work than I could handle... :/
20:08:38<@JAA>https://archive.org/details/git.autistici.org_bundles_20260826
20:10:32<erebion>374 bundles, I am aware of 448 repos
20:10:40<h2ibot>KleaBot edited List of websites excluded from the Wayback Machine (+0, Reordered websites and/or updated count.): https://wiki.archiveteam.org/?diff=63869&oldid=63868
20:11:04<@JAA>→ #codearchiver
20:11:40<h2ibot>Pokechu22 edited Deathwatch (+347, /* 2026-09 */ mention Autistici/Inventati): https://wiki.archiveteam.org/?diff=63870&oldid=63850
20:11:41<klea>JAA: Why not make a channel for AutistiCI instead of redirecting things to another channel?
20:12:20<masterx244|m>especially since its a case of stuff on multiple fronts burning due to the same dumpster fire
20:12:41<pokechu22>IMO I'd rather keep it in #-bs for now
20:13:41<anarcat>yeah, it's pretty much all we're talking about here anyways :p
20:14:31<klea>Yeah, this is the dumpster fire channel!
20:14:56<masterx244|m>and its one of those "shit burns urgently but not a longterm thing" cases
20:15:28<anarcat>yeah, it's going away in 10 days
20:15:57<erebion>I wonder whether the autistici services all really have to shutdown. The git seems to be the easiest service to keep running elsewhere with different domain.
20:16:12<erebion>I know email would be an issue...
20:16:28<anarcat>the problem is not just the domain shutdown, they dealt with that pretty quickly
20:16:40<masterx244|m>smells like more sanction fuckery
20:16:56<masterx244|m>(132 in binary shown with the fingers towards trump....)
20:17:09<anarcat>the US has effectively shutdown their bank accounts, and essentially criminalizes anyone doing "transactions" with A/I the organization, or anyone involved with it
20:18:24<pokechu22>FWIW this channel is publicly logged, while presumably another one would not be... but I think it's probably too late for that to matter
20:18:39<anarcat>this is not private information
20:18:51<anarcat>it's been clearly publicly disclosed by AI, and that's what an OFAC does
20:19:14<pokechu22>Yes, just more of a general point, not the bank account aspect
20:19:18<anarcat>sure
20:19:48<anarcat>well since this channel is publicly logged, i'll take the opportunity to publicly say this
20:20:00<anarcat>fuck donald trump and the united states government
20:20:06<anarcat>there, i said it and will stop there
20:21:21<erebion>Alerta alerta and so on
20:21:21<erebion>yup
20:21:49<erebion>The issue is also: keepitfree.ai... .ai is controlled by a US corp
20:22:04<pokechu22>Wait, isn't that a country TLD?
20:22:17<erebion>yes, but the company managing it is from the US
20:22:51<erebion>They should have looked for a TLD from a country that does not have good relations to the US. Not saying .cn, but perhaps there's something else? idk
20:25:23<masterx244|m>pokechu22: antilles islands
20:25:39<masterx244|m>(similar repurposed how tuvalu was used for TV realted stuff
20:26:05<masterx244|m>not sure if .to works for that purpose
20:26:52<anarcat>erebion: i don't expect keepitfree.ai to stay up after the deadline, we should probably crawl it too
20:27:11<anarcat>ultimately, no domain name is out of the US control, thanks to ICANN's structure
20:28:18<erebion>anarcat: I wouldn't be suprised if it goes down before they shut down the services, probably inventati.org as well.
20:28:37<erebion>I see no reason why they would turn off one domain and not the others
20:28:38<klea>Sad that AT currently doesn't have any AB pipeline for hidden services, or i2p, or yggdrasil.
20:28:41<erebion>(they: US gov)
20:29:07<anarcat>sure
20:29:53<pokechu22>anarcat: keepitfree.ai was crawled 2 days ago
20:30:05<pokechu22>!remindme 8d re-crawl keepitfree.ai
20:30:06<eggdrop>[remind] ok, i'll remind you at 2026-09-16T20:30:05Z
20:30:45Island_ joins
20:30:47<anarcat>perfect, thanks
20:30:52cyanbox_ joins
20:31:03<anarcat>pokechu22: have you considered raising the crawl rates on the current noblogs jobs?
20:31:28<pokechu22>Oh, right, while archivebot retries 429s, they're retried at the end of the job, which could be an issue for the all subdomains one because of those looping subdomains
20:31:50<pokechu22>I'll speed up the newest one as a test and see what happens
20:32:04<anarcat>thanks
20:32:14<erebion>Has anyone tried asking autistici folks whether they could just create a .tar.gz of public data and create a torrent or something like that?
20:32:21<erebion>Seems the easiest way to save a lot
20:33:04VerifiedJ quits [Quit: The Lounge - https://thelounge.chat]
20:33:06<anarcat>those are wordpress sites, you can't just make a tarball and be on your way
20:33:18<anarcat>this is what they've done so far: https://cavallette.noblogs.org/2026/09/10099
20:33:45VerifiedJ (VerifiedJ) joins
20:34:06Island quits [Ping timeout: 255 seconds]
20:34:06scrl__ quits [Ping timeout: 255 seconds]
20:34:44<anarcat>pokechu22: is it me or the request rate went down from 1.x resp/s to more like 0.2 resp/s there?
20:34:45<pokechu22>They do also provide https://warcs.inventati.org/ which we've mostly crawled but that server behaves weirdly
20:35:44<pokechu22>It looks like it's moving faster than that but the dashboard does say it's lower. The dashboard's calculation is a bit weird though
20:36:00<anarcat>right, suspected as much
20:36:03<anarcat>well, thanks for your work
20:39:34Webuser968118 joins
20:45:16TheEnbyperor_ quits [Read error: Connection reset by peer]
20:46:13Webuser968118 quits [Client Quit]
20:47:35kl1n3 quits [Ping timeout: 252 seconds]
20:47:48kline (kline) joins
20:51:41Pendonym quits [Ping timeout: 268 seconds]
20:51:58Pendonym (Pendonym) joins
20:56:18TheEnbyperor_ joins
20:59:34Umbire joins
21:00:49PredatorIWD8 quits [Read error: Connection reset by peer]
21:00:54Island_ quits [Read error: Connection reset by peer]
21:01:04PredatorIWD8 joins
21:01:05MrMcNugg1 quits [Read error: Connection reset by peer]
21:01:09Pendonym quits [Remote host closed the connection]
21:01:26Island_ joins
21:01:29Pendonym (Pendonym) joins
21:01:29MrMcNuggets (MrMcNuggets) joins
21:01:42Bleo182600722719623455222011079 quits [Quit: Ping timeout (120 seconds)]
21:01:45VerifiedJ quits [Client Quit]
21:01:45TunaLobster445 quits [Quit: Ping timeout (120 seconds)]
21:01:57Bleo182600722719623455222011079 joins
21:02:00TunaLobster445 joins
21:02:01VerifiedJ (VerifiedJ) joins
21:03:02nine quits [Quit: See ya!]
21:03:15nine joins
21:12:16Webuser787649 joins
21:12:44Webuser787649 quits [Client Quit]
21:14:13APOLLO03 joins
21:14:53<anarcat>does archivebot crawl the domains related to the requested URLs before the outlinks?
21:15:12<anarcat>because b9nxh9izpv0sr5fd037ps7vlu is already at half a million URLs here, far beyond the other one started yesterday
21:21:12Barto quits [Quit: WeeChat 4.10.0]
21:21:30Barto (Barto) joins
21:21:33<pokechu22>It does not; all requisites/links from a page are shuffled and then added to the queue before those from the next page. (That's why I included wp-sitemap.xml in the lists: those would be discovered eventually but including them somewhat prioritizes onsite URLs)
21:22:18<pokechu22>The job started yesterday is mostly new blogs and very old blogs that weren't discovered by other means, both of which likely have very few posts. While the new list is old blogs that had new posts, which means it's more likely that they have more posts overall
21:29:47Pendonym quits [Read error: Connection reset by peer]
21:30:18<pokechu22>I guess I should also add: I'm redownloading CDX data for the full list of subdomains, since I want to include as many as possible for that job
21:31:19Pendonym (Pendonym) joins
21:36:30TheEnbyperor_ quits [Read error: Connection reset by peer]
21:38:33Pendonym quits [Ping timeout: 252 seconds]
21:38:52Pendonym (Pendonym) joins
21:39:10kline quits [Ping timeout: 268 seconds]
21:39:17@rewby quits [Ping timeout: 252 seconds]
21:39:28kline (kline) joins
21:40:08rewby (rewby) joins
21:40:08@ChanServ sets mode: +o rewby
21:41:38chunkynutz60181694665 quits [Ping timeout: 268 seconds]
21:43:01chunkynutz60181694665 joins
21:46:41TheEnbyperor joins
21:47:17TheEnbyperor_ (TheEnbyperor) joins
21:48:27@rewby quits [Ping timeout: 252 seconds]
21:50:31spelledwrongdotuk joins
22:03:06ice quits [Quit: WeeChat 4.9.5]
22:04:53Mateon1 joins
22:04:57ice joins
22:05:41chunkynutz60181694665 quits [Ping timeout: 268 seconds]
22:08:33nexussfan (nexussfan) joins
22:11:03Mateon1 quits [Remote host closed the connection]
22:11:17Mateon1 joins
22:13:46chunkynutz60181694665 joins
22:15:48rewby (rewby) joins
22:15:48@ChanServ sets mode: +o rewby
22:28:03kline quits [Ping timeout: 252 seconds]
22:28:15kline (kline) joins
22:30:01DogsRNice quits [Read error: Connection reset by peer]
22:35:17Radzig quits [Ping timeout: 268 seconds]
22:36:52BornOn420 quits [Remote host closed the connection]
22:38:14BornOn420 (BornOn420) joins
23:02:39Radzig joins
23:05:12HP_Archivist (HP_Archivist) joins
23:05:26<pokechu22>anarcat: additional job started: https://transfer.archivete.am/lCZHE/noblogs.org_all_subdomains_sep_2026_no_outlinks.txt - combines a bunch of different lists including everything from CDX (which did introduce a bunch of junky subdomains but they should fail fast). That one will skip anything not on noblogs.org (using an ignore, not --no-offsite, so ignored URLs will be in
23:05:28<pokechu22>the job log.)
23:05:28<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/lCZHE/noblogs.org_all_subdomains_sep_2026_no_outlinks.txt
23:05:42BornOn420 quits [Remote host closed the connection]
23:06:44AlsoHP_Archivist quits [Ping timeout: 268 seconds]
23:07:12cascode_ joins
23:08:01kline quits [Ping timeout: 252 seconds]
23:08:06BornOn420 (BornOn420) joins
23:09:11kline (kline) joins
23:09:51@rewby quits [Ping timeout: 252 seconds]
23:10:26cascode quits [Ping timeout: 268 seconds]
23:11:46cascode_ quits [Read error: Connection reset by peer]
23:12:00cascode joins
23:32:36nine quits [Client Quit]
23:32:48nine joins
23:32:51etnguyen03 (etnguyen03) joins
23:36:33rewby (rewby) joins
23:36:33@ChanServ sets mode: +o rewby
23:44:58@rewby quits [Ping timeout: 268 seconds]
23:45:48rewby (rewby) joins
23:45:48@ChanServ sets mode: +o rewby

HomeSearchPrevious dayNext day