| 00:17:12 | | ericgallager quits [Ping timeout: 268 seconds] |
| 00:27:11 | | khaoohs quits [Read error: Connection reset by peer] |
| 00:27:11 | | khaoohs_ joins |
| 00:29:36 | | khaoohs__ joins |
| 00:29:37 | | khaoohs_ quits [Read error: Connection reset by peer] |
| 00:30:14 | | etnguyen03 (etnguyen03) joins |
| 00:34:31 | | Hackerpcs quits [Remote host closed the connection] |
| 00:35:33 | | Hackerpcs (Hackerpcs) joins |
| 01:14:33 | | devkev5 quits [Ping timeout: 268 seconds] |
| 01:14:44 | | devkev5 joins |
| 01:17:07 | | TheEnbyperor_ quits [Read error: Connection reset by peer] |
| 01:27:32 | | linuxgemini quits [Quit: Ping timeout (120 seconds)] |
| 01:27:45 | | linuxgemini (linuxgemini) joins |
| 01:37:14 | | DogsRNice quits [Read error: Connection reset by peer] |
| 01:39:18 | | lev (lev) joins |
| 01:54:57 | | nexussfan quits [Remote host closed the connection] |
| 02:23:37 | | nexussfan (nexussfan) joins |
| 03:06:30 | | unknownsrc quits [Ping timeout: 255 seconds] |
| 03:08:28 | | unknownsrc (unknownsrc) joins |
| 03:11:06 | | etnguyen03 quits [Ping timeout: 268 seconds] |
| 03:19:40 | | etnguyen03 (etnguyen03) joins |
| 03:26:15 | | ericgallager joins |
| 03:32:23 | | etnguyen03 quits [Remote host closed the connection] |
| 03:52:01 | | pedantic-darwin66 quits [Ping timeout: 252 seconds] |
| 04:03:17 | | Wuzado quits [Read error: Connection reset by peer] |
| 04:03:52 | | Wuzado joins |
| 04:36:15 | | ericgallager quits [Client Quit] |
| 04:47:10 | | nexussfan quits [Remote host closed the connection] |
| 04:48:15 | | nexussfan (nexussfan) joins |
| 04:49:05 | | nexussfan quits [Remote host closed the connection] |
| 04:50:12 | | Umbire quits [Quit: [Resetting...]] |
| 05:07:07 | | sirio joins |
| 05:10:42 | | ericgallager joins |
| 05:14:57 | | Umbire (Umbire) joins |
| 05:17:02 | | atphoenix_ (atphoenix) joins |
| 05:19:22 | | atphoenix__ quits [Ping timeout: 268 seconds] |
| 05:57:08 | | Wohlstand1 (Wohlstand) joins |
| 05:59:30 | | Wohlstand1 is now known as Wohlstand |
| 06:16:27 | | Wohlstand quits [Client Quit] |
| 07:05:18 | | BornOn420 quits [Remote host closed the connection] |
| 07:07:37 | | BornOn420 (BornOn420) joins |
| 07:16:38 | | symmret quits [Quit: symmret] |
| 07:35:48 | <twiswist_> | This isn't something that can be scraped, but is Kiliaro (a photo cloud) shutting down? I heard my mother's telecom carrier told her that they will stop offering Kiliaro after August 30th, but it's still there. Looking into it, there is now a recent (2026-08-19, unsure if post date or update date) article with a full backup tool: https://kiliaro.zendesk.com/hc/en-us/articles/14756991589650-Exporting-or-Downloading-All-Your-Data-from-Kil |
| 07:35:48 | <twiswist_> | iaro |
| 07:36:03 | <twiswist_> | (apologies for the cut-off url) |
| 07:37:17 | <twiswist_> | It's also linked from www.kiliaro.com at the bottom, but wbm is down, so I can't see if the button was there before |
| 07:38:22 | <twiswist_> | It's either renaming to or being acquired by Pixfam: https://www.pixfam.com/press-releases/29876EFEA5181ADF-kiliaro-group-ab-blir-pixfam-ab https://mb.cision.com/Public/22089/4240232/8951a4659573fefd.pdf |
| 07:41:19 | <twiswist_> | I'll be afk for 12+ hours |
| 08:12:58 | | d02 joins |
| 08:52:45 | | sirio quits [Quit: Ooops, wrong browser tab.] |
| 09:00:25 | | Bleo182600722719623455222011079 quits [Quit: The Lounge - https://thelounge.chat] |
| 09:03:13 | | Bleo182600722719623455222011079 joins |
| 09:46:34 | <c3manu> | pokechu22: what's the current state of affairs for noblogs.org btw? i haven't been following that closely, but i think the run 2025-10 is incomplete due to a bad ignore, and the list you queued recently is significantly shorter. |
| 09:46:57 | <c3manu> | pokechu22: does that mean we actually never ran the noblogs.org subdomains in their entirety? |
| 09:51:43 | | Umbire quits [Ping timeout: 252 seconds] |
| 10:05:00 | | nune quits [Remote host closed the connection] |
| 10:05:02 | | nune joins |
| 10:33:52 | | jodizzle quits [Ping timeout: 268 seconds] |
| 11:23:14 | | fangfufu quits [Quit: ZNC 1.9.1+deb2+b3 - https://znc.in] |
| 11:27:04 | | fangfufu joins |
| 12:11:37 | | jodizzle (jodizzle) joins |
| 12:29:38 | | h|ignition quits [Ping timeout: 272 seconds] |
| 12:34:31 | | scotrod2 quits [Quit: Ping timeout (120 seconds)] |
| 12:34:39 | | fuzzy80211 quits [Read error: Connection reset by peer] |
| 12:34:41 | | scotrod2 joins |
| 12:34:50 | | Green11 quits [Quit: Ping timeout (120 seconds)] |
| 12:34:55 | | fuzzy80211 (fuzzy80211) joins |
| 12:35:01 | | Green11 joins |
| 12:38:02 | | thewinwin878 joins |
| 12:39:42 | | h|ignition (h) joins |
| 12:39:59 | | adamus1red quits [Quit: SigTerm] |
| 12:40:46 | | adamus1red (adamus1red) joins |
| 12:41:36 | | thewinwin87 quits [Ping timeout: 255 seconds] |
| 12:41:42 | | thewinwin878 is now known as thewinwin87 |
| 12:56:08 | | fuzzy80211 quits [Read error: Connection reset by peer] |
| 12:56:33 | | fuzzy80211 (fuzzy80211) joins |
| 13:20:04 | | pedantic-darwin66 joins |
| 13:23:05 | | etnguyen03 (etnguyen03) joins |
| 13:23:12 | | Umbire joins |
| 13:44:13 | | etnguyen03 quits [Client Quit] |
| 13:48:12 | | TheEnbyperor quits [Ping timeout: 255 seconds] |
| 13:51:46 | | TheEnbyperor_ joins |
| 13:54:27 | | etnguyen03 (etnguyen03) joins |
| 14:14:43 | | MrMcNugg1 (MrMcNuggets) joins |
| 14:18:20 | | khaoohs__ quits [Read error: Connection reset by peer] |
| 14:18:20 | | MrMcNuggets quits [Ping timeout: 268 seconds] |
| 14:21:12 | | khaoohs joins |
| 14:22:17 | | khaoohs quits [Remote host closed the connection] |
| 14:22:53 | | khaoohs joins |
| 14:23:47 | | khaoohs quits [Remote host closed the connection] |
| 14:24:20 | | khaoohs joins |
| 14:24:38 | | pedantic-darwin66 quits [Client Quit] |
| 14:24:50 | | pedantic-darwin66 joins |
| 14:29:02 | | nexussfan (nexussfan) joins |
| 14:33:56 | | thewinwin870 joins |
| 14:35:57 | | McAfee leaves [Disconnected: Hibernating too long] |
| 14:38:04 | | thewinwin87 quits [Ping timeout: 268 seconds] |
| 14:38:08 | | thewinwin870 is now known as thewinwin87 |
| 14:38:41 | | BornOn420 quits [Remote host closed the connection] |
| 14:38:41 | | unknownsrc quits [Ping timeout: 268 seconds] |
| 14:39:29 | | BornOn420 (BornOn420) joins |
| 14:42:13 | | unknownsrc (unknownsrc) joins |
| 15:01:53 | | simon816 quits [Remote host closed the connection] |
| 15:01:54 | | unknownsrc quits [Client Quit] |
| 15:05:42 | | unknownsrc (unknownsrc) joins |
| 15:10:42 | | simon816 (simon816) joins |
| 15:11:34 | | McAfee joins |
| 15:11:50 | | d02 quits [Quit: Leaving] |
| 15:15:07 | | beastbg8 quits [Read error: Connection reset by peer] |
| 15:15:29 | | beastbg8 joins |
| 15:26:25 | | TheEnbyperor_ quits [Read error: Connection reset by peer] |
| 15:29:15 | | @rewby quits [Ping timeout: 268 seconds] |
| 15:31:06 | | TheEnbyperor joins |
| 15:35:17 | | khaoohs quits [Ping timeout: 252 seconds] |
| 15:40:33 | | khaoohs_ joins |
| 15:44:16 | | rewby (rewby) joins |
| 15:44:16 | | @ChanServ sets mode: +o rewby |
| 15:45:32 | | khaoohs joins |
| 15:45:48 | | khaoohs_ quits [Read error: Connection reset by peer] |
| 15:55:54 | | kansei quits [Quit: ZNC 1.10.2 - https://znc.in] |
| 15:56:54 | | ThreeHM quits [Ping timeout: 255 seconds] |
| 15:58:11 | | kansei (kansei) joins |
| 15:58:21 | | murmur is now authenticated as murmur |
| 15:58:49 | | ThreeHM (ThreeHeadedMonkey) joins |
| 15:59:26 | | murmur quits [] |
| 15:59:33 | | murmur (murmur) joins |
| 15:59:42 | | thewinwin871 joins |
| 16:01:35 | | @rewby quits [Read error: Connection reset by peer] |
| 16:01:40 | | rewby (rewby) joins |
| 16:01:40 | | @ChanServ sets mode: +o rewby |
| 16:01:40 | | murmur quits [Client Quit] |
| 16:01:43 | | murmur joins |
| 16:03:12 | | thewinwin87 quits [Ping timeout: 255 seconds] |
| 16:03:19 | | thewinwin871 is now known as thewinwin87 |
| 16:03:35 | | murmur quits [Client Quit] |
| 16:03:39 | | murmur (murmur) joins |
| 16:06:14 | | Umbire is now known as RJHacker25366 |
| 16:06:15 | | Umbire (Umbire) joins |
| 16:07:40 | | Umbire quits [Remote host closed the connection] |
| 16:07:45 | | murmur quits [Changing host] |
| 16:07:45 | | murmur (murmur) joins |
| 16:08:00 | | Umbire joins |
| 16:09:30 | | RJHacker25366 quits [Ping timeout: 255 seconds] |
| 16:15:25 | | Umbire is now known as RJHacker99517 |
| 16:15:26 | | RJHacker99517 quits [Killed (palermo.hackint.org (Nickname regained by services))] |
| 16:15:27 | | Umbire (Umbire) joins |
| 16:16:21 | | Umbire is now authenticated as * |
| 16:16:21 | | Umbire is now known as RJHacker26295 |
| 16:16:21 | | Umbire (Umbire) joins |
| 16:20:26 | | RJHacker26295 quits [Ping timeout: 268 seconds] |
| 16:28:27 | | sirio joins |
| 16:46:35 | | saumon (saumon) joins |
| 16:48:59 | | etnguyen03 quits [Ping timeout: 252 seconds] |
| 16:58:40 | | Umbire quits [Ping timeout: 268 seconds] |
| 16:58:52 | | d02 joins |
| 17:08:28 | <pokechu22> | c3manu: The remaining subdomains from the job with the bad ignore were cleaned up in https://archive.fart.website/archivebot/viewer/job/20260423191907f30pz |
| 17:10:15 | | invadeuz joins |
| 17:22:43 | | d02 quits [Client Quit] |
| 17:23:49 | | Matthww5 quits [Ping timeout: 252 seconds] |
| 17:32:11 | | NeonGlitch (NeonGlitch) joins |
| 17:35:37 | | DogsRNice joins |
| 17:37:38 | | Matthww5 joins |
| 17:39:21 | | invadeuz is now authenticated as invadeuz |
| 17:42:49 | | thewinwin872 joins |
| 17:46:21 | | invadeuze (invadeuz) joins |
| 17:46:33 | | thewinwin87 quits [Ping timeout: 252 seconds] |
| 17:46:38 | | thewinwin872 is now known as thewinwin87 |
| 17:46:43 | | Green11 quits [Quit: Ping timeout (120 seconds)] |
| 17:46:55 | | Green11 joins |
| 17:47:57 | <invadeuz> | hi |
| 17:48:57 | <justauser> | Hello? |
| 17:51:01 | <invadeuz> | we wish to do a bulk download of noblogs sites and people pointed us here. I know a little bit of InternetArchive but not enough to know how it would works |
| 17:52:06 | | xkey quits [Quit: WeeChat 4.10.0] |
| 17:52:59 | | xkey (xkey) joins |
| 17:54:14 | <justauser> | If you want a copy for yourself, start from https://archive.fart.website/archivebot/viewer/?q=noblogs.org |
| 17:55:32 | <justauser> | https://archive.fart.website/archivebot/viewer/job/20250920010916aenij is the largest pile of data and https://archive.fart.website/archivebot/viewer/job/20260826175807648yx is the latest. |
| 17:55:53 | <justauser> | Both of the pages are lists of links to IA. |
| 17:56:35 | | etnguyen03 (etnguyen03) joins |
| 17:56:37 | <pokechu22> | https://archive.fart.website/archivebot/viewer/job/2025103003442267q6q + https://archive.fart.website/archivebot/viewer/job/20260423191907f30pz should be more complete than https://archive.fart.website/archivebot/viewer/job/20250920010916aenij I think - 20250920010916aenij is incomplete due to a pipeline restart. (But it's entirely possible that both contain unique data) |
| 17:57:54 | | atphoenix_ quits [Read error: Connection reset by peer] |
| 17:58:23 | | atphoenix_ (atphoenix) joins |
| 17:58:28 | <invadeuz> | thanks for the links ! do we have an idea of the quantity of data we are talking about ? |
| 17:58:44 | <justauser> | Tens of GBs, I think. |
| 17:59:05 | <pokechu22> | Those noblogs jobs add up to several terabytes, but they also include outlinks |
| 17:59:11 | | Green11 quits [Client Quit] |
| 17:59:52 | <pokechu22> | https://archive.fart.website/archivebot/viewer/job/20260826175807648yx is significantly smaller (~150 GB), containing posts from all blogs linked from https://noblogs.org/activity/ but not necessarily other stuff |
| 18:00:59 | <pokechu22> | and correction, https://archive.fart.website/archivebot/viewer/job/20250920010916aenij was incomplete due to crashing due to https://github.com/ArchiveTeam/wpull/issues/435, not due to a pipeline restart (there was another one with the same list that died early on due to a restart though) |
| 18:02:04 | <invadeuz> | ok, we were told of roughly 2To by people close to IA, so it checks out. |
| 18:03:08 | <invadeuz> | in what format are the archives ? do you think it's easy with them to extract static sites and republish them ? |
| 18:03:21 | | leo60228_ (leo60228) joins |
| 18:04:21 | <klea> | They are stored as WARCs, a container format for storing the raw HTTP(s) transactions ArchiveBot did with those sites. |
| 18:04:33 | | sirio quits [Client Quit] |
| 18:04:42 | | leo60228 quits [Ping timeout: 255 seconds] |
| 18:04:42 | | leo60228_ is now known as leo60228 |
| 18:05:38 | <justauser> | Extracting them to static sites can be sort of done using warcat or similar tools. |
| 18:06:29 | <justauser> | However, it requires that your server considers the files static even if their names end in .php, contain query strings and so on. |
| 18:06:55 | <justauser> | Loading files into pywb or a similar WARC replayer might be easier. |
| 18:07:31 | <klea> | Yeah, and doing that also allows you to later give those warcs to other people, instead of having to give the extracted files out. |
| 18:08:16 | <pokechu22> | noblogs is wordpress at least so treating them as static sites is comparatively easy (most URLs are fairly clean) |
| 18:09:25 | <invadeuz> | justauser "pywb or a similar WARC replayer" -> host them in the same way than waybackmachine ? |
| 18:09:37 | <justauser> | Yes. |
| 18:09:56 | | nine quits [Quit: See ya!] |
| 18:10:08 | | nine joins |
| 18:10:24 | <pokechu22> | Compare https://archive.inventati.org/ |
| 18:12:20 | <invadeuz> | ok I understand, we were thinking of static sites for ease of use and ease of hosting / replicating but it's interesting indeed |
| 18:12:55 | <pokechu22> | Plenty of options once you actually have a copy of the data :) |
| 18:16:28 | | lev quits [Quit: lev] |
| 18:17:27 | <invadeuz> | thanks for your kind answers, we will see. first I have to empty a disk before trying to retrieve all of that ^^ |
| 18:19:27 | <klea> | We could make Deathwatch a index, and then we can have it be small. |
| 18:21:18 | <invadeuz> | klea I didn't get that, dunno what is Deathwatch. But if you have a way to create an archive that is smaller by stripping external links for example, I'm interested |
| 18:21:29 | <justauser> | A very big wiki page. |
| 18:21:49 | <justauser> | I don't think we have good tools for selectively stripping WARCs. |
| 18:22:55 | <invadeuz> | klea what was your idea to "have it be small" then ? |
| 18:23:51 | <justauser> | It's related to the wiki page, not to WARCs. |
| 18:24:13 | <justauser> | Keeping WARCs small is not much of a problem for us, fortunately or not. |
| 18:25:33 | <klea> | IIRC individual WARCs are about 5G? |
| 18:25:55 | | nexussfan quits [Remote host closed the connection] |
| 18:27:21 | <invadeuz> | okay :) |
| 18:30:00 | | msfjarvis quits [Quit: Lurker 2.2.1 (the truth is out there) https://lurker.chat] |
| 18:30:16 | <pokechu22> | Yeah, archivebot creates a new warc at the start of a record when the existing one is at least 5G. (Meaning the warc can be much larger than 5G if it e.g. downloaded 4.9G already and then downloads a 2G file, producing a 6.9G warc... but for small files it'll usually be around 5G) |
| 18:30:58 | | msfjarvis joins |
| 18:32:10 | <klea> | hmm, downloaded 4.9G already, and then downloads a 500G file, not producing a warc, and annoying JAA because they have to manually fix the pipeline. |
| 18:33:04 | <pokechu22> | I think the 4.9G warc would survive in that case, but would still break everything else |
| 18:36:52 | <@JAA> | Correct |
| 18:37:13 | <@JAA> | There is at least one WARC of over 500 GB in the AB collection somewhere. |
| 18:37:30 | <@JAA> | (Probably only the one) |
| 18:52:27 | | Webuser123185 joins |
| 18:52:53 | <Webuser123185> | Hi! I'm trying to find my own old Hyves profile in the ArchiveTeam archive. My name is Lorain Steens, born in 1989, and I lived in Ewijk (Beuningen area), Netherlands. Unfortunately, I don't remember my old Hyves username. I already have the username/WARC index. Is there any way to search the archived Hyves data by real name or location to identify |
| 18:52:53 | <Webuser123185> | my profile? I'd especially love to recover my old photos if they were archived. Thanks! |
| 18:53:30 | | Webuser123185 quits [Client Quit] |
| 18:54:13 | | Webuser710941 joins |
| 18:54:31 | | Webuser710941 quits [Client Quit] |
| 18:55:35 | | Webuser810026 joins |
| 18:55:38 | <Webuser810026> | Hi! I'm trying to recover my own old Hyves profile from the ArchiveTeam archive. I don't remember my Hyves username, but I do know my real name and location from that time. I already downloaded the username/WARC index. Is there any way to search the archived profile data by real name instead of username? I'm mainly hoping to recover my old photos |
| 18:56:18 | | Webuser810026 quits [Client Quit] |
| 19:04:48 | <@JAA> | arkiver: https://core.trac.wordpress.org/ and https://plugins.trac.wordpress.org/ have a Hashcash challenge. |
| 19:20:11 | | sirio joins |
| 19:29:36 | | Wohlstand (Wohlstand) joins |
| 19:30:24 | <invadeuz> | hi, it's me again for the archiving of NoBlogs : is https://archive.fart.website/archivebot/viewer/?q=noblogs.org the only list of NoBlogs blogs who are archived, or is there a broader list ? |
| 19:31:45 | <pokechu22> | That's the list of ones that had individual jobs. There were also !a < list jobs such as https://transfer.archivete.am/ZB8vs/noblogs.org_subdomains_redo_2.txt where every subdomain in that list was recursively crawled |
| 19:31:45 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/ZB8vs/noblogs.org_subdomains_redo_2.txt |
| 19:38:15 | <invadeuz> | thanks a lot ! |
| 19:43:29 | <h2ibot> | Pokechu22 edited List of websites excluded from the Wayback Machine (+22, + https://nltimes.nl/): https://wiki.archiveteam.org/?diff=63862&oldid=63856 |
| 19:44:00 | | BearFortress_ quits [] |
| 19:47:33 | | spelledwrongdotuk quits [Ping timeout: 252 seconds] |
| 19:52:31 | | Umbire joins |
| 20:11:03 | | nexussfan (nexussfan) joins |
| 20:17:20 | | spelledwrongdotuk joins |
| 20:21:59 | | nexussfan quits [Remote host closed the connection] |
| 20:31:27 | | BearFortress joins |
| 20:38:37 | | spelledwrongdotuk quits [Ping timeout: 255 seconds] |
| 20:46:08 | | nine quits [Client Quit] |
| 20:46:22 | | nine joins |
| 21:01:36 | | khaoohs quits [Read error: Connection reset by peer] |
| 21:01:59 | | khaoohs joins |
| 21:04:51 | | etnguyen03 quits [Quit: Konversation terminated!] |
| 21:07:52 | | etnguyen03 (etnguyen03) joins |
| 21:16:48 | | thewinwin879 joins |
| 21:20:27 | | thewinwin87 quits [Ping timeout: 255 seconds] |
| 21:20:29 | | thewinwin879 is now known as thewinwin87 |
| 21:24:46 | | nexussfan (nexussfan) joins |
| 21:25:42 | | sirio quits [Client Quit] |
| 21:39:29 | | Wohlstand quits [Client Quit] |
| 21:53:03 | | AlsoHP_Archivist (HP_Archivist) joins |
| 21:54:27 | | Umbire quits [Read error: Connection reset by peer] |
| 21:54:48 | | Umbire joins |
| 21:56:03 | <invadeuz> | Hi, I just got a list of *all* of the domains of noblogs from someone who got it directly from A/I, there is 22,386 entries while the list you gave me have 14,057 entries. could it means that there could be around 8000 sites not archived by you ? |
| 21:56:03 | <invadeuz> | The list if you are interested : https://cryptpad.fr/file/#/2/file/HDRmnlQvswDDucjwX0whAC4s/ |
| 21:56:15 | | HP_Archivist quits [Ping timeout: 252 seconds] |
| 22:05:16 | <pokechu22> | invadeuz: https://transfer.archivete.am/ZB8vs/noblogs.org_subdomains_redo_2.txt filters out domains that redirected to wp-signup on previous attempts. I'll cross-reference your list with some of the other ones, one minute |
| 22:05:17 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/ZB8vs/noblogs.org_subdomains_redo_2.txt |
| 22:19:58 | | etnguyen03 quits [Ping timeout: 268 seconds] |
| 22:23:24 | | etnguyen03 (etnguyen03) joins |
| 22:33:21 | | entrox quits [Ping timeout: 255 seconds] |
| 22:35:19 | | etnguyen03 quits [Read error: Connection reset by peer] |
| 22:35:31 | | etnguyen03 (etnguyen03) joins |
| 22:37:07 | <invadeuz> | pokechu22 du you have a list of only wp-signup redirect / sites in error by any chance ? |
| 22:37:54 | <invadeuz> | so we can filter out those ourselves |
| 22:38:03 | <pokechu22> | Not in a clean form, only based on having filtered it out of _redo_2.txt. I'm trying to cross-reference things but my organization is a bit of a mess |
| 22:38:37 | <pokechu22> | I think it was based on https://archive.org/download/archiveteam_archivebot_go_20250918234850_785f76c6/urls-transfer.archivete.am-noblogs.org_subdomains.txt-inf-20250918-033008-3knbx-aborted-wpull.log.gz but there were several lists |
| 22:39:19 | | brk275 joins |
| 22:43:38 | <pokechu22> | OK, no, I cross-referenced against all https://transfer.archivete.am/xaQCk/noblogs.org_subdomains.txt https://transfer.archivete.am/Rjy3n/noblogs.org_subdomains_redo.txt https://transfer.archivete.am/8Ax9Z/noblogs.org_subdomains_redo_2.txt https://transfer.archivete.am/12RbHH/noblogs.org_new_subdomains_aug_2026_redo.txt and there are another around 8k subdomains that didn't |
| 22:43:40 | <pokechu22> | exist before and only have some coverage from one of the discovery crawls or no coverage at all (e.g. https://antifainfoharburg.noblogs.org/ https://musiciansinswitzerlandagainstapartheid.noblogs.org/ https://testbed.noblogs.org/). I think most of these are just new? |
| 22:43:40 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/xaQCk/noblogs.org_subdomains.txt https://transfer.archivete.am/inline/Rjy3n/noblogs.org_subdomains_redo.txt https://transfer.archivete.am/inline/8Ax9Z/noblogs.org_subdomains_redo_2.txt https://transfer.archivete.am/inline/12RbHH/noblogs.org_new_subdomains_aug_2026_redo.txt |
| 22:45:25 | <pokechu22> | https://transfer.archivete.am/12RbHH/noblogs.org_new_subdomains_aug_2026_redo.txt was started before the noblogs.org/activity job finished |
| 22:45:25 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/12RbHH/noblogs.org_new_subdomains_aug_2026_redo.txt |
| 22:47:06 | | mgrytbak quits [Ping timeout: 268 seconds] |
| 22:49:46 | <invadeuz> | wow, would it be possible to run another crawl with the new ones then ? |
| 22:50:23 | | mgrytbak joins |
| 22:51:15 | <invadeuz> | I have no idea about the amount of work it could represent, seems a lot of sites to me |
| 22:52:46 | <pokechu22> | It shouldn't be too bad to do another !a < list, just sorting through things further |
| 22:58:01 | <pokechu22> | The main difficulty was that a few subdomains produced long nonsense URLs like |
| 22:58:03 | <pokechu22> | https://fa86.noblogs.org/www.monde-libertaire.fr/venez-nombreux-lundi-6-janvier-2014-a-14h-accompagner-cerasela-a-la-prefecture/Les-parents-soutiennent-la-famille-russe-menacee-d-expulsion-1867616/Les-parents-soutiennent-la-famille-russe-menacee-d-expulsion-1867616/Les-parents-soutiennent-la-famille-russe-menacee-d-expulsion-1867616/?paged=105 (which is the same as |
| 22:58:05 | <pokechu22> | https://fa86.noblogs.org/?paged=105 ). That probably won't be an issue for these sites because they're probably small if they were never discovered before |
| 22:59:04 | <@JAA> | List rehosted for posterity because CryptPad is a pain: https://transfer.archivete.am/OiDp3/noblogs_2026-08-27_domains.txt |
| 22:59:06 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/OiDp3/noblogs_2026-08-27_domains.txt |
| 23:00:40 | <pokechu22> | Huh, https://-.-.-.-.noblogs.org actually has content despite HSTS rejecting it |
| 23:00:52 | <pokechu22> | ah, no, it's a 404 |
| 23:00:53 | | Webuser876742 joins |
| 23:01:05 | <Webuser876742> | Is there any hope for saving deviantart" |
| 23:01:32 | <pokechu22> | but https://o.r.a.-officina-radiostream-autogestita.noblogs.org seems to work? |
| 23:01:39 | <pokechu22> | Webuser876742: What's going on with deviantart? |
| 23:04:21 | <Webuser876742> | pokechu22 it's not closing down, but it's been in decline for a long time. More recently, over the last few years, it's been very "enshittified". One big thing that happened earlier this year was they set a limit of only ten downloads per two weeks (previously it required an account for downloading full quality, but you could still download as much |
| 23:04:21 | <Webuser876742> | as you want) |
| 23:05:59 | <invadeuz> | pokechu22 how do you obtain the data ? on firefox it doesn't seems to work ^^ |
| 23:06:00 | <pokechu22> | Looks like some of this is documented at https://wiki.archiveteam.org/index.php/DeviantArt#Download_restriction - there was an archiveteam project to save some data, but not images specifically |
| 23:06:29 | <pokechu22> | invadeuz: `curl -k https://o.r.a.-officina-radiostream-autogestita.noblogs.org`. Firefox doesn't let you bypass HSTS at all. (Archivebot ignores it, though, so it'll be able to save those fine.) |
| 23:06:48 | <Webuser876742> | pokechu22 I am aware, I'm wondering if anyone has found any workarounds or ideas. |
| 23:07:45 | <Webuser876742> | Also I should note that having a paid account doesn't even lift the limit, it merely increases it to be a few times higher. Total cash grab. |
| 23:11:30 | <twiswist_> | Umm just make hundreds of deviantart accounts and one application each :^) What's the problem |
| 23:12:02 | <twiswist_> | I made less than a handful before noticing that gallery-dl doesn't know how to rotate accounts |
| 23:12:22 | <Webuser876742> | twiswist_ Would require approximately ~90,000,000 deviantart accounts |
| 23:12:43 | <twiswist_> | (And wastes a weekly download when it checks a deviation that's already in the download-archive) |
| 23:12:51 | <twiswist_> | gallery-dl-- |
| 23:12:51 | <eggdrop> | [karma] 'gallery-dl' now has 0 karma! |
| 23:13:00 | <pokechu22> | Possibly helpful: https://e926.net/wiki_pages/31895#deviantart (this page is SFW) |
| 23:13:06 | <pokechu22> | Apparently wix bought deviantart? |
| 23:13:16 | <twiswist_> | Yes, it uses wix cdn |
| 23:13:50 | <pokechu22> | New noblogs list running in archivebot, 8136 entries: https://transfer.archivete.am/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt |
| 23:13:51 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/mA5Go/noblogs.org_additional_subdomains_sep_2026.txt |
| 23:13:53 | <pokechu22> | wix-- |
| 23:13:54 | <eggdrop> | [karma] 'wix' now has -54 karma! |
| 23:14:13 | | etnguyen03 quits [Ping timeout: 268 seconds] |
| 23:15:32 | <invadeuz> | amazing pokechu22 , thanks a lot ! |
| 23:16:12 | <invadeuz> | I'll have another 500 Go to download after that I guess x) |
| 23:16:57 | <invadeuz> | I'll get some sleep it's going late, have a great day / night and thanks again for your works ! |
| 23:17:00 | <pokechu22> | Eh, it should probably be smaller |
| 23:17:29 | <pokechu22> | That one will start appearing at https://archive.fart.website/archivebot/viewer/job/2o9yg when it downloads 5GB+ |
| 23:20:39 | <klea> | oh wait, you can't get the exact job date, until it gets any chunk of it uploaded to IA right? |
| 23:22:03 | <pokechu22> | Might be able to do it via ab2f, but the dashboard doesn't give it at least |
| 23:22:21 | | Webuser876742 quits [Client Quit] |
| 23:24:05 | | pepsi2024 quits [Ping timeout: 268 seconds] |
| 23:24:25 | <Umbire> | Didn't wanna be the Danny Downer, but I don't think that site's got much left to BE salvaged, especially not post-wix |
| 23:28:28 | <klea> | "started_at": "1788822766.9568012", |
| 23:28:30 | <klea> | perhaps. |
| 23:28:39 | <klea> | We'll know once one of the chunks gets on IA. |
| 23:30:58 | <@JAA> | #devianttart is the channel for DeviantArt. |
| 23:46:35 | | etnguyen03 (etnguyen03) joins |
| 23:48:16 | <Umbire> | Gonna have to forward that to whoever expressed interest originally |
| 23:59:36 | | TheEnbyperor quits [Read error: Connection reset by peer] |