| 00:04:31 | | AFly quits [Quit: 🪰] |
| 00:05:06 | | AFly (AFly) joins |
| 00:15:42 | | useretail__ quits [Ping timeout: 268 seconds] |
| 00:16:36 | | Wohlstand quits [Quit: Wohlstand] |
| 00:26:22 | <cruller|irc> | Re data.world, for now I compiled a url list to enumerate all items except column and datatable. Could someone !ao this? https://transfer.archivete.am/CR0SS/data.world_enumeration_except_column_and_datatable.txt |
| 00:26:23 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/CR0SS/data.world_enumeration_except_column_and_datatable.txt |
| 00:42:37 | <pokechu22> | (running in AB) |
| 00:47:49 | | useretail joins |
| 00:55:04 | | etnguyen03 (etnguyen03) joins |
| 01:00:47 | | Shard76 quits [Quit: Im doing something rq. Il brb] |
| 01:00:55 | | nine quits [Quit: See ya!] |
| 01:01:07 | | nine joins |
| 01:01:51 | <pokechu22> | I brought up https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Video_games#International_Computer_Game_Collection_shutdown a week ago but didn't have time to look into it. https://db.games-archive.org/ seems to be all 403s at the moment and https://db.internationale-computerspielesammlung.de/ is down, but it looks like they did add stuff to wikidata for it. Is this |
| 01:01:53 | <pokechu22> | region-locked, or actually closed? |
| 01:02:03 | <pokechu22> | (no AB job was ever started for it) |
| 01:02:39 | | Shard768 (Shard) joins |
| 01:05:04 | | nine quits [Client Quit] |
| 01:05:17 | | nine joins |
| 01:05:53 | <pokechu22> | Ah, the 403 message is "Access denied: Only European IPs are allowed. This is a measure against AI bots." but firefox has decided it doesn't like to show short 403 messages |
| 01:55:16 | | Webuser111351 quits [Quit: Ooops, wrong browser tab.] |
| 02:05:28 | | BitByBit4100 quits [Ping timeout: 268 seconds] |
| 02:06:37 | <Pendonym> | Tech Robo said No idea, I thought they were fairly high when mnbot first started, but I haven't heard much since about the odds of getting on wayback machine which is kinda upsettings |
| 02:35:34 | <h2ibot> | TripleCamera edited Codearchiver (+117, dead): https://wiki.archiveteam.org/?diff=63132&oldid=58335 |
| 02:51:04 | | etnguyen03 quits [Client Quit] |
| 02:51:31 | | Island quits [Read error: Connection reset by peer] |
| 02:53:59 | | etnguyen03 (etnguyen03) joins |
| 02:56:20 | | @Fusl quits [Quit: K-Lined] |
| 02:56:29 | | Fusl (Fusl) joins |
| 02:56:29 | | @ChanServ sets mode: +o Fusl |
| 02:58:18 | | etnguyen03 quits [Remote host closed the connection] |
| 03:15:33 | <cruller|irc> | pokechu22++ |
| 03:15:34 | <eggdrop> | [karma] 'pokechu22' now has 377 karma! |
| 03:20:08 | | Pendonym quits [Quit: Pendonym] |
| 03:23:48 | <cruller|irc> | Re data.world, it would be great if we could also retrieve detailed metadata for each item. I realized too late that there was a "filters" key. |
| 03:26:43 | <mindrecent> | cruller: I have the metadata for the datasets themselves via /api/$agent/dataset/$dataset. I was thinking about grabbing queries and all the agents the same way, if that's what you mean? |
| 03:33:55 | | thewinwin83 joins |
| 03:37:58 | | thewinwin8 quits [Ping timeout: 268 seconds] |
| 03:38:03 | | thewinwin83 is now known as thewinwin8 |
| 03:40:28 | <mindrecent> | cruller|irc: see previous |
| 03:44:33 | | IDK (IDK) joins |
| 03:48:52 | <cruller|irc> | mindrecent: Yes, that's what I mean. |
| 03:52:15 | <cruller|irc> | https://irclogs.archivete.am/archiveteam-bs/2026-07-12#l6e841a9a Ah, you've already downloaded dataset metadata. However, the number of agent is 2875125. I think it may be a little too big. |
| 03:53:03 | <cruller|irc> | Given the ETA |
| 03:56:57 | <mindrecent> | cruller|irc: agreed. I am pulling all the agents for the public datasets fwiw. |
| 04:35:05 | <cruller|irc> | mindrecent: As you may already know, ArchiveBot works with endpoints that don't require authentication. I asked an ABer to archive all the search results, excluding datatable and column. |
| 05:03:18 | | DogsRNice quits [Read error: Connection reset by peer] |
| 05:16:51 | | atphoenix_ (atphoenix) joins |
| 05:19:43 | | atphoenix quits [Ping timeout: 268 seconds] |
| 05:32:01 | <h2ibot> | Manu edited Anubis/uncategorized (+79, https://www.deutsche-digitale-bibliothek.de/…): https://wiki.archiveteam.org/?diff=63133&oldid=63074 |
| 05:46:51 | | f_ quits [Ping timeout: 268 seconds] |
| 05:47:15 | | f_ (funderscore) joins |
| 05:53:38 | | f_ quits [Ping timeout: 268 seconds] |
| 05:54:58 | | f_ (funderscore) joins |
| 06:00:45 | | rohvani quits [Quit: The Lounge - https://thelounge.chat] |
| 06:04:18 | | rohvani joins |
| 06:51:59 | | Erebus quits [Ping timeout: 248 seconds] |
| 06:56:14 | | Erebus (Erebus) joins |
| 07:03:30 | | IDK quits [Quit: Connection closed for inactivity] |
| 07:17:35 | | Kabaya quits [Ping timeout: 248 seconds] |
| 07:18:39 | | cyanbox quits [Read error: Connection reset by peer] |
| 07:20:11 | | cyanbox joins |
| 07:30:05 | | Kabaya joins |
| 07:43:46 | <that_lurker> | https://old.reddit.com/r/DataHoarder/s/NegsgrHbsW |
| 07:43:51 | <that_lurker> | ”We have finally released the largest available Minecraft world, totalling 15 TB (13.7 TiB) of highly compressed world data of the Minecraft server 2b2t.” |
| 07:51:52 | | Pendonym joins |
| 07:51:53 | | Pendonym is now known as RJHacker55340 |
| 07:53:18 | | RJHacker55340 is now authenticated as Pendonym |
| 07:53:18 | | RJHacker55340 is now known as Pendonym |
| 08:01:21 | | Sidpatchy quits [Quit: Ping timeout (120 seconds)] |
| 08:01:40 | | Sidpatchy (Sidpatchy) joins |
| 08:08:20 | | thewinwin82 joins |
| 08:12:23 | | thewinwin8 quits [Ping timeout: 268 seconds] |
| 08:12:27 | | thewinwin82 is now known as thewinwin8 |
| 08:25:20 | | bladem quits [Ping timeout: 268 seconds] |
| 08:48:22 | <cruller|irc> | https://data.world/api/search?type=file&size=100&filters=updated=2019-07-24 38,944 files in one day? |
| 08:51:14 | | error quits [Remote host closed the connection] |
| 08:51:58 | | error (disinfecttm) joins |
| 09:04:28 | <@arkiver> | cruller|irc: hmm almost forgot, yeah this one is shutting down this month i believe |
| 09:04:59 | <@arkiver> | cruller|irc: thank you for the research! |
| 09:05:08 | <@arkiver> | i see deadline is the 13th, hmm |
| 09:08:35 | <cruller|irc> | Now I've extracted almost all pagination of the search results: https://transfer.archivete.am/14h5fR/data.world_enumeration_except_column_and_datatable_pagination.txt |
| 09:08:36 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/14h5fR/data.world_enumeration_except_column_and_datatable_pagination.txt |
| 09:10:31 | | Webuser404789 joins |
| 09:10:48 | <@arkiver> | cruller|irc: working on something, will keep you updated |
| 09:11:03 | <@arkiver> | in my head data.world was shutting down end of this month :( |
| 09:11:46 | <cruller|irc> | arkiver: For now, could you !ao < this list? |
| 09:12:09 | <@arkiver> | cruller|irc: running yes |
| 09:12:34 | <@arkiver> | imer: we have an emergency project coming up for data.world, lost track of the deadline :/. if you happen to be around, can we have a target for |
| 09:12:38 | <@arkiver> | archiveteam_dataworld_ |
| 09:12:41 | <@arkiver> | dataworld_ |
| 09:12:46 | <@arkiver> | Archive Team data.world: |
| 09:15:31 | <cruller|irc> | Also, the following five queries appear to require further filtering. I will take care of that: type=file&size=100&filters=updated=2019-07-24 type=file&size=100&filters=updated=2019-09-11 type=file&size=100&filters=updated=2021-09-25 type=file&size=100&filters=updated=2019-10-15 type=file&size=100&filters=updated=2021-08-26 |
| 09:16:31 | <@arkiver> | i wonder if this search API lists both public and private stuff |
| 09:17:40 | <@arkiver> | not much may be public |
| 09:17:49 | <@arkiver> | we'll scoop in as much as possible while they are online |
| 09:18:05 | <cruller|irc> | I don't have an account so I don't know, but mindrecent should know. |
| 09:22:09 | | Webuser404789 quits [Client Quit] |
| 09:34:00 | <skankhunt42> | is that for archivebot or DPoS? |
| 09:35:13 | <@arkiver> | skankhunt42: the URLs list cruller|irc posted is being archived by ArchiveBot (AB) in case the DPoS project does not come in time |
| 09:37:10 | <skankhunt42> | understood. could you ping me if it becomes DPoS? I have some spare resources |
| 09:37:52 | <@arkiver> | skankhunt42: i'd recommend watching #archiveteam , it'll be posted there |
| 09:37:58 | <@arkiver> | but yes in this case i can ping you if i don't forget |
| 09:38:15 | <skankhunt42> | oh lol, wasn't even in that channel, thx :D |
| 09:38:35 | <@arkiver> | !remindme 1s test |
| 09:38:36 | <eggdrop> | [remind] ok, i'll remind you at 2026-07-13T09:38:36Z |
| 09:38:38 | <eggdrop> | [remind] arkiver: test |
| 09:38:47 | <@arkiver> | !remindme 1s test++ |
| 09:38:48 | <eggdrop> | [remind] ok, i'll remind you at 2026-07-13T09:38:48Z |
| 09:38:49 | <eggdrop> | [karma] '!remindme 1s test' now has 1 karma! |
| 09:38:50 | <eggdrop> | [remind] arkiver: test++ |
| 09:39:02 | <@arkiver> | had to check |
| 09:39:03 | <skankhunt42> | no karma farming |
| 09:39:04 | <skankhunt42> | :( |
| 09:52:54 | | ThreeHM quits [Ping timeout: 268 seconds] |
| 09:55:04 | <cruller|irc> | I wpulled all the URLs found in the WBM CDX file. I haven't been able to do a detailed analysis, but I'm sharing the resulting CDXJ file. |
| 09:55:10 | <cruller|irc> | https://transfer.archivete.am/EyXsQ/data.world_api_found_from_wbmcdx.cdxj |
| 09:55:11 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/EyXsQ/data.world_api_found_from_wbmcdx.cdxj |
| 10:10:07 | | skankhunt424 (skankhunt42) joins |
| 10:12:38 | | skankhunt42 quits [Ping timeout: 268 seconds] |
| 10:17:57 | | skankhunt424 is now known as skankhunt42 |
| 10:18:17 | | skankhunt42 quits [Client Quit] |
| 10:18:36 | | skankhunt42 (skankhunt42) joins |
| 10:30:49 | <cruller|irc> | solved the rest pagination: https://transfer.archivete.am/Ty8UO/data.world_enumeration_pagination_misc.txt |
| 10:30:50 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/Ty8UO/data.world_enumeration_pagination_misc.txt |
| 10:32:47 | | thewinwin88 joins |
| 10:36:41 | | thewinwin8 quits [Ping timeout: 268 seconds] |
| 10:36:47 | | thewinwin88 is now known as thewinwin8 |
| 10:42:14 | | f_ quits [Ping timeout: 268 seconds] |
| 10:45:15 | | Dango360 quits [Quit: The Lounge - https://thelounge.chat] |
| 10:47:06 | | Dango360 (Dango360) joins |
| 10:49:53 | | goro (goro) joins |
| 10:53:23 | | f_ (funderscore) joins |
| 11:00:17 | | f_ quits [Remote host closed the connection] |
| 11:00:19 | | Bleo18260072271962345522201107 quits [Quit: The Lounge - https://thelounge.chat] |
| 11:00:34 | | f_ (funderscore) joins |
| 11:03:05 | | Bleo18260072271962345522201107 joins |
| 11:16:07 | <h2ibot> | Manu edited IRC/Logs (+45, Queued https://mg.pov.lt/maemo-irclog/): https://wiki.archiveteam.org/?diff=63134&oldid=63121 |
| 11:18:00 | | f_ quits [Ping timeout: 268 seconds] |
| 11:21:52 | | f_ (funderscore) joins |
| 11:28:09 | <h2ibot> | Manu edited IRC/Logs (+13, JAA queued https://irclogs.nim-lang.org/ in 2025): https://wiki.archiveteam.org/?diff=63135&oldid=63134 |
| 11:28:29 | | f_ quits [Ping timeout: 268 seconds] |
| 11:30:29 | | f_ (funderscore) joins |
| 11:32:09 | <h2ibot> | Manu edited IRC/Logs (+13, Queued https://stendhalgame.org/chat/): https://wiki.archiveteam.org/?diff=63136&oldid=63135 |
| 11:37:33 | | BitByBit4100 (BitByBit) joins |
| 11:38:18 | <mindrecent> | arkiver: data.world. search only lists public afaik. I haven't had any access issues. |
| 11:39:10 | <h2ibot> | Manu edited IRC/Logs (+14, Queued https://irclogs.ai0867.net/): https://wiki.archiveteam.org/?diff=63137&oldid=63136 |
| 11:47:12 | <h2ibot> | Manu edited Wikinews/archived (-434, /* Web */): https://wiki.archiveteam.org/?diff=63138&oldid=63086 |
| 11:48:12 | <h2ibot> | Manu edited Wikinews/archived (+38, /* Web */): https://wiki.archiveteam.org/?diff=63139&oldid=63138 |
| 11:52:12 | <h2ibot> | Manu edited Wikinews/archived (+49, /* Web */): https://wiki.archiveteam.org/?diff=63140&oldid=63139 |
| 11:53:13 | <h2ibot> | Manu edited Wikinews/archived (+2, /* Web */): https://wiki.archiveteam.org/?diff=63141&oldid=63140 |
| 12:11:39 | | masterX244 quits [Ping timeout: 268 seconds] |
| 12:13:28 | | masterX244 (masterX244) joins |
| 12:17:00 | | error quits [Remote host closed the connection] |
| 12:17:20 | | error (disinfecttm) joins |
| 12:36:33 | | nine quits [Quit: See ya!] |
| 12:36:46 | | nine joins |
| 12:53:20 | <justauser> | ericgallager: GCC -> #codearchiver , forwarded. |
| 12:55:32 | | nine quits [Client Quit] |
| 12:55:45 | | nine joins |
| 12:58:34 | | Cuphead2527480 (Cuphead2527480) joins |
| 13:22:13 | <@arkiver> | mindrecent: many datasets i find for example require a login |
| 13:33:41 | | thewinwin81 joins |
| 13:37:22 | | thewinwin8 quits [Ping timeout: 268 seconds] |
| 13:37:26 | | thewinwin81 is now known as thewinwin8 |
| 13:57:30 | <justauser> | https://www.flamesofwar.com/ is still here, despite claims of closure "in coming weeks" in '25, but its forum is now login-only. I wonder if mailing them would help. |
| 14:29:46 | | Webuser959005 joins |
| 14:30:26 | | Webuser959005 is now known as raintheelite5000 |
| 14:32:15 | | moth3 quits [Ping timeout: 248 seconds] |
| 15:01:09 | | Tom9 joins |
| 15:01:56 | | Tom1992 joins |
| 15:08:24 | | Cuphead2527480 quits [Client Quit] |
| 15:09:23 | <Tom1992> | Hello, |
| 15:09:23 | <Tom1992> | Please archive this Qt repository, URL: https://fami-net.no-ip.org/mirrors/qt/unofficial_builds. It is an only one left of unofficial Qt builds collection. Actually, it was closed but I ask the admin of this site to reopen it. It will be available until 1 August 2026. |
| 15:09:41 | | Yna joins |
| 15:10:54 | | raintheelite5000 quits [Client Quit] |
| 15:16:03 | | Tom9 quits [Client Quit] |
| 15:16:03 | | Tom1992 quits [Client Quit] |
| 15:18:04 | | Tom1992 joins |
| 15:22:23 | | Tom1992 quits [Client Quit] |
| 15:23:28 | | Tom1992 joins |
| 15:23:31 | | IDK (IDK) joins |
| 15:24:12 | | Island joins |
| 15:25:13 | | MrMcNugg1 (MrMcNuggets) joins |
| 15:25:32 | | garbanz joins |
| 15:25:35 | | MrMcNuggets quits [Ping timeout: 248 seconds] |
| 15:29:32 | <justauser> | Tom1992: Started a job. |
| 15:30:47 | | traxys quits [Quit: The Lounge - https://thelounge.chat] |
| 15:33:19 | | nathang218438 quits [Quit: The Lounge - https://thelounge.chat] |
| 15:35:54 | | traxys (traxys) joins |
| 15:37:12 | | McAfee quits [Quit: Gateway shutdown] |
| 15:50:10 | | thewinwin82 joins |
| 15:52:12 | <@arkiver> | we'll not get historical files for data.world |
| 15:53:51 | | thewinwin8 quits [Ping timeout: 248 seconds] |
| 15:53:57 | | thewinwin82 is now known as thewinwin8 |
| 15:55:45 | | Tom1992 quits [Client Quit] |
| 16:00:35 | | Cornelius7052865 quits [Quit: Cornelius7052865] |
| 16:03:22 | | Cornelius7052865 (Cornelius) joins |
| 16:06:02 | | Cornelius7052865 quits [Client Quit] |
| 16:06:52 | | Cornelius7052865 (Cornelius) joins |
| 16:07:38 | | bladem (bladem) joins |
| 16:35:04 | | nathang218438 joins |
| 16:36:26 | | DogsRNice joins |
| 16:36:42 | | nathang218438 quits [Client Quit] |
| 16:43:12 | | nathang218438 joins |
| 16:46:08 | <h2ibot> | Brad edited Distributed recursive crawls (+55, Added TMDB): https://wiki.archiveteam.org/?diff=63142&oldid=63131 |
| 16:54:05 | | Yna quits [Ping timeout: 268 seconds] |
| 16:59:16 | | Yna joins |
| 17:21:48 | | goro quits [Remote host closed the connection] |
| 17:22:12 | | goro (goro) joins |
| 17:49:29 | | goro quits [Client Quit] |
| 17:49:42 | | goro (goro) joins |
| 17:50:33 | | TheEnbyperor quits [Read error: Connection reset by peer] |
| 19:35:04 | | Wohlstand (Wohlstand) joins |
| 19:35:40 | | nexussfan (nexussfan) joins |
| 19:46:45 | | BornOn420 quits [Ping timeout: 268 seconds] |
| 19:55:20 | | garbanz quits [Quit: Ooops, wrong browser tab.] |
| 19:56:45 | | goro quits [Client Quit] |
| 20:23:28 | | BornOn420 (BornOn420) joins |
| 20:33:00 | | APOLLO03 quits [Ping timeout: 268 seconds] |
| 20:33:33 | | APOLLO03 joins |
| 20:36:05 | | Radzig quits [Ping timeout: 268 seconds] |
| 20:45:08 | | Radzig joins |
| 20:56:17 | | moth3 joins |
| 21:00:28 | | Pendonym quits [Quit: Pendonym] |
| 21:00:48 | | Pendonym joins |
| 21:00:49 | | Pendonym is now known as RJHacker95878 |
| 21:01:06 | | RJHacker95878 is now known as Pendonym |
| 21:01:06 | | Pendonym is now authenticated as Pendonym |
| 21:03:19 | | thewinwin84 joins |
| 21:06:26 | | Pendonym quits [Client Quit] |
| 21:06:29 | | Pendonym (Pendonym) joins |
| 21:06:40 | | Pendonym quits [Client Quit] |
| 21:06:46 | | Pendonym (Pendonym) joins |
| 21:06:55 | | thewinwin8 quits [Ping timeout: 248 seconds] |
| 21:07:01 | | Pendonym quits [Client Quit] |
| 21:07:03 | | thewinwin84 is now known as thewinwin8 |
| 21:07:07 | | Pendonym (Pendonym) joins |
| 21:08:00 | | Pendonym quits [Client Quit] |
| 21:08:08 | | Pendonym (Pendonym) joins |
| 21:10:41 | | Starchives__ (Starchives) joins |
| 21:14:19 | | Starchives_ quits [Ping timeout: 268 seconds] |
| 21:25:02 | <h2ibot> | Arkiver uploaded File:Data.world-icon.png: https://wiki.archiveteam.org/?title=File%3AData.world-icon.png |
| 21:27:02 | <h2ibot> | MindRecent created Data.world (+760, Site hosting hundreds of thousands of datasets…): https://wiki.archiveteam.org/?oldid=63144 |
| 21:29:02 | <h2ibot> | BOplaid edited YouTube (+459, /* (Manual) Recommended way to archive YouTube…): https://wiki.archiveteam.org/?diff=63147&oldid=62689 |
| 21:29:03 | <h2ibot> | Pendonym edited URLTeam (+81, /* Alive */): https://wiki.archiveteam.org/?diff=63148&oldid=61018 |
| 21:30:02 | <h2ibot> | Gemkritical edited Shemmy (+438, its back): https://wiki.archiveteam.org/?diff=63149&oldid=62843 |
| 21:30:03 | <h2ibot> | Coolerplate edited Miraheze (+0, Fixed link to IA): https://wiki.archiveteam.org/?diff=63150&oldid=54943 |
| 21:30:04 | <h2ibot> | Kittyluv edited TikTok (+503, /* Archiving tools */): https://wiki.archiveteam.org/?diff=63151&oldid=61532 |
| 21:30:05 | <h2ibot> | Tom Kozak uploaded File:Earth.gif: https://wiki.archiveteam.org/?title=File%3AEarth.gif |
| 21:30:50 | | etnguyen03 (etnguyen03) joins |
| 21:48:42 | <@arkiver> | dataworld coming |
| 21:48:48 | <@arkiver> | we're going to use the #// target |
| 21:53:47 | | Yna quits [Ping timeout: 268 seconds] |
| 21:54:24 | | nine quits [Ping timeout: 268 seconds] |
| 21:56:58 | | hamouda joins |
| 22:03:02 | | h|ca2 quits [Ping timeout: 268 seconds] |
| 22:03:06 | | nine joins |
| 22:03:47 | | h|ca2 (h) joins |
| 22:06:48 | | nine quits [Client Quit] |
| 22:07:01 | | nine joins |
| 22:08:15 | | chunkynutz6018169 joins |
| 22:08:38 | <@arkiver> | JAA: if you are around, can you please poke drone for https://github.com/ArchiveTeam/dataworld-grab ? |
| 22:10:26 | | chunkynutz601816 quits [Ping timeout: 268 seconds] |
| 22:10:27 | | chunkynutz6018169 is now known as chunkynutz601816 |
| 22:10:53 | <skankhunt42> | so not urls but standalone DPoS? |
| 22:15:07 | | error2 (disinfecttm) joins |
| 22:16:02 | | error quits [Remote host closed the connection] |
| 22:18:31 | <@JAA> | arkiver: Done |
| 22:18:51 | <@arkiver> | thank you very much! |
| 22:19:05 | <@arkiver> | skankhunt42: yes, but target is URLs since we couldn't get a target for this in time |
| 22:23:45 | <skankhunt42> | sorry, still kinda new. what’s the target in this case? |
| 22:28:15 | <Dango360> | so any items completed will show up in https://tracker.archiveteam.org/urls/ instead? |
| 22:28:25 | <klea> | No. |
| 22:28:34 | <Dango360> | yeah just saw arkiver's item |
| 22:28:38 | <Dango360> | in https://tracker.archiveteam.org/dataworld/ |
| 22:29:01 | <@arkiver> | yeah they show up there, but will end up with #// WARCs |
| 22:29:08 | <@arkiver> | so no separate collection on IA |
| 22:29:24 | <klea> | Because no imer available now? |
| 22:29:25 | <@arkiver> | though we could separate it out since #// writes Zst WARCs, while dataworld writes gz |
| 22:29:33 | <@arkiver> | klea: temporarily yes :) |
| 22:29:55 | <klea> | I thought arkiver was able to self-setup stuff too. *hides* |
| 22:30:15 | <@arkiver> | i am incredibly tired though. normally this would be a moment i'd go "wait for tomorrow, rather not start now", but we have no time so :P |
| 22:30:49 | <Dango360> | too bad the 429s for https://download.data.world/datapackage are pretty stingy |
| 22:31:09 | | klea puts arkiver to eep, since eeping is good, and IIRC has already setup everything. |
| 22:31:32 | | ThreeHM (ThreeHeadedMonkey) joins |
| 22:31:41 | <@arkiver> | Dango360: they'll go away, just run it at concurrency 1 |
| 22:32:15 | <Dango360> | is unauthorised to be expected? 43=429 https://download.data.world/datapackage/prince25/don25?dwr=US |
| 22:33:46 | <@arkiver> | should work after some tries |
| 22:33:49 | <@arkiver> | let me know if it works |
| 22:35:42 | <Dango360> | !remindme 5m check if project:prince25/don25 completes |
| 22:35:44 | <eggdrop> | [remind] ok, i'll remind you at 2026-07-13T22:40:43Z |
| 22:36:31 | | bladem quits [Ping timeout: 248 seconds] |
| 22:38:13 | <h2ibot> | KleaBot edited Main Page/Current Warrior Project (+112, Default projects are now telegram (weight 9),…): https://wiki.archiveteam.org/?diff=63153&oldid=63122 |
| 22:38:21 | | bladem (bladem) joins |
| 22:39:23 | <klea> | arkiver: Why not put all the Warriors on data.world and limit the rate you give items?, since #telegrab is paused. |
| 22:40:04 | <@arkiver> | klea: because i need to go, can't work out a good rate at the moment |
| 22:40:09 | <klea> | Ah |
| 22:40:27 | <@arkiver> | the moment is coming though that we can do that through IRC and other here with +v or +o (or in special channels) can help with this |
| 22:40:43 | <eggdrop> | [remind] Dango360: check if project:prince25/don25 completes |
| 22:40:48 | <Dango360> | still nothing |
| 22:41:03 | | etnguyen03 quits [Client Quit] |
| 22:42:36 | | @arkiver is off |
| 22:43:12 | | Tom1992 joins |
| 22:43:25 | <@arkiver> | if someone else is around with right privs to handle rate, please do |
| 22:44:59 | <@arkiver> | for now i set the rate at 1k/min |
| 22:45:28 | | etnguyen03 (etnguyen03) joins |
| 22:48:06 | | Tom1992 quits [Client Quit] |
| 22:52:31 | | BitByBit4100 quits [Ping timeout: 248 seconds] |
| 22:54:50 | | multisn8 quits [Ping timeout: 268 seconds] |
| 22:59:09 | <Dango360> | it got past that item, so that's good |
| 23:01:56 | | szczot3k quits [Read error: Connection reset by peer] |
| 23:02:05 | | szczot3k (szczot3k) joins |
| 23:07:28 | | @rewby quits [Ping timeout: 248 seconds] |
| 23:08:43 | <nicolas17> | I am once again asking for a way to archive with dedup :/ |
| 23:11:02 | <nicolas17> | I need a way to say "archive these URLs into a deduplicated WARC https://transfer.archivete.am/inline/qPpwh/list-macOS-25G5057c.txt" whether it's DPoS-like or archivebot-like |
| 23:11:06 | <nicolas17> | I thought #Y would eventually let me do that but idk if there has been progress on it |
| 23:11:35 | | multisn8 (multisn8) joins |
| 23:22:23 | | TheEnbyperor joins |
| 23:24:06 | | rewby (rewby) joins |
| 23:24:06 | | @ChanServ sets mode: +o rewby |
| 23:24:58 | | thewinwin84 joins |
| 23:28:45 | | thewinwin8 quits [Ping timeout: 268 seconds] |
| 23:28:48 | | thewinwin84 is now known as thewinwin8 |
| 23:32:33 | <Flashfire42> | Just getting constant 429s on the dataworld |
| 23:33:41 | | etnguyen03 quits [Client Quit] |
| 23:35:53 | <mindrecent> | They are very, very, very 429 happy. |
| 23:38:37 | | Wohlstand quits [Quit: Wohlstand] |
| 23:42:44 | <Flashfire42> | mindrecent will it eventually go through or what? I might just switch back to youtube at this point |
| 23:43:49 | <mindrecent> | flashfire42: it should eventually go through, and as late a date as it is, I'd say every single machine helps. |
| 23:46:00 | <Flashfire42> | oh shit just looked at the wiki i can deal with 429s if thats the deadline |
| 23:52:20 | | moth3 quits [Remote host closed the connection] |
| 23:52:24 | | moth3 joins |
| 23:52:39 | | moth3 quits [Remote host closed the connection] |
| 23:52:58 | | moth3 joins |
| 23:58:00 | | moth3 quits [Remote host closed the connection] |
| 23:58:05 | | moth3 joins |
| 23:58:47 | | moth3 quits [Remote host closed the connection] |
| 23:59:01 | | moth3 joins |
| 23:59:11 | | hamouda quits [Client Quit] |