| 00:12:04 | | hackbug quits [Remote host closed the connection] |
| 00:14:54 | | tekulvw (tekulvw) joins |
| 00:16:27 | | hackbug (hackbug) joins |
| 00:19:14 | | Dada quits [Remote host closed the connection] |
| 00:22:00 | | tekulvw quits [Ping timeout: 268 seconds] |
| 00:27:32 | | tekulvw (tekulvw) joins |
| 00:32:37 | | tekulvw quits [Ping timeout: 272 seconds] |
| 00:36:52 | | tekulvw (tekulvw) joins |
| 00:36:52 | <klea> | https://discourse.nixos.org/t/garbage-collecting-cache-nixos-org/74249/10 |
| 00:37:05 | <klea> | Tonight we enabled Bucket Versioning and configured a Lifecycle Rule to delete the non-default object version after 30 days. See enable bucket versioning · NixOS/infra@9cf1919 · GitHub for details. |
| 00:37:05 | <klea> | Then we deleted everything contained in the following datasets. This should amount to roughly 100 TiB, so something like 10% of the total S3 size. |
| 00:37:05 | <klea> | datasets/narinfos-nixos-images-2026-01-06T01-13Z.parquet · brianmcgee/nix-cache-dataset at main |
| 00:37:05 | <klea> | datasets/narinfos-nixos-images-dangling-refs-2026-01-06T01-13Z.parquet · brianmcgee/nix-cache-dataset at main |
| 00:37:05 | <klea> | The result is that the default version served by cache.nixos.org will return a HTTP 404 response. The non-default version is still around and can be restored within the next 30 days should the deletion cause severe issues. |
| 00:40:30 | <klea> | Tho, since it's a cache, shouldn't affect much? |
| 00:41:31 | | @JAA doesn't know enough about how any of that works. |
| 00:41:34 | <klea> | hexa-: is there a way to make sure things for which sources have linkrotted, but there's still the source code in the cache, for the source code to be archived from the cache, or to have the cache avoid losing source code? |
| 00:42:19 | <@JAA> | Does 'we deleted' mean it's already gone from public view and restorable until next month, or is it still publicly accessible until then? |
| 00:43:26 | <klea> | I suppose it means it's not the latest version, and would 404 from cache.nixos.org, but would still work if you do a aws s3 signed request with requester-pays to the bucket asking for the older revision? |
| 00:45:02 | <nicolas17> | I think if a file gets "overwritten" today, the old version will get deleted after 30 days |
| 00:45:09 | <nicolas17> | oh wait |
| 00:45:13 | <nicolas17> | those were different steps |
| 00:46:03 | | tekulvw quits [Ping timeout: 268 seconds] |
| 00:46:17 | <nicolas17> | yeah ok, they "deleted" a bunch of files that they determined through external processes (dangling refs?), but they're still retrievable via versioning |
| 00:46:23 | <nicolas17> | for the next 30 days |
| 00:46:28 | <nicolas17> | maybe not publicly |
| 00:47:37 | <nicolas17> | is the bucket public? |
| 00:48:10 | <klea> | yes, but requires requester-pays. |
| 00:48:38 | <nicolas17> | what's the bucket name? |
| 00:49:44 | <klea> | I suppose cache.nixos.org? |
| 00:50:00 | <nicolas17> | seems it's https://s3.amazonaws.com/nix-cache |
| 00:51:29 | <nicolas17> | listing is disabled anyway |
| 00:51:47 | <klea> | There's also https://s3.amazonaws.com/nix-channels |
| 00:52:05 | <klea> | 2026-02-21 00:51:29 <nicolas17> listing is disabled anyway <- The parquet file (database format) should contain the removed data? |
| 00:52:34 | <nicolas17> | if file listing is disabled, I suspect I also can't say "list versions of file X" |
| 00:52:46 | <nicolas17> | unless the parquet file has the version IDs? |
| 00:52:50 | <klea> | jhttps://blog.erethon.com/blog/2025/07/31/how-nixos-is-built/ was interesting. |
| 00:53:11 | <klea> | I don't know, I should try to check the parquet file. |
| 00:53:32 | <nicolas17> | I have no idea how to read parquet :P |
| 00:53:54 | <nicolas17> | lol @ using huggingface for this |
| 00:54:01 | <klea> | I mean, it's a CDN :p |
| 00:56:07 | <klea> | https://transfer.archivete.am/inline/iWYaN/2026-02-21T00:55:40Z--console.txt |
| 00:59:36 | <klea> | nicolas17: there's a python thing called parquet-tools apparently. |
| 01:00:14 | <klea> | otherwise, no it doesn't seem to include version ids? |
| 01:09:31 | | tekulvw (tekulvw) joins |
| 01:14:25 | | tekulvw quits [Ping timeout: 268 seconds] |
| 01:16:32 | | Wohlstand quits [Quit: Wohlstand] |
| 01:26:51 | | Cupping1285 quits [Quit: bye] |
| 01:27:45 | | Cupping1285 joins |
| 01:39:03 | | Arcorann_ (Arcorann) joins |
| 01:42:10 | | Arcorann quits [Ping timeout: 268 seconds] |
| 02:25:27 | <h2ibot> | Hans5958 created Roblox Groups (+57, Redirected page to [[Roblox#Group Walls…): https://wiki.archiveteam.org/?oldid=60532 |
| 03:22:27 | | tekulvw (tekulvw) joins |
| 03:27:00 | | tekulvw quits [Ping timeout: 268 seconds] |
| 04:02:35 | <pabs> | https://arstechnica.com/tech-policy/2026/02/wikipedia-bans-archive-today-after-site-executed-ddos-and-altered-web-captures/ |
| 04:02:44 | <pabs> | woops, already posted |
| 04:05:14 | | lennier2 quits [Ping timeout: 268 seconds] |
| 04:06:04 | | lennier2 joins |
| 04:14:36 | | etnguyen03 quits [Remote host closed the connection] |
| 04:18:05 | | nexussfan quits [Read error: Connection reset by peer] |
| 04:35:10 | | Bog joins |
| 04:37:29 | | Bog quits [Client Quit] |
| 05:02:50 | | rover joins |
| 05:04:57 | | roverinexile quits [Ping timeout: 272 seconds] |
| 05:04:57 | | n9nes quits [Ping timeout: 272 seconds] |
| 05:05:12 | | n9nes joins |
| 05:16:25 | | tekulvw (tekulvw) joins |
| 05:18:05 | <tmg1|michelson> | a few hours later, opendiary still full of bad responses |
| 05:21:05 | | tekulvw quits [Ping timeout: 268 seconds] |
| 05:40:34 | | Stvkimension11 (Stvkimension11) joins |
| 05:49:03 | | tekulvw (tekulvw) joins |
| 05:51:02 | | Stvkimension11 quits [Client Quit] |
| 05:53:43 | | tekulvw quits [Ping timeout: 272 seconds] |
| 06:02:18 | <steering> | > roughly 100 TiB ... 10% of the total size. |
| 06:02:29 | <steering> | W. A. T. |
| 06:02:49 | <BlankEclair> | a lil chonker |
| 06:19:03 | | midou quits [Ping timeout: 272 seconds] |
| 06:31:59 | | midou joins |
| 06:36:52 | | aliz joins |
| 06:37:16 | | Island quits [Read error: Connection reset by peer] |
| 06:58:57 | | aliz quits [Client Quit] |
| 07:16:19 | <hexa-> | JAA: ig by passing version id for the object |
| 07:18:14 | <hexa-> | the version id for all old objects is null fwiw |
| 07:23:13 | <hexa-> | https://gist.github.com/Mic92/7bcacea70a8babf327e45dc102489445 |
| 07:24:49 | <hexa-> | what got deleted is things we really don't need anymore, like images created for nixos tests |
| 07:24:59 | <hexa-> | and also old installers iirc |
| 07:30:40 | <hexa-> | oh, I think they're not queryable over the fastly cache, likely due to missing permissions |
| 07:33:52 | <hexa-> | or maybe delete markers just shadow everything over the s3 web api, dunno |
| 07:37:11 | | ducky quits [Remote host closed the connection] |
| 07:41:00 | | ducky (ducky) joins |
| 07:41:08 | | SootBector quits [Remote host closed the connection] |
| 07:42:18 | | SootBector (SootBector) joins |
| 07:50:31 | | ducky quits [Remote host closed the connection] |
| 07:54:15 | | ducky (ducky) joins |
| 07:57:13 | | lflare quits [Ping timeout: 272 seconds] |
| 07:58:56 | | lflare (lflare) joins |
| 08:00:38 | | tekulvw (tekulvw) joins |
| 08:04:53 | | ducky_ (ducky) joins |
| 08:05:44 | | tekulvw quits [Ping timeout: 268 seconds] |
| 08:07:21 | | ducky quits [Ping timeout: 272 seconds] |
| 08:09:53 | | ducky_ quits [Ping timeout: 272 seconds] |
| 08:13:41 | | lflare quits [Ping timeout: 272 seconds] |
| 08:15:46 | | lflare (lflare) joins |
| 08:21:45 | | ducky (ducky) joins |
| 08:26:33 | | ducky quits [Remote host closed the connection] |
| 08:28:15 | | lflare quits [Client Quit] |
| 08:38:34 | | lflare (lflare) joins |
| 08:47:44 | | AlsoHP_Archivist joins |
| 08:48:31 | | HP_Archivist quits [Ping timeout: 272 seconds] |
| 09:01:56 | | lflare quits [Client Quit] |