| 00:10:34 | | etnguyen03 (etnguyen03) joins |
| 00:11:23 | | etnguyen03 quits [Remote host closed the connection] |
| 00:12:25 | | etnguyen03 (etnguyen03) joins |
| 00:55:28 | | etnguyen03 quits [Client Quit] |
| 01:20:41 | | vitzli (vitzli) joins |
| 01:26:51 | <nicolas17> | there's now 48 builds of the Hytale game (including release and beta) and they're like 1.5GiB each so I guess I will *not* be feeding all to archivebot... |
| 01:26:52 | | vitzli quits [Client Quit] |
| 01:26:57 | | etnguyen03 (etnguyen03) joins |
| 01:29:50 | <pokechu22> | That's probably fine still, though it sounds like they don't deduplicate assets between versions (unlike Minecraft where they only duplicate assets for each major version... but that also works because translations can be updated independently of major/minor/snapshot releases) |
| 01:30:40 | <nicolas17> | pokechu22: they have deltas |
| 01:31:25 | <pokechu22> | ah, but also have non-delta versions of the same and those would be big? I guess maybe you could just save the deltas for beta versions? |
| 01:32:16 | <nicolas17> | afaik if you install the game from scratch, it downloads windows/amd64/release/0/4.pwr which is 1.46GiB |
| 01:33:06 | <pokechu22> | (and that also reminds me that I still need to automate sending new Minecraft versions to archivebot - I have automatic detection of new ones set up for an old and now-broken automatic analysis tool, which even feeds stuff to IRC, but I haven't hooked that up to archivebot) |
| 01:33:11 | <nicolas17> | if you already have the game at internal version 3, it downloads windows/amd64/release/3/4.pwr ("patch from 3 to 4") which is 67MiB |
| 01:34:12 | <nicolas17> | 2/4.pwr doesn't exist, so maybe it applies 2/3.pwr and 3/4.pwr in that case |
| 01:34:33 | <nicolas17> | or maybe it goes "screw you" and downloads the whole 1.5GiB again |
| 01:38:15 | <nicolas17> | for local data-hoarding purposes, I wasted 2 days figuring out how to decompress a .pwr and *re-compress it back to the original*, so I can do my own delta'ing of the large files |
| 01:38:59 | <nicolas17> | but that's useless for WBM purposes :P |
| 01:41:40 | | lucifer_sam joins |
| 02:04:07 | | Guest58 quits [Read error: Connection reset by peer] |
| 02:04:33 | | Guest58 joins |
| 02:49:31 | | lucifer_sam quits [Ping timeout: 272 seconds] |
| 02:53:17 | | lucifer_sam joins |
| 03:30:34 | | Wohlstand quits [Quit: Wohlstand] |
| 03:34:02 | | tzt (tzt) joins |
| 03:52:45 | | datechnoman quits [Read error: Connection reset by peer] |
| 03:53:46 | | datechnoman (datechnoman) joins |
| 03:59:08 | | lennier2 joins |
| 04:01:50 | | lennier2_ quits [Ping timeout: 256 seconds] |
| 04:28:46 | | DogsRNice quits [Read error: Connection reset by peer] |
| 04:40:38 | | Wohlstand1 (Wohlstand) joins |
| 04:43:01 | | Wohlstand1 is now known as Wohlstand |
| 04:53:05 | | Wohlstand quits [Client Quit] |
| 04:56:54 | | Wohlstand1 (Wohlstand) joins |
| 04:59:18 | | Wohlstand1 is now known as Wohlstand |
| 05:04:44 | | n9nes quits [Ping timeout: 256 seconds] |
| 05:04:47 | | n9nes joins |
| 05:10:18 | | Wohlstand quits [Client Quit] |
| 05:21:06 | | etnguyen03 quits [Remote host closed the connection] |
| 05:47:07 | <h2ibot> | TriangleDemon edited Main Page/Current Projects (+0): https://wiki.archiveteam.org/?diff=60322&oldid=60302 |
| 06:01:01 | | wotd joins |
| 06:09:10 | <h2ibot> | TriangleDemon edited Maxmodels.pl (+51): https://wiki.archiveteam.org/?diff=60324&oldid=60313 |
| 06:20:41 | | APOLLO03- quits [Read error: Connection reset by peer] |
| 06:21:57 | | APOLLO03 joins |
| 06:23:52 | | nexussfan quits [Quit: Konversation terminated!] |
| 07:36:16 | | Hackerpcs quits [Quit: Hackerpcs] |
| 07:46:55 | | Island quits [Read error: Connection reset by peer] |
| 08:29:11 | | Hackerpcs (Hackerpcs) joins |
| 08:52:32 | | midou quits [Ping timeout: 256 seconds] |
| 09:01:12 | | tertu (tertu) joins |
| 09:04:27 | | tertu2 quits [Ping timeout: 272 seconds] |
| 09:45:59 | | Dada joins |
| 09:53:13 | | midou joins |
| 10:05:04 | | Dada quits [Remote host closed the connection] |
| 10:05:16 | | Dada joins |
| 10:16:37 | | Guest58 quits [Quit: My Mac has gone to sleep. ZZZzzz…] |
| 10:17:06 | | Guest58 joins |
| 10:21:41 | | Guest58 quits [Client Quit] |
| 10:22:31 | | Guest58 joins |
| 10:28:41 | | midou quits [Ping timeout: 272 seconds] |
| 10:58:45 | | beastbg8_ quits [Read error: Connection reset by peer] |
| 11:03:19 | | Woodie quits [] |
| 11:17:25 | | jonte6 (jonte4) joins |
| 11:19:59 | | jonte quits [Ping timeout: 272 seconds] |
| 11:19:59 | | jonte6 is now known as jonte |
| 11:39:06 | <cruller> | nicolas17 pokechu22:Out of curiosity, I tried running `hdiffz -s-64 -d game-patches.hytale.com-patches-windows-amd64-release-0-3.pwr.warc game-patches.hytale.com-patches-windows-amd64-release-0-4.pwr.warc output.patch` |
| 11:39:23 | <cruller> | The file sizes are as follows: 3.pwr.warc = 1,578,281,510; 4.pwr.warc = 1,578,171,694; hdiff-patch = 860,744,931; bsdiff-pacth = 850,700,130; vcdiff-patch = 860,790,261 |
| 11:39:25 | | jonte9 (jonte4) joins |
| 11:39:42 | | jonte quits [Ping timeout: 256 seconds] |
| 11:39:42 | | jonte9 is now known as jonte |
| 11:41:29 | <cruller> | -m option should yield better results, but I don't have enough memory. |
| 11:44:03 | | Dada quits [Remote host closed the connection] |
| 11:48:45 | | beastbg8 (beastbg8) joins |
| 11:50:56 | <cruller> | Courgette might be better too, but I hear it's difficult to install. However, there are https://github.com/hiromi-mi/standalone-courgette and https://github.com/rgov/courgette-build |
| 11:51:45 | <h|ca2> | cruller: out of curiosity, RE: memory: (a) would zram maybe help at all? and (b) how much do you have? |
| 11:58:27 | <cruller> | h|ca2: (a) I have no idea... (b) Since it was just a test, I used a machine with a measly 8GB of RAM. My main machine has 16GB, which isn't exactly high-end, but it might just be enough? |
| 11:59:37 | <h|ca2> | cruller: I'd suggest trying to set up 16gb of zram as a test on the 8G machine in case that ends up being enough (in my experience, you can generally get away with that much, although ofc it varies a lot based on what's actually happening) |
| 12:00:05 | | Bleo182600722719623455222 quits [Quit: The Lounge - https://thelounge.chat] |
| 12:01:56 | <cruller> | Thanks. Btw according to https://github.com/sisong/HDiffPatch , it requres {9+O(1)} GB. (In theory) |
| 12:02:53 | | Bleo182600722719623455222 joins |
| 12:23:25 | | gosc joins |
| 12:24:17 | <gosc> | I've got 160mb of urls from mobile games and I've been hanging on to them for about 10 days now, I'm guessing just upload the zip to #archivebot? |
| 12:24:31 | <gosc> | it's not a single 100mb file so it shouldn't be an issue I think |
| 13:07:07 | <h2ibot> | Manu edited Discourse/archived (+100, Queued chapel.discourse.group): https://wiki.archiveteam.org/?diff=60325&oldid=60317 |
| 13:07:39 | | pseudorizer quits [Ping timeout: 272 seconds] |
| 13:34:33 | | etnguyen03 (etnguyen03) joins |
| 13:49:07 | <cruller> | <cruller> "The file sizes are as follows: 3..." <- with -m option: hdiff = 850162804; bsdiff = 843431466; vcdiff = 854538976 |
| 13:52:39 | <cruller> | diff time: hdiff = 757; bsdiff = 815; vcdiff = 714 |
| 14:05:58 | | KerwoodDerby quits [Quit: WeeChat 4.1.1] |
| 14:14:29 | | etnguyen03 quits [Client Quit] |
| 14:37:41 | | etnguyen03 (etnguyen03) joins |
| 14:58:05 | | Dada joins |
| 15:00:26 | <h2ibot> | Hans5958 edited Main Page/Current Projects (+21, Add none currently): https://wiki.archiveteam.org/?diff=60326&oldid=60322 |
| 15:00:49 | | nexussfan (nexussfan) joins |
| 15:04:11 | | mls quits [Ping timeout: 272 seconds] |
| 15:05:26 | | mls (mls) joins |
| 15:06:38 | | etnguyen03 quits [Client Quit] |
| 15:39:58 | | ducky quits [Ping timeout: 256 seconds] |
| 16:14:34 | | ducky (ducky) joins |
| 16:31:08 | | DogsRNice joins |
| 16:38:12 | | NatTheCat0 (NatTheCat) joins |
| 16:39:11 | | NatTheCat quits [Ping timeout: 272 seconds] |
| 16:39:11 | | NatTheCat0 is now known as NatTheCat |
| 17:01:41 | <@Fusl> | i recently got my hand on some old tech magazines from the years 197x-198x in pretty decent condition and im trying to figure out what the best approach is to get those digitized and archived in the IA. did anyone ever have to deal with that and can share some notes, experiences, or maybe even guide me on where and how to get started? |
| 17:04:53 | <justauser> | Jason, I guess? |
| 17:05:32 | <klea> | Fusl: https://diybookscanner.org/ ? but if you can get them to IA's physical stashes that could be another way. I don't know your location. |
| 17:06:00 | <@Fusl> | im in austria, i dont know how well the magazines survive getting shipped to the US |
| 17:07:13 | <klea> | oh. |
| 17:07:50 | <klea> | i guess read around that community i was linked to when i asked about physical scan of very obscure book and maybe make your own scanning jig? |
| 17:26:12 | | sec^nd quits [Remote host closed the connection] |
| 17:27:14 | | sec^nd (second) joins |
| 17:52:00 | | Kotomind quits [Ping timeout: 256 seconds] |
| 18:01:00 | | nexussfan quits [Quit: Konversation terminated!] |
| 18:01:40 | | PC joins |
| 18:02:18 | <PC> | does anyone know who's doing these archives? i've got a text file full of twitter links that i'd love saved, but no idea how to get them saved like these ones are getting https://archive.org/details/twitterarchive |
| 18:05:14 | <justauser> | Apparently that's IA themselves. |
| 18:06:29 | <PC> | huh |
| 18:06:33 | <PC> | so emailing them...? |
| 18:06:46 | <justauser> | Without details, I suspect that's the same deal as https://archive.org/details/twitterstream?tab=about |
| 18:07:09 | <justauser> | I.e. a random subset of fresh tweets for a day, in API form. |
| 18:08:20 | <PC> | yeah, seems to be JSONs so probably from the API |
| 18:08:47 | | Webuser190477 joins |
| 18:08:54 | <justauser> | How did you figure out it's JSONs? |
| 18:08:57 | <klea> | collection made by <https://archive.org/details/@markjgraham> <mailto:mark@archive.org> :p |
| 18:09:26 | <klea> | oh wait it's IA access restricted also. |
| 18:10:45 | | LddPotato quits [Read error: Connection reset by peer] |
| 18:10:54 | <PC> | https://web.archive.org/web/20250501000200/https://twitter.com/Velinxi/status/1917731191324827671 |
| 18:10:56 | <eggdrop> | nitter: https://nitter.net/Velinxi/status/1917731191324827671 |
| 18:11:00 | <PC> | this says it was generated from a JSON |
| 18:11:26 | | LddPotato (LddPotato) joins |
| 18:11:41 | <justauser> | Huh, neat. |
| 18:12:09 | | Webuser190477 quits [Client Quit] |
| 18:12:10 | <PC> | yeah, it's a nifty way to get the posts. i'd just like to figure out how to get the ones i need to save in there, hah |
| 18:12:15 | <justauser> | And the short links are automatically unshortened. |
| 18:12:24 | <PC> | oh yeah, that too |
| 18:12:34 | <justauser> | Individual tweets may or may not work in #jseater. |
| 18:12:53 | <justauser> | It currently doesn't feed Wayback, but this is intended to change. |
| 18:13:14 | <PC> | hmm good to know |
| 18:13:51 | <justauser> | And the image in the post is marked an SPN... |
| 18:14:05 | <PC> | SPN? |
| 18:14:11 | <PC> | oh, save page now |
| 18:14:57 | | etnguyen03 (etnguyen03) joins |
| 18:14:58 | <PC> | i've already got all the images for the tweets i need to get, that just leaves the actual post bodies and metadata (and having the image loaded when looking the post up, or at least the URL somewhere in the HTML) |
| 18:15:20 | <PC> | so i've just got... 529 tweets, if they could get grabbed via this method, that'd be great. just not sure whom to contact about it |
| 18:17:50 | <justauser> | Our wiki says Mnbot handles tweets. So, upload a list and it will be klea's turn. |
| 18:18:06 | <klea> | kh2i's? |
| 18:18:15 | <justauser> | Can you insert some extra delays between requests? |
| 18:18:27 | <klea> | justauser: if you mean between lines sent by !bulk, i could. |
| 18:18:48 | <klea> | however, that's probably a pita |
| 18:18:48 | <justauser> | I'd guess it would be a good idea, to avoid getting it blocked. |
| 18:18:57 | <klea> | from #jseater?, it's voiced afaik? |
| 18:19:05 | <klea> | and people should be using -n anyways :p |
| 18:19:13 | <justauser> | Getting Mnbot blocked from X. |
| 18:19:25 | <klea> | #jseater |
| 18:19:33 | <klea> | i should make another command to queue slowly |
| 18:20:21 | <PC> | justauser: thank you! just gimme a mo |
| 18:21:03 | <justauser> | Unless someone (JAA?) has a better estimate, I'd go with one request per 5..15 minutes. |
| 18:21:26 | <klea> | that'd probably require reworking the logic :p |
| 18:21:48 | <klea> | and maybe not making it be a bash script that just throws a bunch of requests into the http2irc api |
| 18:21:57 | <klea> | and relies on the fact http2irc flood limits itself. |
| 18:22:44 | | BornOn420 quits [Quit: Textual IRC Client: www.textualapp.com] |
| 18:23:29 | <justauser> | For single use, curl in a for loop could work. |
| 18:24:41 | <klea> | that's more or less what it does: <https://transfer.archivete.am/inline/8hb2E/at-jseater-bulk.sh> |
| 18:31:39 | | Bajdzo joins |
| 18:31:39 | | LddPotato quits [Read error: Connection reset by peer] |
| 18:31:48 | | Bajdzo quits [Client Quit] |
| 18:32:15 | <PC> | https://transfer.archivete.am/ZDDan/twitter_2026-01-24.txt |
| 18:32:16 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/ZDDan/twitter_2026-01-24.txt |
| 18:32:24 | | LddPotato (LddPotato) joins |
| 18:32:58 | <PC> | that should be it! appreciate the help <3 |
| 18:41:23 | | Kabaya quits [Read error: Connection reset by peer] |
| 18:41:25 | | Kabaya3 joins |
| 18:42:29 | | LddPotato quits [Read error: Connection reset by peer] |
| 18:43:07 | | LddPotato (LddPotato) joins |
| 18:45:18 | | Webuser254925 joins |
| 18:45:34 | | Webuser254925 quits [Client Quit] |
| 18:50:39 | | BornOn420 (BornOn420) joins |
| 19:01:19 | | Wohlstand (Wohlstand) joins |
| 19:01:23 | | nexussfan (nexussfan) joins |
| 19:09:02 | | Dango3604 (Dango360) joins |
| 19:11:18 | | ATinySpaceMarine quits [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.] |
| 19:11:46 | | ATinySpaceMarine joins |
| 19:12:28 | | Dango360 quits [Ping timeout: 256 seconds] |
| 19:12:28 | | Dango3604 is now known as Dango360 |
| 19:38:20 | | gosc quits [Quit: Leaving] |
| 19:40:41 | | ATinySpaceMarine quits [Client Quit] |
| 19:42:26 | | ATinySpaceMarine joins |
| 19:48:58 | <klea> | PC: should i send a request every 5 minutes to my endpoint with params -u stealth -n 100? |
| 19:49:37 | <nicolas17> | cruller: never heard of hdiffz... but yeah I'm getting 20MB my way ;) |
| 19:50:04 | | lukash984 quits [Quit: The Lounge - https://thelounge.chat] |
| 19:53:43 | <PC> | klea: i'd really appreciate that! |
| 19:54:06 | <klea> | I'm waiting for the urls i queued myself there to finish, then i'll start it. |
| 19:54:10 | <PC> | thank youu |
| 19:54:14 | <klea> | you're welcome. |
| 19:57:16 | <klea> | aaa i hate zsh/sh shells |
| 20:02:58 | | nine quits [Quit: See ya!] |
| 20:03:12 | | nine joins |
| 20:03:12 | | nine is now authenticated as nine |
| 20:03:12 | | nine quits [Changing host] |
| 20:03:12 | | nine (nine) joins |
| 20:19:24 | | datechnoman quits [Quit: The Lounge - https://thelounge.chat] |
| 20:21:16 | <h2ibot> | Calmevening edited Android Applications/https (+366): https://wiki.archiveteam.org/?diff=60327&oldid=58203 |
| 20:22:16 | <h2ibot> | Calmevening edited Android Applications/https (+8): https://wiki.archiveteam.org/?diff=60328&oldid=60327 |
| 20:24:16 | <h2ibot> | Calmevening edited IOS (-90, removed not ios related link): https://wiki.archiveteam.org/?diff=60329&oldid=58945 |
| 20:34:45 | | Cornelius quits [Quit: Cornelius] |
| 20:35:32 | | Cornelius (Cornelius) joins |
| 20:45:58 | | APOLLO03 quits [Ping timeout: 256 seconds] |
| 21:07:44 | | Island joins |
| 21:14:29 | | datechnoman (datechnoman) joins |
| 21:25:21 | | DopefishJustin quits [Remote host closed the connection] |
| 21:31:42 | | DopefishJustin joins |
| 21:31:42 | | DopefishJustin is now authenticated as DopefishJustin |
| 21:33:18 | | Webuser937457 joins |
| 21:33:35 | | Webuser937457 quits [Client Quit] |
| 21:35:32 | | Webuser163948 joins |
| 21:35:51 | | Webuser163948 quits [Client Quit] |
| 21:52:41 | | SootBector quits [Remote host closed the connection] |
| 21:53:51 | | SootBector (SootBector) joins |
| 21:59:25 | | ATinySpaceMarine quits [Client Quit] |
| 22:12:00 | <nukke> | Is there a project for the Minnesota protests or ICE in general? |
| 22:12:00 | | APOLLO03 joins |
| 22:12:11 | | midou joins |