HomeSearchPrevious dayNext day

00:10:34etnguyen03 (etnguyen03) joins
00:11:23etnguyen03 quits [Remote host closed the connection]
00:12:25etnguyen03 (etnguyen03) joins
00:55:28etnguyen03 quits [Client Quit]
01:20:41vitzli (vitzli) joins
01:26:51<nicolas17>there's now 48 builds of the Hytale game (including release and beta) and they're like 1.5GiB each so I guess I will *not* be feeding all to archivebot...
01:26:52vitzli quits [Client Quit]
01:26:57etnguyen03 (etnguyen03) joins
01:29:50<pokechu22>That's probably fine still, though it sounds like they don't deduplicate assets between versions (unlike Minecraft where they only duplicate assets for each major version... but that also works because translations can be updated independently of major/minor/snapshot releases)
01:30:40<nicolas17>pokechu22: they have deltas
01:31:25<pokechu22>ah, but also have non-delta versions of the same and those would be big? I guess maybe you could just save the deltas for beta versions?
01:32:16<nicolas17>afaik if you install the game from scratch, it downloads windows/amd64/release/0/4.pwr which is 1.46GiB
01:33:06<pokechu22>(and that also reminds me that I still need to automate sending new Minecraft versions to archivebot - I have automatic detection of new ones set up for an old and now-broken automatic analysis tool, which even feeds stuff to IRC, but I haven't hooked that up to archivebot)
01:33:11<nicolas17>if you already have the game at internal version 3, it downloads windows/amd64/release/3/4.pwr ("patch from 3 to 4") which is 67MiB
01:34:12<nicolas17>2/4.pwr doesn't exist, so maybe it applies 2/3.pwr and 3/4.pwr in that case
01:34:33<nicolas17>or maybe it goes "screw you" and downloads the whole 1.5GiB again
01:38:15<nicolas17>for local data-hoarding purposes, I wasted 2 days figuring out how to decompress a .pwr and *re-compress it back to the original*, so I can do my own delta'ing of the large files
01:38:59<nicolas17>but that's useless for WBM purposes :P
01:41:40lucifer_sam joins
02:04:07Guest58 quits [Read error: Connection reset by peer]
02:04:33Guest58 joins
02:49:31lucifer_sam quits [Ping timeout: 272 seconds]
02:53:17lucifer_sam joins
03:30:34Wohlstand quits [Quit: Wohlstand]
03:34:02tzt (tzt) joins
03:52:45datechnoman quits [Read error: Connection reset by peer]
03:53:46datechnoman (datechnoman) joins
03:59:08lennier2 joins
04:01:50lennier2_ quits [Ping timeout: 256 seconds]
04:28:46DogsRNice quits [Read error: Connection reset by peer]
04:40:38Wohlstand1 (Wohlstand) joins
04:43:01Wohlstand1 is now known as Wohlstand
04:53:05Wohlstand quits [Client Quit]
04:56:54Wohlstand1 (Wohlstand) joins
04:59:18Wohlstand1 is now known as Wohlstand
05:04:44n9nes quits [Ping timeout: 256 seconds]
05:04:47n9nes joins
05:10:18Wohlstand quits [Client Quit]
05:21:06etnguyen03 quits [Remote host closed the connection]
05:47:07<h2ibot>TriangleDemon edited Main Page/Current Projects (+0): https://wiki.archiveteam.org/?diff=60322&oldid=60302
06:01:01wotd joins
06:09:10<h2ibot>TriangleDemon edited Maxmodels.pl (+51): https://wiki.archiveteam.org/?diff=60324&oldid=60313
06:20:41APOLLO03- quits [Read error: Connection reset by peer]
06:21:57APOLLO03 joins
06:23:52nexussfan quits [Quit: Konversation terminated!]
07:36:16Hackerpcs quits [Quit: Hackerpcs]
07:46:55Island quits [Read error: Connection reset by peer]
08:29:11Hackerpcs (Hackerpcs) joins
08:52:32midou quits [Ping timeout: 256 seconds]
09:01:12tertu (tertu) joins
09:04:27tertu2 quits [Ping timeout: 272 seconds]
09:45:59Dada joins
09:53:13midou joins
10:05:04Dada quits [Remote host closed the connection]
10:05:16Dada joins
10:16:37Guest58 quits [Quit: My Mac has gone to sleep. ZZZzzz…]
10:17:06Guest58 joins
10:21:41Guest58 quits [Client Quit]
10:22:31Guest58 joins
10:28:41midou quits [Ping timeout: 272 seconds]
10:58:45beastbg8_ quits [Read error: Connection reset by peer]
11:03:19Woodie quits []
11:17:25jonte6 (jonte4) joins
11:19:59jonte quits [Ping timeout: 272 seconds]
11:19:59jonte6 is now known as jonte
11:39:06<cruller>nicolas17 pokechu22:Out of curiosity, I tried running `hdiffz -s-64 -d game-patches.hytale.com-patches-windows-amd64-release-0-3.pwr.warc game-patches.hytale.com-patches-windows-amd64-release-0-4.pwr.warc output.patch`
11:39:23<cruller>The file sizes are as follows: 3.pwr.warc = 1,578,281,510; 4.pwr.warc = 1,578,171,694; hdiff-patch = 860,744,931; bsdiff-pacth = 850,700,130; vcdiff-patch = 860,790,261
11:39:25jonte9 (jonte4) joins
11:39:42jonte quits [Ping timeout: 256 seconds]
11:39:42jonte9 is now known as jonte
11:41:29<cruller>-m option should yield better results, but I don't have enough memory.
11:44:03Dada quits [Remote host closed the connection]
11:48:45beastbg8 (beastbg8) joins
11:50:56<cruller>Courgette might be better too, but I hear it's difficult to install. However, there are https://github.com/hiromi-mi/standalone-courgette and https://github.com/rgov/courgette-build
11:51:45<h|ca2>cruller: out of curiosity, RE: memory: (a) would zram maybe help at all? and (b) how much do you have?
11:58:27<cruller>h|ca2: (a) I have no idea... (b) Since it was just a test, I used a machine with a measly 8GB of RAM. My main machine has 16GB, which isn't exactly high-end, but it might just be enough?
11:59:37<h|ca2>cruller: I'd suggest trying to set up 16gb of zram as a test on the 8G machine in case that ends up being enough (in my experience, you can generally get away with that much, although ofc it varies a lot based on what's actually happening)
12:00:05Bleo182600722719623455222 quits [Quit: The Lounge - https://thelounge.chat]
12:01:56<cruller>Thanks. Btw according to https://github.com/sisong/HDiffPatch , it requres {9+O(1)} GB. (In theory)
12:02:53Bleo182600722719623455222 joins
12:23:25gosc joins
12:24:17<gosc>I've got 160mb of urls from mobile games and I've been hanging on to them for about 10 days now, I'm guessing just upload the zip to #archivebot?
12:24:31<gosc>it's not a single 100mb file so it shouldn't be an issue I think
13:07:07<h2ibot>Manu edited Discourse/archived (+100, Queued chapel.discourse.group): https://wiki.archiveteam.org/?diff=60325&oldid=60317
13:07:39pseudorizer quits [Ping timeout: 272 seconds]
13:34:33etnguyen03 (etnguyen03) joins
13:49:07<cruller><cruller> "The file sizes are as follows: 3..." <- with -m option: hdiff = 850162804; bsdiff = 843431466; vcdiff = 854538976
13:52:39<cruller>diff time: hdiff = 757; bsdiff = 815; vcdiff = 714
14:05:58KerwoodDerby quits [Quit: WeeChat 4.1.1]
14:14:29etnguyen03 quits [Client Quit]
14:37:41etnguyen03 (etnguyen03) joins
14:58:05Dada joins
15:00:26<h2ibot>Hans5958 edited Main Page/Current Projects (+21, Add none currently): https://wiki.archiveteam.org/?diff=60326&oldid=60322
15:00:49nexussfan (nexussfan) joins
15:04:11mls quits [Ping timeout: 272 seconds]
15:05:26mls (mls) joins
15:06:38etnguyen03 quits [Client Quit]
15:39:58ducky quits [Ping timeout: 256 seconds]
16:14:34ducky (ducky) joins
16:31:08DogsRNice joins
16:38:12NatTheCat0 (NatTheCat) joins
16:39:11NatTheCat quits [Ping timeout: 272 seconds]
16:39:11NatTheCat0 is now known as NatTheCat
17:01:41<@Fusl>i recently got my hand on some old tech magazines from the years 197x-198x in pretty decent condition and im trying to figure out what the best approach is to get those digitized and archived in the IA. did anyone ever have to deal with that and can share some notes, experiences, or maybe even guide me on where and how to get started?
17:04:53<justauser>Jason, I guess?
17:05:32<klea>Fusl: https://diybookscanner.org/ ? but if you can get them to IA's physical stashes that could be another way. I don't know your location.
17:06:00<@Fusl>im in austria, i dont know how well the magazines survive getting shipped to the US
17:07:13<klea>oh.
17:07:50<klea>i guess read around that community i was linked to when i asked about physical scan of very obscure book and maybe make your own scanning jig?
17:26:12sec^nd quits [Remote host closed the connection]
17:27:14sec^nd (second) joins
17:52:00Kotomind quits [Ping timeout: 256 seconds]
18:01:00nexussfan quits [Quit: Konversation terminated!]
18:01:40PC joins
18:02:18<PC>does anyone know who's doing these archives? i've got a text file full of twitter links that i'd love saved, but no idea how to get them saved like these ones are getting https://archive.org/details/twitterarchive
18:05:14<justauser>Apparently that's IA themselves.
18:06:29<PC>huh
18:06:33<PC>so emailing them...?
18:06:46<justauser>Without details, I suspect that's the same deal as https://archive.org/details/twitterstream?tab=about
18:07:09<justauser>I.e. a random subset of fresh tweets for a day, in API form.
18:08:20<PC>yeah, seems to be JSONs so probably from the API
18:08:47Webuser190477 joins
18:08:54<justauser>How did you figure out it's JSONs?
18:08:57<klea>collection made by <https://archive.org/details/@markjgraham> <mailto:mark@archive.org> :p
18:09:26<klea>oh wait it's IA access restricted also.
18:10:45LddPotato quits [Read error: Connection reset by peer]
18:10:54<PC>https://web.archive.org/web/20250501000200/https://twitter.com/Velinxi/status/1917731191324827671
18:10:56<eggdrop>nitter: https://nitter.net/Velinxi/status/1917731191324827671
18:11:00<PC>this says it was generated from a JSON
18:11:26LddPotato (LddPotato) joins
18:11:41<justauser>Huh, neat.
18:12:09Webuser190477 quits [Client Quit]
18:12:10<PC>yeah, it's a nifty way to get the posts. i'd just like to figure out how to get the ones i need to save in there, hah
18:12:15<justauser>And the short links are automatically unshortened.
18:12:24<PC>oh yeah, that too
18:12:34<justauser>Individual tweets may or may not work in #jseater.
18:12:53<justauser>It currently doesn't feed Wayback, but this is intended to change.
18:13:14<PC>hmm good to know
18:13:51<justauser>And the image in the post is marked an SPN...
18:14:05<PC>SPN?
18:14:11<PC>oh, save page now
18:14:57etnguyen03 (etnguyen03) joins
18:14:58<PC>i've already got all the images for the tweets i need to get, that just leaves the actual post bodies and metadata (and having the image loaded when looking the post up, or at least the URL somewhere in the HTML)
18:15:20<PC>so i've just got... 529 tweets, if they could get grabbed via this method, that'd be great. just not sure whom to contact about it
18:17:50<justauser>Our wiki says Mnbot handles tweets. So, upload a list and it will be klea's turn.
18:18:06<klea>kh2i's?
18:18:15<justauser>Can you insert some extra delays between requests?
18:18:27<klea>justauser: if you mean between lines sent by !bulk, i could.
18:18:48<klea>however, that's probably a pita
18:18:48<justauser>I'd guess it would be a good idea, to avoid getting it blocked.
18:18:57<klea>from #jseater?, it's voiced afaik?
18:19:05<klea>and people should be using -n anyways :p
18:19:13<justauser>Getting Mnbot blocked from X.
18:19:25<klea>#jseater
18:19:33<klea>i should make another command to queue slowly
18:20:21<PC>justauser: thank you! just gimme a mo
18:21:03<justauser>Unless someone (JAA?) has a better estimate, I'd go with one request per 5..15 minutes.
18:21:26<klea>that'd probably require reworking the logic :p
18:21:48<klea>and maybe not making it be a bash script that just throws a bunch of requests into the http2irc api
18:21:57<klea>and relies on the fact http2irc flood limits itself.
18:22:44BornOn420 quits [Quit: Textual IRC Client: www.textualapp.com]
18:23:29<justauser>For single use, curl in a for loop could work.
18:24:41<klea>that's more or less what it does: <https://transfer.archivete.am/inline/8hb2E/at-jseater-bulk.sh>
18:31:39Bajdzo joins
18:31:39LddPotato quits [Read error: Connection reset by peer]
18:31:48Bajdzo quits [Client Quit]
18:32:15<PC>https://transfer.archivete.am/ZDDan/twitter_2026-01-24.txt
18:32:16<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/ZDDan/twitter_2026-01-24.txt
18:32:24LddPotato (LddPotato) joins
18:32:58<PC>that should be it! appreciate the help <3
18:41:23Kabaya quits [Read error: Connection reset by peer]
18:41:25Kabaya3 joins
18:42:29LddPotato quits [Read error: Connection reset by peer]
18:43:07LddPotato (LddPotato) joins
18:45:18Webuser254925 joins
18:45:34Webuser254925 quits [Client Quit]
18:50:39BornOn420 (BornOn420) joins
19:01:19Wohlstand (Wohlstand) joins
19:01:23nexussfan (nexussfan) joins
19:09:02Dango3604 (Dango360) joins
19:11:18ATinySpaceMarine quits [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
19:11:46ATinySpaceMarine joins
19:12:28Dango360 quits [Ping timeout: 256 seconds]
19:12:28Dango3604 is now known as Dango360
19:38:20gosc quits [Quit: Leaving]
19:40:41ATinySpaceMarine quits [Client Quit]
19:42:26ATinySpaceMarine joins
19:48:58<klea>PC: should i send a request every 5 minutes to my endpoint with params -u stealth -n 100?
19:49:37<nicolas17>cruller: never heard of hdiffz... but yeah I'm getting 20MB my way ;)
19:50:04lukash984 quits [Quit: The Lounge - https://thelounge.chat]
19:53:43<PC>klea: i'd really appreciate that!
19:54:06<klea>I'm waiting for the urls i queued myself there to finish, then i'll start it.
19:54:10<PC>thank youu
19:54:14<klea>you're welcome.
19:57:16<klea>aaa i hate zsh/sh shells
20:02:58nine quits [Quit: See ya!]
20:03:12nine joins
20:03:12nine quits [Changing host]
20:03:12nine (nine) joins
20:19:24datechnoman quits [Quit: The Lounge - https://thelounge.chat]
20:21:16<h2ibot>Calmevening edited Android Applications/https (+366): https://wiki.archiveteam.org/?diff=60327&oldid=58203
20:22:16<h2ibot>Calmevening edited Android Applications/https (+8): https://wiki.archiveteam.org/?diff=60328&oldid=60327
20:24:16<h2ibot>Calmevening edited IOS (-90, removed not ios related link): https://wiki.archiveteam.org/?diff=60329&oldid=58945
20:34:45Cornelius quits [Quit: Cornelius]
20:35:32Cornelius (Cornelius) joins
20:45:58APOLLO03 quits [Ping timeout: 256 seconds]
21:07:44Island joins
21:14:29datechnoman (datechnoman) joins
21:25:21DopefishJustin quits [Remote host closed the connection]
21:31:42DopefishJustin joins
21:33:18Webuser937457 joins
21:33:35Webuser937457 quits [Client Quit]
21:35:32Webuser163948 joins
21:35:51Webuser163948 quits [Client Quit]
21:52:41SootBector quits [Remote host closed the connection]
21:53:51SootBector (SootBector) joins
21:59:25ATinySpaceMarine quits [Client Quit]
22:12:00<nukke>Is there a project for the Minnesota protests or ICE in general?
22:12:00APOLLO03 joins
22:12:11midou joins

HomeSearchPrevious dayNext day