| 01:12:14 | | steering quits [Quit: ZNC - https://znc.in] |
| 01:12:28 | | steering (steering) joins |
| 05:13:21 | | DogsRNice quits [Read error: Connection reset by peer] |
| 05:21:58 | | nepeat quits [Quit: ZNC - https://znc.in] |
| 05:25:23 | | nepeat (nepeat) joins |
| 05:31:39 | | sg72 quits [Remote host closed the connection] |
| 05:32:49 | | sg72 joins |
| 05:33:08 | | nepeat quits [Client Quit] |
| 05:33:33 | | nepeat (nepeat) joins |
| 06:05:31 | <pabs> | https://www.betaarchive.com/wiki/ - cloudflare |
| 07:07:05 | | nepeat quits [Client Quit] |
| 07:09:10 | | nepeat (nepeat) joins |
| 16:01:57 | <justauser> | CF bypassed, but I don't have enough space on the server. |
| 16:02:43 | <justauser> | Anybody with a good idea how to move 40GB of images to IA, 7zipped, when only 10GB of scratch space are available? |
| 16:21:59 | <justauser> | Workstation is not much better with about 60GB. Pondering archiving to a pipe over SSH, but this has no error recovery. |
| 16:35:41 | <@arkiver> | i believe JAA has something to upload in chunks, but i also believe it's relatively espensive on the side of IA, so should be used sparingly |
| 16:39:38 | <TheTechRobo> | 7zip allows you to add to an existing archive, right? |
| 16:39:52 | <TheTechRobo> | So I'd add a bunch, delete the original files, and repeat. |
| 16:40:20 | <TheTechRobo> | ia-upload-stream (if that is what ark.iver is referring to) requires the whole file to already exist, so I don't think that'd help here? |
| 16:40:24 | <justauser> | Neat idea. |
| 16:40:36 | <justauser> | TheTechRobo++ |
| 16:40:36 | <eggdrop> | [karma] 'TheTechRobo' now has 23 karma! |
| 16:52:28 | <justauser> | Has been cleaning my tabs up and found this: |
| 16:52:29 | <justauser> | https://doc.eedomus.com/en/index.php/Main_Page |
| 16:52:47 | <justauser> | It's not in my IRC logs. It's not something I was looking for. |
| 16:52:59 | <justauser> | Yet it's open as view-source. |
| 16:55:43 | <justauser> | Mystery solved. It's a Wikibot failure before I had IRC. |
| 17:17:11 | <@JAA> | ia-upload-stream can upload from a stream, as its name suggests. But in that mode, it needs to be buffered whole in memory. |
| 17:17:41 | <@JAA> | (The mode to upload from a file on disk is much newer, actually.) |
| 17:35:36 | | DogsRNice joins |
| 17:39:21 | <justauser> | TheTechRobo: Nope. It looks like 7z does atomic updates with a temp file, which needs (old-size + new-size) at peak. |
| 17:45:42 | | sg72 quits [Remote host closed the connection] |
| 17:48:33 | | sg72 joins |
| 18:27:19 | <TheTechRobo> | JAA: Right, but can 7z output to a stream? |
| 18:28:09 | <TheTechRobo> | Ah, apparently not with the 7z format, only with e.g. tar |
| 18:31:13 | <justauser> | I'll probably ship as multiple archives. A to D done, getting is off the server. |
| 18:31:18 | <justauser> | *it |
| 18:36:12 | | @JAA hasn't used the 7z format much. |
| 18:37:14 | <@JAA> | But yeah, not too surprised. Almost none of these container formats are structured for streaming. And tar only is because it's specifically designed for tapes. |
| 18:38:48 | <justauser> | ZIP is designed to work with "windowed" data access. |
| 18:39:06 | <justauser> | I.e. only one floppy in a drive at time. |
| 18:39:26 | <justauser> | RAR inherited much of this design AFAIK. |
| 18:41:02 | <justauser> | One of the consequences is that you can "delete" a file without touching the volume it actually lives at, by changing the Central Directory. |
| 18:42:14 | <justauser> | ZPAQ has an explicit support for stream-writing. |
| 18:42:15 | <justauser> | That is, write a chunk, send it off to the permanent storage and never need it again unless you decide to unpack. |
| 18:50:34 | <@JAA> | Yeah, there are tools for streaming ZIPs, too. Although you obviously need to keep all the file metadata in memory until the end so you can write the central directory. |
| 18:50:52 | <@JAA> | tar is unique in that you can just process the input files one-by-one and don't need to maintain any state. |
| 18:55:48 | <justauser> | You sorta can with ZIP. That's what 7z does. |
| 18:56:08 | <justauser> | Ignore the central directory and parse file headers. |
| 19:19:32 | <@JAA> | The central directory is mandatory for a valid ZIP file, no? |
| 19:48:33 | | leo60228 quits [Remote host closed the connection] |
| 19:49:04 | | leo60228 (leo60228) joins |
| 21:43:29 | | @rewby quits [Ping timeout: 272 seconds] |
| 21:51:25 | | rewby (rewby) joins |
| 21:51:25 | | @ChanServ sets mode: +o rewby |