| 00:01:36 | | qw3rty_ quits [Read error: Connection reset by peer] |
| 00:02:15 | | qw3rty_ joins |
| 00:58:13 | | DogsRNice quits [Read error: Connection reset by peer] |
| 01:29:24 | <nicolas17> | https://catalogd.archive.org/log/4569979369 this task has said "TASK FINISHED WITH SUCCESS" for 47 minutes now but it's still listed as a running task in the task list |
| 01:38:25 | <nicolas17> | unstuck now |
| 02:14:05 | | Doran is now known as Doranwen |
| 04:07:16 | <HP_Archivist> | Hey JAA - quick question. When you mentioned a few weeks ago about creating hash values on a per txt basis, did you mean on a per file basis or per item basis? |
| 04:09:34 | <@JAA> | HP_Archivist: Uh, I don't remember the exact context, but usually, I go with items. |
| 04:10:03 | <@JAA> | That is, one file contains the hashes for all files in one item. |
| 04:10:22 | <@JAA> | That's probably what I meant then, too. |
| 04:11:11 | <HP_Archivist> | Yeah, context was that on my first attempt - I created on big txt with all hashes in it vs your suggestion. But you never mentioned if it was per item (until now) |
| 04:12:14 | <HP_Archivist> | Per file seemed like it wasn't likely anyway now that I think about it. That would be hundreds of thousands of files |
| 04:23:40 | <@JAA> | A single file for all items works, too, assuming your paths are then `item-identifier/file`, not just `file`. |
| 04:24:08 | <@JAA> | You just need to account for it on generating the comparison file from IA. |
| 04:27:39 | | BearFortress quits [Ping timeout: 260 seconds] |
| 04:35:12 | <HP_Archivist> | JAA you mentioned (then) that a single file would be messy. But maybe that's because of my paths at the time. Everything now is 1:1 to how it is IA-side. `item-identifier/file' or `item-identifier/folder/file' |
| 04:36:21 | <HP_Archivist> | Although, I guess that means 22K txts... heh |
| 04:36:37 | <HP_Archivist> | Hm |
| 04:36:51 | <@JAA> | Yeah, I think you didn't have the identifier in the paths at the time, so a single file would've been broken with that. |
| 04:37:14 | <HP_Archivist> | Yup ^^ |
| 05:15:49 | <nicolas17> | btw |
| 05:15:57 | <nicolas17> | "net.core.default_qdisc=fq net.ipv4.tcp_congestion_control=bbr" alone helped with my upload speeds |
| 05:16:08 | <nicolas17> | later I should try if just one of those two is the key |
| 05:19:17 | <@JAA> | It's likely that the buffer stuff only matters for high speeds. |
| 05:21:37 | <HP_Archivist> | JAA: Can iasha1check handle './' ? |
| 05:23:30 | | BearFortress joins |
| 05:24:41 | <@JAA> | HP_Archivist: I think you're still misunderstanding what `iasha1check` does. |
| 05:26:33 | <@JAA> | It takes one argument, the item name. It then compares the IA item of that name to the directory you're running it in. |
| 05:26:46 | <@JAA> | It runs sha1sum itself; it doesn't care about any hash files you already generated. |
| 05:37:01 | <HP_Archivist> | JAA: Yeah, I just realized that. Again. Idk I keep forgetting. Thanks for the reminder ^ |
| 05:37:42 | <@JAA> | When you hash it yourself, you'll want to run `diff` or whatever to compare it against a list you generate from IA's metadata. |
| 05:38:55 | <@JAA> | And the generation is easiest with jq as shown last time we spoke about it. `iasha1check` is insane by comparison; it processes XML with regex. |
| 05:39:16 | <@JAA> | It does work though, so whatever. :-P |
| 05:39:28 | <@JAA> | Maybe I'll rewrite it with jq sometime. |
| 05:49:26 | <HP_Archivist> | I'll stick with letting iasha1check (and by that I mean, sha1sum) do the hashing, much easier :) |
| 06:21:46 | <HP_Archivist> | Got it up and running. I actually modified it to ignore the overall item "FAIL" warning, since it will always do that per it failing on the derivatives. |
| 06:22:30 | <HP_Archivist> | https://transfer.archivete.am/FCbnj/hp_archivists-iasha1check-hash-custom-arg-output.txt |
| 06:22:31 | <eggdrop> | inline (for browser viewing): https://transfer.archivete.am/inline/FCbnj/hp_archivists-iasha1check-hash-custom-arg-output.txt |
| 06:22:58 | <HP_Archivist> | JAA Have a look. I doubled checked that it won't modify each individual's status, OK or FAIL. Another set of eyes wouldn't hurt though |
| 06:24:07 | <HP_Archivist> | The FAIL in red in the terminal is a bit sightly (even though the files are so far all OK). I wanted to have it say 'Success' since the important files are passing. |
| 06:28:00 | <@JAA> | HP_Archivist: Yes, the script deliberately filters out derivatives and fails loudly to grab attention. I'd copy and modify the script rather than trying to filter the output afterwards, but that's up to you. |
| 08:48:30 | | fuzzy80211 (fuzzy80211) joins |
| 08:49:05 | | fuzzy8021 quits [Read error: Connection reset by peer] |
| 09:17:00 | | fuzzy80211 quits [Read error: Connection reset by peer] |
| 09:17:28 | | fuzzy80211 (fuzzy80211) joins |
| 14:22:46 | | datechnoman quits [Quit: The Lounge - https://thelounge.chat] |
| 14:24:04 | | datechnoman (datechnoman) joins |
| 15:05:33 | | MrMcNuggets (MrMcNuggets) joins |
| 15:30:53 | | HP_Archivist quits [Quit: Leaving] |
| 15:42:29 | | IDK (IDK) joins |
| 15:43:59 | | HP_Archivist (HP_Archivist) joins |
| 15:55:14 | | qwertyasdfuiopghjkl quits [Quit: Client closed] |
| 16:14:34 | | qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins |
| 16:25:35 | <HP_Archivist> | RE: custom script output for iasha1check |
| 16:26:45 | <HP_Archivist> | JAA - I had gone to bed. Thanks. So you're saying modify the actual script (iasha1check) vs edit the output. This makes sense. I wouldn't want some custom output to get in the way of indicating a file passed (when it really didn't.) |
| 16:52:25 | | MrMcNuggets quits [Client Quit] |
| 18:42:36 | <@JAA> | Ah, today is 'someone messed up the deploy and the WBM is giving 404s on scripts, breaking everything' day again. :-) |
| 19:42:19 | | IDK quits [Client Quit] |
| 21:09:46 | | JaffaCakes118 (JaffaCakes118) joins |
| 22:34:45 | | SootBector quits [Ping timeout: 240 seconds] |
| 22:36:32 | | SootBector (SootBector) joins |
| 22:40:47 | | SootBector quits [Remote host closed the connection] |
| 22:41:03 | | SootBector (SootBector) joins |