HomeSearchPrevious dayNext day

00:00:00<eggdrop>[remind] arkiver: pabs3 account issue :)
00:23:15xkey quits [Quit: WeeChat 4.9.1]
00:23:27xkey (xkey) joins
00:28:36<pabs>JAA: what are revisit records? and why is the first text/html 301 on 20231222143008 *after* the three revisit records?
00:30:22<@JAA>pabs: A revisit record in a WARC is a deduplicated reference to a previous response, almost always when the payload (decoded HTTP body) is identical.
00:31:40<@JAA>In this case, at least now, http://git.exotic.sh/ returns a standard nginx response for 301s, so it was deduped against some other similar record on a different URL in those first captures.
00:32:14<pabs>huh, so dedups are archive-wide?
00:32:30<pabs>or just around the same time or in the same WARC?
00:32:45<@JAA>No, it's something the software writing those WARCs did.
00:32:53<pabs>oh
00:32:57<pabs>thanks
00:33:00<pabs>JAA++
00:33:01<eggdrop>[karma] 'JAA' now has 396 karma!
00:33:26<@JAA>wget-at and qwarc dedupe within the WARC(s) written by a single process, for example.
00:34:16<@JAA>wget also has a mechanism for loading a CDX for deduping across multiple runs, but IIRC, that's somewhat broken, and it's not really used by anything at AT at least.
00:34:37<@JAA>Other tools may have everything backed by a database or similar.
01:23:23<pabs>Jake: repeated the SPN of https://savannah.gnu.org/news/ and the favicon snapshot thing didn't happen this time, hmm
03:01:58DogsRNice joins
04:47:13DogsRNice quits [Read error: Connection reset by peer]
05:16:47atphoenix_ (atphoenix) joins
05:19:34atphoenix__ quits [Ping timeout: 268 seconds]
08:35:20tzt quits [Remote host closed the connection]
08:35:40tzt (tzt) joins
09:10:08KoalaBear quits [Read error: Connection reset by peer]
09:12:26KoalaBear joins
12:17:22Yakov8 (Yakov) joins
12:19:31Yakov quits [Ping timeout: 268 seconds]
12:19:32Yakov8 is now known as Yakov
16:19:49fuzzy80211 quits [Killed (NickServ (GHOST command used by fuzzy8021))]
16:19:59fuzzy80211 (fuzzy80211) joins
16:20:32fuzzy80211 quits [Excess Flood]
16:20:54fuzzy80211 (fuzzy80211) joins
16:21:27fuzzy80211 quits [Excess Flood]
16:21:49fuzzy80211 (fuzzy80211) joins
16:22:21fuzzy80211 quits [Excess Flood]
16:22:43fuzzy80211 (fuzzy80211) joins
16:23:17fuzzy80211 quits [Excess Flood]
16:23:25DogsRNice joins
16:23:39fuzzy80211 (fuzzy80211) joins
16:24:11fuzzy80211 quits [Excess Flood]
16:24:33fuzzy80211 (fuzzy80211) joins
16:25:06fuzzy80211 quits [Excess Flood]
16:25:28fuzzy80211 (fuzzy80211) joins
16:26:01fuzzy80211 quits [Excess Flood]
16:26:22fuzzy80211 (fuzzy80211) joins
16:26:54fuzzy80211 quits [Excess Flood]
16:27:16fuzzy80211 (fuzzy80211) joins
16:27:49fuzzy80211 quits [Excess Flood]
16:28:11fuzzy80211 (fuzzy80211) joins
16:28:43fuzzy80211 quits [Excess Flood]
16:29:04fuzzy80211 (fuzzy80211) joins
16:29:36fuzzy80211 quits [Excess Flood]
16:29:58fuzzy80211 (fuzzy80211) joins
16:30:30fuzzy80211 quits [Excess Flood]
16:30:52fuzzy80211 (fuzzy80211) joins
16:31:25fuzzy80211 quits [Excess Flood]
16:31:46fuzzy80211 (fuzzy80211) joins
18:52:06Starchives__ (Starchives) joins
18:56:02Starchives_ quits [Ping timeout: 268 seconds]
19:52:06ThreeHM quits [Ping timeout: 268 seconds]
19:52:59ThreeHM (ThreeHeadedMonkey) joins
20:12:27Lord_Nightmare quits [Ping timeout: 268 seconds]
20:51:55ThreeHM quits [Ping timeout: 268 seconds]
20:52:43ThreeHM (ThreeHeadedMonkey) joins
21:18:26<HP_Archivist>Hey JAA - hope you are well. I am curious to hear your thoughts about this. I posed a question in r/Archivists over the weekend, as I got to thinking about ways to better package data in individual items. Does anyone here follow BagIt hierarchies?
21:18:29<HP_Archivist>https://www.reddit.com/r/Archivists/comments/1tybvhd/bagit_10_specification_feedback/
21:22:13<klea>AFAIK AT mostly doesn't need to follow any specific way to package items since most of what AT generates is WARCs meant to be consumed by machines?
21:23:34<@JAA>HP_Archivist: I know that BagIt exists and that it's some sort of data container but nothing beyond that. So no thoughts on it.
21:24:21<HP_Archivist>klea - I think what the comment on my post was saying was, generally speaking, was "see if AT can offer guidance". That was my interpretation. By AT, I assume they meant, like, all things IA. Not just WARCs.
21:24:40<klea>Ah.
21:26:46<HP_Archivist>JAA yeah, no worries. It's basically just a standardized filesystem hierarchy packaged with checksum manifests/additional metadata. Was curious to see who here actually followed it, if anyone.
21:27:07<@JAA>Not that I've heard of or seen.
21:27:37<@JAA>But this is the IA channel, so I'm sure someone somewhere has uploaded something to IA following it. :-)
21:27:53<HP_Archivist>:)
21:29:14<klea>https://datatracker.ietf.org/doc/html/rfc8493#section-2.1 Seems easy to follow it :P
21:29:33<HP_Archivist>Lol
21:29:58<HP_Archivist>Yeah, it's very basic. But the naming convention and how folders are nested and with what *is* the standard, I guess
21:31:05<HP_Archivist>If you look at the LoC repo for it, no work has been done on it. But I guess it's more for DAMs focused people who are... like me.. and meticulous about conventions.
21:31:14<HP_Archivist>Done on it recently*
21:31:25<klea>DAM?
21:31:32<@JAA>TIL IANA has a registry for hash algorithms. And it's weird.
21:31:41<klea>Link?
21:31:48<@JAA>It has truncated versions of SHA-256. I don't think I've ever seen those.
21:31:57<@JAA>But no truncated versions of SHA-512, which are everywhere.
21:31:59<@JAA>https://www.iana.org/assignments/named-information/named-information.xhtml
21:32:00<klea>Also, requiring a fetch.txt with sources seems annoying to do.
21:32:06<HP_Archivist>Digital Asset Management System*
21:32:16<klea>Ah.
23:28:50nicolas17 (nicolas17) joins

HomeSearchPrevious dayNext day