HomeSearchPrevious dayNext day

00:05:38BlueMaxima joins
00:08:39Arcorann_ joins
00:13:03HP_Archivist quits [Read error: Connection reset by peer]
00:13:31HP_Archivist (HP_Archivist) joins
00:22:43benjins quits [Ping timeout: 258 seconds]
00:23:19benjins joins
00:24:15benjins quits [Remote host closed the connection]
00:32:21<@JAA>I tried to speed up the OpenGrey grab with more parallelism. Turns out that their OAI-PMH API does *not* scale at all. Welp, slow sequential iteration it is then.
00:32:39<@JAA>That'll probably take another 4 hours. After that, it should be much faster.
00:32:53HP_Archivist quits [Read error: Connection reset by peer]
00:33:03<@JAA>It's just the ListIdentifiers and ListRecords iteration that takes so long.
00:34:59HP_Archivist (HP_Archivist) joins
01:02:44dm4v joins
01:02:46dm4v quits [Changing host]
01:02:46dm4v (dm4v) joins
01:20:24onetruth quits [Read error: Connection reset by peer]
01:30:06HP_Archivist quits [Ping timeout: 250 seconds]
01:35:29webdownload quits [Remote host closed the connection]
01:40:56systwi quits [Ping timeout: 250 seconds]
02:02:14driib0 (driib) joins
02:04:20driib quits [Ping timeout: 250 seconds]
02:04:20driib0 is now known as driib
02:57:35Selanda joins
02:58:38benjins joins
03:00:19Selanda quits [Client Quit]
03:16:50qw3rty__ joins
03:20:35qw3rty_ quits [Ping timeout: 258 seconds]
03:25:10HP_Archivist (HP_Archivist) joins
03:29:45za3k joins
03:30:22<za3k>I made some software to back up files onto paper. It's still alpha, but if anyone wants to alpha-test the linux cli, let me know and go ahead. https://github.com/za3k/qr-backup/
03:47:11<@JAA>za3k: I assume you're aware of PaperBack, JAB Code, etc.?
03:47:25<za3k>PaperBack yes, JAB Code no
03:54:48<@JAA>There are quite a few other attempts as well. But obviously doesn't really work for large amounts of data.
03:56:12<@JAA>(Also, there are at least two different projects called 'paperback'.)
03:57:10<za3k>Yeah, this one is aimed more at being usable than packing data in super densely
04:10:57<za3k>I especially wanted to be able to restore with a webcam rather than a scanner
04:25:23<za3k>OK well, I don't have a bouncer on hackint yet, so laters
04:25:29za3k quits [Remote host closed the connection]
04:28:27DogsRNice quits [Read error: Connection reset by peer]
04:33:10<@JAA>To the surprise of absolutely noone, OpenGrey is slowing down more the further I'm getting with the pagination. It was at 50 % four hours ago (= about four hours after the start), now it's at 75 %... :-|
04:33:54<@JAA>That percentage's just the initial pagination of those endpoints, afterwards there are still the individual entry pages etc.
04:48:10godane2 joins
04:50:40godane1 quits [Ping timeout: 258 seconds]
05:03:20lun4 quits [Remote host closed the connection]
05:03:20ave quits [Remote host closed the connection]
05:36:35Arcorann_ is now known as Arcorann
05:51:43C4K3 joins
06:11:01AlsoHP_Archivist joins
06:11:01HP_Archivist quits [Read error: Connection reset by peer]
06:16:03AlsoHP_Archivist quits [Read error: Connection reset by peer]
06:16:26AlsoHP_Archivist joins
06:17:58pcr joins
07:10:42flashfire42 quits [Quit: Ping timeout (120 seconds)]
07:11:08flashfire42 (flashfire42) joins
07:38:11AlsoHP_Archivist quits [Ping timeout: 258 seconds]
07:51:21atphoenix quits [Read error: Connection reset by peer]
07:51:58rsn quits [Read error: Connection reset by peer]
07:52:01atphoenix (atphoenix) joins
07:52:17rsn joins
07:52:32marked quits [Quit: Ping timeout (120 seconds)]
07:52:52marked joins
08:15:20NIC007a83 joins
08:17:22NIC007a83 quits [Client Quit]
08:18:26systwi (systwi) joins
08:21:30driib quits [Ping timeout: 258 seconds]
08:25:04driib (driib) joins
09:34:16Wayward (wayward) joins
09:57:51Doranwen quits [Remote host closed the connection]
10:04:12Doranwen (Doranwen) joins
10:43:42ave (ave) joins
10:43:47lun4 (lun4) joins
10:59:49britmob quits [Ping timeout: 258 seconds]
11:07:15BlueMaxima quits [Read error: Connection reset by peer]
11:46:38genofire joins
13:23:01<masterX244>JAA: slowdown on pagination reminds me on tm-exchange. The pages further down got slower and slower back when i iterated down
13:59:11Daloader joins
14:01:45britmob joins
14:05:44hexa- quits [Quit: WeeChat 3.1]
14:19:05hexa- (hexa-) joins
15:27:38hexa- quits [Killed (andi- (maybe next year))]
15:28:03hexa- (hexa-) joins
15:28:32Arcorann quits [Ping timeout: 258 seconds]
15:47:30<@JAA>Pagination finished earlier, so it's now going through the records at 200 req/s. :-)
15:47:52<@JAA>Occasional disconnects and stuff, but mostly fine.
15:48:13<@JAA>ETA is 5 hours or so.
15:51:30<@JAA>And to give an idea of the impact of the ListRecords API: the request rate more than doubled when that pagination finished. (ListIdentifiers had finished earlier, so it was already doing records in parallel.)
16:03:17LeGoupil joins
16:03:24lun4 quits [Read error: Connection reset by peer]
16:03:24ave quits [Read error: Connection reset by peer]
16:05:02<@Kaz>chromewebstore channel pls?
16:07:28<@arkiver>#chromeweblore
16:40:34ave (ave) joins
16:40:37lun4 (lun4) joins
17:48:05AlsoHP_Archivist joins
18:25:56mutantmonkey quits [Remote host closed the connection]
18:26:13mutantmonkey (mutantmonkey) joins
19:01:21AlsoHP_Archivist quits [Client Quit]
19:01:39HP_Archivist (HP_Archivist) joins
19:11:46Daloader quits [Ping timeout: 250 seconds]
19:12:31DogsRNice (Webuser299) joins
19:24:09spirit quits [Client Quit]
19:45:42KRG` is now known as KRG
20:34:52Jake5 (Jake) joins
20:36:44Jake quits [Ping timeout: 258 seconds]
20:36:44Jake5 is now known as Jake
21:01:41LeGoupil quits [Client Quit]
21:05:15marked4 joins
21:07:02marked quits [Ping timeout: 250 seconds]
21:07:02marked4 is now known as marked
21:07:24Doranwen quits [Ping timeout: 258 seconds]
21:22:21Sylirana quits [Ping timeout: 244 seconds]
21:23:10Sylirana (Sylirana) joins
21:24:01Doranwen (Doranwen) joins
21:27:43<@JAA>Apart from a couple hundred errors, OpenGrey is done.
21:34:06Church joins
21:37:29Church quits [Client Quit]
21:39:00<@EggplantN>Hola all i will be less active over the next few weeks as i can only use 1 hand. Bad burns on my main right hand. if anyone needs me to do stuff annoy HCross or Kaz as they have my phone number
21:44:12Church joins
21:46:53Church quits [Client Quit]
21:47:05Church joins
21:55:02Church quits [Client Quit]
22:01:26Church joins
22:01:29Church quits [Client Quit]
22:07:40Church joins
22:08:22Church quits [Client Quit]
22:10:33godane2 quits [Client Quit]
22:10:44godane (godane) joins
22:16:31Church joins
22:16:44Church quits [Client Quit]
22:58:34HP_Archivist quits [Ping timeout: 258 seconds]
23:04:00HP_Archivist (HP_Archivist) joins
23:08:25HP_Archivist quits [Read error: Connection reset by peer]
23:08:49HP_Archivist (HP_Archivist) joins
23:12:09Lord_Nightmare quits [Quit: ZNC - http://znc.in]
23:19:00Lord_Nightmare (Lord_Nightmare) joins
23:36:31HP_Archivist quits [Ping timeout: 258 seconds]
23:40:41Church joins
23:41:07phiresky quits [Ping timeout: 258 seconds]
23:43:45phiresky joins
23:45:27Ruthalas2 (Ruthalas) joins
23:45:43Ruthalas quits [Ping timeout: 258 seconds]
23:45:43Ruthalas2 is now known as Ruthalas
23:47:50benjinsmith joins
23:48:01benjins quits [Ping timeout: 258 seconds]
23:50:17Church quits [Client Quit]
23:50:47Church joins
23:55:10Ruthalas quits [Ping timeout: 250 seconds]
23:55:49Ruthalas (Ruthalas) joins

HomeSearchPrevious dayNext day