| 00:05:42 | <myself> | BigBrain: Yeah after a while, if the browser suspends the tab but the VM keeps working in the background, the totals at the top can get out of sync, just refresh the view |
| 00:06:19 | <myself> | M--mlv|m Did yours return to functionality after stopping and restarting the VM? |
| 00:07:34 | <M--mlv|m> | yes but it was a while till I noticed something was amiss |
| 00:22:02 | | Dango360 quits [Ping timeout: 252 seconds] |
| 00:24:22 | | BearFortress joins |
| 00:55:26 | | sec^nd quits [Ping timeout: 245 seconds] |
| 00:55:43 | | sec^nd (second) joins |
| 02:52:54 | | JackThompson3 quits [Quit: The Lounge - https://thelounge.chat] |
| 03:44:42 | | JackThompson3 joins |
| 03:58:37 | | fireonlive quits [Excess Flood] |
| 03:59:39 | | fireonlive (fireonlive) joins |
| 05:49:12 | | trc (trc) joins |
| 06:30:52 | | fireonlive|s quits [Quit: quitters never quit] |
| 07:36:37 | | fireonlive quits [Client Quit] |
| 07:37:35 | | fireonlive (fireonlive) joins |
| 07:51:57 | | n9nes quits [Quit: ZNC 1.8.2 - https://znc.in] |
| 07:52:35 | | n9nes joins |
| 08:16:51 | | BigBrain quits [Remote host closed the connection] |
| 08:17:08 | | BigBrain (bigbrain) joins |
| 08:47:44 | | dave joins |
| 09:12:03 | | Aoede_ quits [Quit: ZNC - https://znc.in] |
| 09:12:21 | | Aoede (Aoede) joins |
| 09:21:10 | | Elizabeth (Elizabeth) joins |
| 09:27:36 | | Minkafighter quits [Quit: The Lounge - https://thelounge.chat] |
| 09:28:15 | | Minkafighter joins |
| 10:33:00 | | qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins |
| 10:38:45 | | Aoede quits [Client Quit] |
| 10:39:08 | | Aoede (Aoede) joins |
| 10:58:04 | | Dango360 (Dango360) joins |
| 12:15:01 | | BigBrain quits [Ping timeout: 245 seconds] |
| 12:33:11 | | BigBrain (bigbrain) joins |
| 14:20:55 | | diggan joins |
| 14:22:51 | <diggan> | any particular reason items in "Upload" isn't considered completed in terms of concurrency? For example, if I have 4 set as concurrency, and 3 are waiting for upload to work, it would maybe make sense to still allow 4 download tasks |
| 14:23:05 | <diggan> | or alternatively, split upload/download concurrency so people could configure it themselves |
| 14:32:58 | | icaotix|m joins |
| 16:12:28 | <diggan> | is my guess that https://github.com/ArchiveTeam/seesaw-kit is the current UI correct? |
| 16:13:12 | <diggan> | seems to be, but hasn't been updated since 2019 so doubting I'm looking in the right place |
| 16:28:06 | <BigBrain> | i think the pipeline is just consecutive: get item -> download item -> upload item -> confirm item. and concurrent is the number of concurrent pipelines that are running, not concurrent downloaders. |
| 16:35:46 | <diggan> | yeah, no, I've guessed as much too. But I'm wondering if there is any particular reason for that, rather than continuing with downloads even though upload is bottlenecked |
| 16:38:47 | <BigBrain> | easier to do, maybe data integrity. do not need async uploader and no communication between. read during imgur archival that IA is rarely the bottleneck |
| 16:40:21 | <DigitalDragon> | I think they also don't want to fill people's disks, or end up with a situation where a client goes down with hundreds/thousands of items downloaded that would then need to be re-downloaded by someone else |
| 16:42:12 | <@kaz> | diggan: seesaw-kit is the baseline for all of our projects (not just the UI) |
| 16:43:25 | <@kaz> | but we use things like wget-at (previously wget-lua) for heavy lifting of the scraping itself https://github.com/ArchiveTeam/wget-lua |
| 16:44:43 | <@kaz> | as for your first question - an upload is not complete because the data is not 'safe' yet. Yes, it would be nice to have improvements there so that you could start on a new job before that upload finishes as you suggest |
| 16:48:26 | <diggan> | ah coolio. Thanks kaz! Mainly asking about the UI as I'm trying to understand the WebSocket endpoint and how it knows which task is in which state, think I've figured it out |
| 17:54:34 | | n9nes quits [Ping timeout: 252 seconds] |
| 18:34:53 | | Meli quits [Ping timeout: 252 seconds] |
| 18:35:45 | | vegbrasil joins |
| 18:35:59 | <@JAA> | vegbrasil: https://github.com/ArchiveTeam/warrior-dockerfile/issues/56 |
| 18:36:23 | <@JAA> | The last couple comments summarise where we're at. |
| 18:36:36 | <vegbrasil> | JAA: thanks! |
| 18:37:36 | | that_lurker (that_lurker) joins |
| 18:37:59 | | Meli (Meli) joins |
| 18:43:39 | | that_lurker quits [Client Quit] |
| 18:43:59 | | that_lurker (that_lurker) joins |
| 18:55:44 | | beario_ joins |
| 18:56:18 | | threedeeitguy (threedeeitguy) joins |
| 19:00:48 | | imer (imer) joins |
| 20:02:40 | | diggan quits [Ping timeout: 265 seconds] |
| 20:13:20 | | BearFortress quits [Ping timeout: 252 seconds] |
| 20:43:34 | | Unholy2361 (Unholy2361) joins |
| 20:46:10 | | Iki1 quits [Ping timeout: 265 seconds] |
| 20:47:46 | | diggan joins |
| 21:00:53 | | vegbrasi_ joins |
| 21:01:09 | | Unholy2361 quits [Quit: The Lounge - https://thelounge.chat] |
| 21:01:34 | | Unholy2361 (Unholy2361) joins |
| 21:03:56 | | vegbrasil quits [Ping timeout: 252 seconds] |
| 21:05:35 | | vegbrasi_ quits [Ping timeout: 252 seconds] |
| 21:16:08 | | diggan quits [Ping timeout: 265 seconds] |
| 21:19:57 | | vegbrasil joins |
| 21:23:08 | | diggan joins |
| 21:25:17 | | n9nes joins |
| 21:29:04 | | vegbrasil quits [Ping timeout: 252 seconds] |
| 21:33:08 | | n9nes quits [Client Quit] |
| 21:34:17 | | n9nes joins |
| 21:51:31 | | vegbrasil joins |
| 21:56:11 | | vegbrasil quits [Ping timeout: 252 seconds] |
| 22:15:29 | | vegbrasil joins |
| 22:20:02 | | vegbrasil quits [Ping timeout: 252 seconds] |
| 22:22:00 | | vegbrasil joins |
| 22:29:56 | | vegbrasil quits [Ping timeout: 252 seconds] |
| 22:32:09 | | tzt (tzt) joins |
| 22:53:39 | | g0tmk joins |