| 00:10:11 | | torv quits [Remote host closed the connection] |
| 00:22:37 | | torv (torv) joins |
| 00:39:37 | | kdqep (kdqep) joins |
| 02:26:08 | | JackThompson quits [Quit: The Lounge - https://thelounge.chat] |
| 02:38:14 | | torv_ (torv) joins |
| 02:39:53 | | torv quits [Ping timeout: 252 seconds] |
| 05:15:22 | | kdqep quits [Read error: Connection reset by peer] |
| 05:31:15 | | kdqep (kdqep) joins |
| 06:27:23 | | Iki1 joins |
| 06:31:06 | | Iki quits [Ping timeout: 240 seconds] |
| 12:04:13 | | torv_ is now known as torv |
| 14:39:23 | | JackThompson joins |
| 15:54:14 | | AnotherIki joins |
| 15:58:16 | | Iki1 quits [Ping timeout: 265 seconds] |
| 17:19:53 | | HackMii quits [Ping timeout: 252 seconds] |
| 17:21:38 | | HackMii (hacktheplanet) joins |
| 19:58:15 | | Zerote__ joins |
| 19:58:15 | | kdqep_ joins |
| 19:58:15 | | yano1 (yano) joins |
| 19:58:15 | | yano quits [Read error: Connection reset by peer] |
| 20:00:17 | | kdqep quits [Ping timeout: 265 seconds] |
| 20:00:17 | | Zerote_ quits [Ping timeout: 265 seconds] |
| 20:43:57 | | f00860 joins |
| 20:44:04 | <f00860> | hi |
| 20:44:24 | <f00860> | is there a way to increase the number of concurrent threads above 6? |
| 20:45:00 | <@JAA> | Not with the warrior, no. You'd have to run the project Docker images instead. |
| 20:45:42 | <f00860> | i use the atdr.meo.ws/archiveteam/warrior-dockerfile image with the CONCURRENT_ITEMS parameter |
| 20:45:58 | <f00860> | but I can't increase it above 6 |
| 20:46:08 | <@JAA> | Those have a higher limit of 20. Beyond that, you'd need to run multiple containers. Note that per-IP rate limits are common, so you can't blindly throw very high concurrencies at all projects. |
| 20:46:48 | <@JAA> | The project images are atdr.meo.ws/archiveteam/${project}-grab normally. E.g. reddit-grab for the Reddit project. |
| 20:47:09 | <@JAA> | (Names match those of the GitHub repositories, which are usually listed on the wiki.) |
| 20:50:00 | <f00860> | ah i see |
| 20:50:01 | <f00860> | is there a list of projects? |
| 20:50:15 | <f00860> | (especially the names of the images) |
| 20:50:27 | <@JAA> | Not really, no. |
| 20:50:54 | <@JAA> | The wiki homepage, but it's often not quite complete. |
| 20:57:36 | <f00860> | a search for "-grab" on the github org is fine :) |
| 20:58:14 | <f00860> | is there a way to activate the web ui for the specific project grab image? |
| 21:00:19 | | yano1 is now known as yano |
| 21:00:45 | <@JAA> | There are many old -grab repos though. That doesn't tell you which ones are still active. |
| 21:01:24 | <f00860> | ok |
| 21:01:39 | <@JAA> | Project images don't have the warrior WUI, and I think they're configured to not have any web interface by default. |
| 21:01:56 | | kdqep (kdqep) joins |
| 21:01:59 | <f00860> | ok thanks |
| 21:03:36 | <@JAA> | From memory, I think you'd have to override the entrypoint. You'd also need to expose the port to be able to actually access it, plus probably remap it for every container so they don't collide, etc. Quickly becomes a pain and is not really useful for scaling, which is why it's disabled by default I believe. |
| 21:05:11 | | kdqep_ quits [Ping timeout: 265 seconds] |
| 22:15:09 | | HackMii quits [Remote host closed the connection] |
| 22:16:24 | | HackMii (hacktheplanet) joins |
| 22:50:54 | | f00860 quits [Remote host closed the connection] |