| 00:52:03 | | Boltie quits [Quit: Leaving] |
| 01:11:20 | | Boltie (Boltie) joins |
| 03:20:22 | | Atom-- joins |
| 03:23:34 | | Atom quits [Ping timeout: 252 seconds] |
| 03:41:03 | | qw3rty_ joins |
| 03:44:44 | | qw3rty__ quits [Ping timeout: 250 seconds] |
| 03:50:09 | | Boltie2 (Boltie) joins |
| 03:51:52 | | Boltie quits [Ping timeout: 244 seconds] |
| 08:22:18 | | Jake quits [Client Quit] |
| 08:23:33 | | Jake (Jake) joins |
| 09:17:02 | | Atom-- quits [Read error: Connection reset by peer] |
| 09:17:14 | | Atom-- joins |
| 12:03:13 | | Matthww8 quits [Ping timeout: 244 seconds] |
| 12:15:38 | | Matthww8 joins |
| 15:16:23 | | flashmeow quits [Quit: ZNC 1.8.2 - https://znc.in] |
| 15:19:26 | | flashmeow (flashmeow) joins |
| 18:15:17 | <gazorpazorp> | Is it normal to have an error with a traceback after a minute or two of "No free items available currently." on the Warrior? I tried to copy it, but the item disappeared after a second or so |
| 18:15:35 | <gazorpazorp> | Warrior is currently working OK and I'm scraping |
| 18:16:11 | <[42]> | it should show you in the same line that it's expected |
| 18:16:21 | <[42]> | or possible the line after |
| 18:18:11 | <gazorpazorp> | Sorry, I couldn't understand. It should show me what (the traceback) in the same line as what? I got the traceback (which is 10+ lines) after I got the "no free items" error a few times and then the item disappeared (just like the items that finish normally) |
| 18:18:37 | <gazorpazorp> | I'll try to copy everything if I catch it again |
| 18:19:21 | <gazorpazorp> | Or "excepted" as in "exception"/error? |
| 18:20:36 | <[42]> | i would assume you're seeing something like this: Error communicating with tracker: 404 Client Error: No free items available currently. Don't worry; this is normal. You will be assigned items soon. for url: https://tracker.archiveteam.org:1338/api/get. |
| 18:21:01 | <[42]> | i don't get exceptions for that, just the "error" message |
| 18:21:23 | <[42]> | i'm running pipeline though, not warrior, idk if that makes a difference |
| 18:21:23 | <@JAA> | Yeah, exceptions sometimes happen with that, but you can ignore it I believe. |
| 18:21:34 | <@JAA> | It doesn't, I've seen it many times with the pipeline. |
| 18:25:43 | <gazorpazorp> | Also, is there a reason to prefer the console/pipeline/whatever to running separate Warrior projects for each AT project? |
| 18:26:16 | <gazorpazorp> | I started with the Warrior because it was easy to set up, but could switch |
| 18:33:12 | <[42]> | i run pipeline containers as i don't need capabilities to switch between projects inside the container and this should be less overhead |
| 18:33:28 | <[42]> | if i want to run a different project i just deploy a different pipeline container |
| 18:40:43 | <gazorpazorp> | I'm reading about it now. How do you see the output/progress? With `docker logs --tail 0 -f archiveteam. --tail 0`? |
| 18:41:29 | <[42]> | depends on how you run it and how you want to see it |
| 18:41:35 | <[42]> | pipeline can also run with webinterface |
| 18:42:54 | <[42]> | something to note, i think warrior updates the projects automatically, with the pipeline containers you need another way to make sure you're running the latest version |
| 18:43:04 | <[42]> | for docker you could look into e.g. watchtower |
| 18:44:47 | <gazorpazorp> | By "pipeline" do you mean the standard Docker container from https://wiki.archiveteam.org/index.php/Running_Archive_Team_Projects_with_Docker |
| 18:45:41 | <gazorpazorp> | So you don't manually use "run-pipeline", as in the https://github.com/ArchiveTeam/seesaw-kit example? (found it by searching for "pipeline" in the wiki) |
| 18:46:18 | <gazorpazorp> | Or that's used when developing only? Sorry if I ask lots of questions |
| 18:46:31 | <[42]> | yes, images like atdr.meo.ws/archiveteam/urls-grab or for this one atdr.meo.ws/archiveteam/terroroftinytown-client-grab |
| 18:49:10 | <gazorpazorp> | Thanks. For some reason I thought there was a non-containerized version (not just for xtube) that was better in certain ways |
| 19:00:31 | <gazorpazorp> | Here's the traceback: https://transfer.archivete.am/ROnJi/1 |
| 19:12:55 | <@JAA> | Yep |
| 19:13:09 | <@JAA> | Just looked it up in the code, totally expected. |
| 19:13:19 | <@JAA> | It happens when five consecutive attempts return 404. |
| 19:13:33 | <@JAA> | (Consecutive within one pipeline item = one concurrency) |
| 19:13:51 | <@JAA> | https://github.com/ArchiveTeam/terroroftinytown-client-grab/blob/b9421f4de7c65984f3beef0dabc3a849413ab09e/scraper.py#L73-L74 |
| 19:16:12 | <@JAA> | Or well, when five attempts fail, but mostly it'll be 404s. |
| 19:23:00 | <gazorpazorp> | I see thanks for the explanation. Glad it's expected |
| 22:30:30 | | gazorpazorp quits [Changing host] |
| 22:30:30 | | gazorpazorp (gazorpazorp) joins |
| 22:32:23 | | driib6 (driib) joins |
| 22:36:10 | | driib quits [Ping timeout: 250 seconds] |
| 22:36:10 | | driib6 is now known as driib |