HomeSearchPrevious dayNext day

00:25:25<@JAA>https://github.com/ArchiveTeam/ArchiveBot/issues/564
00:31:52lennier2 quits [Read error: Connection reset by peer]
00:32:10lennier2 joins
00:32:14<fireonlive>akamai bad
00:36:29marktheworst joins
00:37:07<marktheworst>Hi, I was the one who asked about divested.dev's exclusion a couple days ago or so
00:37:20<marktheworst>would like to note that the AB snapshot showed up for opinionplatform.org but still no snapshots for divested
00:37:26<marktheworst>on WBM
00:38:31icedice (icedice) joins
00:39:24ThreeHM quits [Remote host closed the connection]
00:39:52ThreeHM (ThreeHeadedMonkey) joins
01:20:44nulldata quits [Ping timeout: 255 seconds]
01:30:22nulldata (nulldata) joins
01:31:03marktheworst quits [Remote host closed the connection]
01:31:03qwertyasdfuiopghjkl quits [Remote host closed the connection]
01:31:22qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
01:31:56marktheworst joins
02:04:41cascode quits [Read error: Connection reset by peer]
02:06:54cascode joins
02:12:07<anarcat>JAA: do we have igsets for drupal sites?
02:13:10<@JAA>anarcat: Funny timing, I just explained an hour ago in #archivebot why we don't. :-P
02:13:48<@JAA>The ignore depends on the Drupal version as well as the path prefix. So an igset isn't really very possible.
02:14:03<anarcat>oic
02:14:07<anarcat>to pabs i bet :p
02:14:25<@JAA>AB does actually have code to deal with that mess, but it's broken. :-/
02:15:35<fireonlive>:(
02:16:03<fireonlive>me 🤝 AB's code
02:16:05cascode quits [Ping timeout: 255 seconds]
02:17:28cascode joins
02:33:51<marktheworst>Since divested.dev appears to have some sort of time exclusion but not an explicit whole site exclusion, should it go on the WBM exclusion list wiki?
02:36:58cascode quits [Read error: Connection reset by peer]
02:37:27cascode joins
03:10:55qwertyasdfuiopghjkl quits [Remote host closed the connection]
03:10:56marktheworst quits [Remote host closed the connection]
03:11:38qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
03:11:57marktheworst joins
03:16:25DogsRNice joins
03:19:43ace24x (ace24x) joins
03:32:23DogsRNice_ joins
03:35:44DogsRNice quits [Ping timeout: 255 seconds]
03:44:17<h2ibot>PaulWise edited Deathwatch (+474, Open Collective Foundation shutting down end of…): https://wiki.archiveteam.org/?diff=51805&oldid=51800
04:03:27<pabs>JAA: can you shove Mannie's list above (maybe fix the merged URL) into queueh2ibot? then I'll try to do subdomains for them after lad's ones
04:04:04<pabs>AB needs some more capacity :)
04:04:17<fireonlive>soon: waterpipe
04:04:20<fireonlive>maybe
04:04:21<fireonlive>lol
04:04:37<pabs>:)
04:04:42<fireonlive>=]
04:04:56pabs ponders if SWH would fund one
04:05:03<fireonlive>ooh
04:08:59<pabs>I'm guessing not, didn't talk to them about it yet
04:09:40<@JAA>Seems too far away from their mission.
04:10:41<pabs>there was a comment in olasd's DebConf talk that made me think they might
04:10:52<fireonlive>oh?
04:11:27<pabs>something like we aren't working on archiving websites but we would like to suport that or something
04:11:39<@JAA>Interesting
04:11:41<fireonlive>ooh
04:12:06<pabs>it was towards the end IIRC if you want the exact wording
04:38:16<@JAA>pabs: So I can't easily stop what queueh2ibot is running through currently. :-/
04:38:36<pabs>ok, no worries
04:42:21marktheworst quits [Client Quit]
05:06:34hackbug quits [Remote host closed the connection]
05:07:02JohnnyJ quits [Quit: Ping timeout (120 seconds)]
05:15:12SF joins
05:15:18hackbug (hackbug) joins
05:26:26SF quits [Ping timeout: 255 seconds]
05:39:33SF joins
05:40:50DogsRNice_ quits [Ping timeout: 255 seconds]
05:43:44Island quits [Read error: Connection reset by peer]
05:46:59wickedplayer494 quits [Ping timeout: 272 seconds]
05:47:20wickedplayer494 joins
05:58:39qwertyasdfuiopghjkl quits [Ping timeout: 265 seconds]
06:05:05BearFortress joins
06:10:36BlueMaxima quits [Read error: Connection reset by peer]
07:22:47Maika (Maika) joins
07:39:19Arcorann (Arcorann) joins
07:54:34<@arkiver>the bot to queue items was killed due to memory usage. too much memory was consumed by other processes on the machine, not the bot
07:54:53<@arkiver>the bot it back up now
08:53:04Maika quits [Read error: Connection reset by peer]
09:12:47f_ quits [Ping timeout: 255 seconds]
09:28:15qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
09:58:54qwertyasdfuiopghjkl quits [Remote host closed the connection]
09:59:07qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
10:00:03Bleo18260 quits [Client Quit]
10:01:21Bleo18260 joins
10:16:08<c3manu>how much would a useful chunk of ab capacity roughly be per month?
10:16:25<c3manu>or does anyone know when the next "black friday style" offers will come around? ^^"
10:22:31BearFortress_ joins
10:26:35BearFortress quits [Ping timeout: 255 seconds]
10:53:44Wohlstand (Wohlstand) joins
10:53:47f_ (funderscore) joins
11:05:42BearFortress_ quits [Client Quit]
11:34:02Wohlstand quits [Client Quit]
11:38:54<AK>OOh that's one I can answer, (I run the ak-was-here-hel3 + ak-was-here-hel4 boxes), lemme find some numbers
11:39:23<AK>(Pay for them, JA_A does all the hard work for me)
11:40:42<AK>So we've got one AX41-NVMe and one EX42 NVMe, each one is ~€45 per month at the moment
11:42:20<AK>🤔Hel3 is the EX42 which only has 8 cores vs hel4's 12
11:48:38<c3manu>thanks! i've not been good with financial decisions lately, but in principle i should be able to handle at least one of those (i'd be starting with only one either way)
11:51:47<c3manu>i'll think this through for a bit. i'm on vacation in a month from now so i would have time to set it up then.
11:51:55<c3manu>i think that's a plan :)
12:02:48tertu2 (tertu) joins
12:03:20tertu quits [Ping timeout: 255 seconds]
12:24:58<katia>fwiw, the EX44 seem to be a great deal, even better than the AX41-NVMe wrt. CPU perf
12:25:02<katia>if you can justify the setup fee
12:31:06<murb>they often have promos for free setup.
12:36:18<katia>r
12:36:20<katia>ops
12:42:02sarge quits [Ping timeout: 255 seconds]
12:51:57Arcorann quits [Ping timeout: 272 seconds]
13:29:30Maika (Maika) joins
13:30:45mr_sarge (sarge) joins
13:42:23<AK>Yeah I'm tempted to look at replacing/upgrading hel3. But the drawdown time on pipelines can be up to 6 months, so I'd basically need to deploy hel8 and then wait 6 months to cancel hel3 which is quite a cost
14:01:58<imer>might be worth setting it up in a container/vm so you can just migrate it next time?
14:03:35<AK>That's probably a JAA question on how easy it would be to pause/migrate a running pipeline. I suspect the answer is "Not at all"
14:04:17<AK>I knew when I offered these servers that I've gotta keep them going for months (And now years) because some of the jobs can take months
14:04:46BearFortress joins
14:08:07Wohlstand (Wohlstand) joins
14:10:02<imer>as in qemu does live migrates, so the vm won't even notice it got moved
14:11:46icedice quits [Client Quit]
14:16:50<murb>how does a pipeline deal with powerfailure?
14:17:06<murb>or a reboot for other (important) reasons?
14:36:37icedice (icedice) joins
15:01:38Maika quits [Client Quit]
15:03:01<kiska>It doesn';t
15:05:13<imer>oof.
15:15:01<murb>so no checkpointing / saving of state?
15:19:23<kiska>Nope
15:23:25<kiska>Here is the former kiskaVultrJP instance: Linux vultr.guest 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u5 (2019-08-11) x86_64 GNU/Linux
15:23:48<kiska>Its running Debian 9.9 and has: 15:22:36 up 328 days, 4:16, 10 users, load average: 0.32, 0.22, 0.09
15:34:01<@arkiver>we now have a maximum to the number of concurrently running jobs by a queuing bot in a channel
15:34:15<@arkiver>see !help for details on the maximum
16:15:45Sanqui joins
16:15:47Sanqui quits [Changing host]
16:15:47Sanqui (Sanqui) joins
16:15:47@ChanServ sets mode: +o Sanqui
16:16:29@Sanqui quits [Remote host closed the connection]
16:19:44Sanqui joins
16:19:46Sanqui quits [Changing host]
16:19:46Sanqui (Sanqui) joins
16:19:46@ChanServ sets mode: +o Sanqui
16:21:52<AK>Hel3: 16:21:13 up 310 days, 22:45, 13 users, load average: 5.16, 5.51, 5.80
16:21:52<AK>Hel4: 16:21:35 up 593 days, 18:08, 17 users, load average: 8.88, 9.16, 8.89
16:22:01<AK>I win :P
16:53:18<kiska>Congrats! :D
16:59:53mr_sarge quits [Ping timeout: 255 seconds]
17:33:09SF quits [Ping timeout: 272 seconds]
17:34:51<nicolas17>c3manu: are you married? these servers tend to have a pretty bad Spouse Acceptance Factor
17:35:04SF joins
17:35:51<myself>So you're saying I'm the perfect candidate...
17:36:15<fireonlive>what’s “married”?
17:43:43<c3manu>nicolas17: no, i’m not ^^ but that also means my tax bracket is called "get fucked"
18:04:12<fireonlive>o3o
18:08:02<murb>couples are taxed at a lower rate? that seems strnage, surely they save money with economies of scale etc?
18:09:21<katia>i think different countries have different implementations of that, but it does seem weird
18:30:09razul quits [Ping timeout: 272 seconds]
18:33:13benjins2 quits [Quit: Leaving]
18:40:10razul joins
18:42:38qwertyasdfuiopghjkl quits [Client Quit]
18:52:39Island joins
18:54:37Maika (Maika) joins
18:57:45benjins2 joins
19:02:57Ketchup901 quits [Ping timeout: 255 seconds]
19:03:29Ketchup901 (Ketchup901) joins
19:11:16pixel leaves
19:11:17pixel (pixel) joins
19:18:28<c3manu>just reading up on that a bit, apparently i have a wrong understanding on how tax brackets work here. it's different brackets, yes, but that mostly enables a couple to shift things around a bit. but it's not a difference in principle, as i thought until now.
19:20:23<c3manu>so nevermind what i said
19:23:59<pokechu22>Yeah my understanding is it's (at least in the US) roughly just letting you choose to merge your money with your spouse and scale up the values for 2 people (though you can also choose to file separately in which case the brackets are for one person), but I haven't looked at exactly how it works. Also remember that getting bumped up a tax bracket generally only means you pay
19:24:01<pokechu22>more in tax for money over that point; tax bracket wise there's no situation where earning an additional dollar will result in an increase of taxes over a dollar (i.e. if you make more money, even after taxes you'll make more money)
19:27:34<fireonlive>had we not broken up we would have become "common law" automatically here...
19:27:42<fireonlive>(but also best to continue in -ot)
19:28:40<AK>I am squarely in the "As long as my girlfriend doesn't know exactly how much I spend then I'll be okay" camp
19:33:22<fireonlive>;)
19:35:15<fireonlive>if it ever comes up just say 'hey at least it's not heroin or whatever'
20:12:48<@JAA>AK: Rookie numbers
20:13:03<@JAA> 20:12:59 up 2443 days, 1:52
20:14:13<@JAA>imer: IP will change though, unless you route everything through another box, which may cause pain.
20:15:07<fireonlive>everyone cross your fingers for firepipe not losing power or whatver x3
20:22:13ace24x quits [Remote host closed the connection]
20:24:53BlueMaxima joins
20:32:42Ketchup901 quits [Remote host closed the connection]
20:33:00Ketchup901 (Ketchup901) joins
20:38:34Wohlstand quits [Client Quit]
20:44:08mr_sarge (sarge) joins
21:11:57BlueMaxima quits [Read error: Connection reset by peer]
21:17:00uu joins
21:17:27uu quits [Remote host closed the connection]
21:35:05cascode quits [Ping timeout: 272 seconds]
21:35:23cascode joins
21:40:07<imer>JAA: yeah, that's the one downside. there is some hackyness you can do (done that before), assuming it handles the network disconnect and having another ip for outgoing stuff just fine, reverse proxy any incoming (http?) connections to the new ip until dns updates
21:40:55<@JAA>imer: We're talking about AB pipelines, so the outgoing IP is what matters.
21:42:17<@JAA>Reason being URLs or cookies that are IP-bound, of course.
21:42:19<imer>is it? not familiar how that stuff works, I was assuming it just does a recursive crawl and then hands it off once finished, ip change shouldn't matter for most things there unless a site restricts session to the ip?
21:42:24<imer>beat me to it.
21:42:33<imer>makes sense
21:42:51<@JAA>It's basically impossible to know how much of a problem it would actually be without trying it out.
21:47:27<@JAA>Also, we did experiment with containers before. I wanted to use CRIU to resume things after host maintenance, and that worked fine in tests. The resumption never happened though for unrelated reasons.
21:54:21cascode quits [Read error: Connection reset by peer]
21:54:38cascode joins
22:09:00<NickS|m>Does anyone know if archivebot has grabbed https://dcist.com yet? Or how to find out?
22:10:01<@JAA>Yes, it was: https://archive.fart.website/archivebot/viewer/domain/dcist.com
22:10:41<NickS|m>Oh that's great, thank you!
22:13:12<Vokun>I know it would take quite some time to set up, (which doesn't seem like something we're ever going to have at this point) but would getting #Y running be better than another AB pipeline or two?
22:23:39Dalek quits []
22:26:35Dalek (Dalek) joins
23:00:43Guest54 joins
23:06:37Maika quits [Read error: Connection reset by peer]
23:39:02Guest54 quits [Ping timeout: 255 seconds]
23:43:36Guest54 joins
23:52:05fuzzy8021 quits [Ping timeout: 255 seconds]
23:52:33fuzzy8021 (fuzzy8021) joins
23:55:03ThreeHM quits [Ping timeout: 272 seconds]
23:56:15ThreeHM (ThreeHeadedMonkey) joins
23:59:20rohvani joins

HomeSearchPrevious dayNext day