HomeSearchPrevious dayNext day

00:02:53jtagcat quits [Client Quit]
00:03:05jtagcat (jtagcat) joins
00:13:37Jake (Jake) joins
00:14:16Arcorann (Arcorann) joins
00:16:48Jake quits [Client Quit]
00:22:16Jake (Jake) joins
00:26:38Jake quits [Client Quit]
00:29:50BigBrain_ quits [Remote host closed the connection]
00:30:26BigBrain_ (bigbrain) joins
00:41:45Jake (Jake) joins
02:07:49<nicolas17>2023-09-09 01:52:51 (1.36 MB/s) - Read error at byte 290689826387/484259014419 (Error decoding the received TLS packet.). Retrying.
02:07:57<nicolas17>using wget instead of curl just saved me
02:08:48<nicolas17>it resumed from where it failed
02:10:15<nicolas17>2023-09-09 01:52:39 (940 KB/s) - Read error at byte 38657556051/377308233499 (Error decoding the received TLS packet.). Retrying.
02:10:23<nicolas17>interesting that both downloads failed at similar times
02:20:50shinji257 quits [Client Quit]
02:26:18shinji257 (shinji257) joins
02:48:46sfsefse quits [Remote host closed the connection]
02:49:34eawfeaw joins
02:50:58atphoenix (atphoenix) joins
02:52:34<TheTechRobo>interesting, I've only ever seen TLS errors from wayback machine
03:00:16pseudorizer quits [Quit: ZNC 1.8.2 - https://znc.in]
03:00:57pseudorizer (pseudorizer) joins
03:05:39<nicolas17>yeah this was from archive.org items, not wbm
03:07:49DogsRNice_ joins
03:11:45DogsRNice quits [Ping timeout: 265 seconds]
03:26:04krvme joins
03:29:11decagon__ quits [Ping timeout: 252 seconds]
04:08:18railen63 quits [Ping timeout: 265 seconds]
04:31:04DogsRNice_ quits [Read error: Connection reset by peer]
04:35:47<pabs>does anyone know if AB/IA/WBM/something can save magnet links? this band publishes raw video files, audio stem files and fan-made bootleg videos via magnet: links https://kinggizzardandthelizardwizard.com/automation https://kinggizzardandthelizardwizard.com/bootlegger
04:39:17<pabs>not in danger (except if seeder counts go down), but stem files are so rarely published that it would be good to have them on IA
05:00:45etnguyen03 quits [Client Quit]
05:02:35Unholy236131661808515 (Unholy2361) joins
05:05:59Unholy23613166180851 quits [Ping timeout: 252 seconds]
05:05:59Unholy236131661808515 is now known as Unholy23613166180851
05:06:51<pabs>https://www.theregister.com/2023/09/08/atari_snaps_up_atariage/
05:25:47Unholy23613166180851 quits [Client Quit]
05:25:53Unholy23613166180851 (Unholy2361) joins
05:34:15<nicolas17>pabs: I was going to say no but it seems actually yes
05:34:33<nicolas17>"If a valid .torrent file is uploaded (e.g. through our Uploader) into an item, when that item is derived, we will instantiate a BitTorrent client (Transmission) and attempt to retrieve the Torrent. If the Torrent is successfully retrieved, its contents will be added to the item. ‘Valid’ in this case means, well-formed and seeded."
05:34:53<nicolas17>"Bonus feature: if you have only a magnet link, and not a Torrent file, you can create a dummy .torrent file by pasting that magnet link into a text file and naming it foo.torrent."
05:35:30<nicolas17>not sure if you can do multiple .torrent files per item this way
05:40:48<pabs>nice, so just write to a file and ia upload
05:41:05<nicolas17>worth noting this will not seed the *original* torrents
05:41:22pabs not sure about doing this with the current IA upload bandwidth issues hmm
05:41:34<nicolas17>the end result is equivalent to you downloading the torrent and then uploading the files
05:42:15<pabs>hmm, guess I should check the torrent are fully seeded before doing this...
05:42:34<nicolas17>it saves *you* a ton of bandwidth
05:42:58<pabs>yeah
05:43:24<pabs>where did you read that btw?
05:43:47<nicolas17>https://help.archive.org/help/archive-bittorrents/
05:44:07<nicolas17>the first half of this page is all about IA-generated torrents for items
05:44:41<nicolas17>later it talks about using torrents to upload, which is an entirely different feature
05:45:39<pabs>thanks, adding some TODO notes
05:57:02pabs goes to AB atariage stuff
06:23:32nicolas17 quits [Ping timeout: 252 seconds]
06:44:26dumbgoy quits [Ping timeout: 252 seconds]
07:01:28railen63 joins
07:05:01Unholy23613166180851 quits [Remote host closed the connection]
07:06:48Unholy236131661808515 (Unholy2361) joins
07:14:52nulldata quits [Ping timeout: 265 seconds]
07:17:54nulldata (nulldata) joins
07:20:39Exorcism|TheLounge is now known as Exorcism
07:21:52Exorcism quits [Excess Flood]
07:22:04Exorcism (exorcism) joins
07:38:33Island quits [Read error: Connection reset by peer]
08:13:24DLoader_ joins
08:13:24DLoader_ quits [Excess Flood]
08:13:50nulldata quits [Ping timeout: 265 seconds]
08:13:52DLoader_ joins
08:13:52DLoader_ quits [Excess Flood]
08:14:28DLoader_ joins
08:16:15DLoader quits [Ping timeout: 265 seconds]
08:16:20DLoader_ is now known as DLoader
08:16:44nulldata (nulldata) joins
08:24:09eawfeaw quits [Remote host closed the connection]
08:41:35project10 quits [Ping timeout: 252 seconds]
08:42:35project10 (project10) joins
08:48:38project10 quits [Ping timeout: 265 seconds]
08:49:41project10 (project10) joins
08:57:09efeafewa joins
10:00:01railen63 quits [Remote host closed the connection]
10:00:19railen63 joins
10:04:31Perk quits [Ping timeout: 265 seconds]
10:07:34Perk joins
10:11:06toss (toss) joins
10:12:15rohvani quits [Ping timeout: 265 seconds]
10:15:09qwertyasdfuiopghjkl quits [Ping timeout: 265 seconds]
10:17:10Perk quits [Read error: Connection reset by peer]
10:22:28rohvani joins
10:26:59Perk joins
10:28:23Perk quits [Read error: Connection reset by peer]
10:35:38Perk joins
10:39:17Perk quits [Read error: Connection reset by peer]
10:45:47Perk joins
10:48:30Perk quits [Read error: Connection reset by peer]
10:51:32Perk joins
10:53:35Perk quits [Read error: Connection reset by peer]
10:56:16Perk joins
10:57:29erkinalp joins
11:09:32BearFortress quits [Ping timeout: 252 seconds]
11:11:37BlueMaxima quits [Read error: Connection reset by peer]
11:19:14BearFortress joins
11:35:45Earendil7 quits [Quit: No Ping reply in 180 seconds.]
11:36:03Earendil7 (Earendil7) joins
11:50:28Bleo quits [Client Quit]
11:50:44Bleo joins
11:58:35BearFortress quits [Ping timeout: 265 seconds]
11:59:37kiryu quits [Read error: Connection reset by peer]
12:01:01kiryu joins
12:01:01kiryu quits [Changing host]
12:01:01kiryu (kiryu) joins
12:07:44erkinalp quits [Remote host closed the connection]
12:12:20BearFortress joins
12:21:29kiryu quits [Client Quit]
12:21:41Minkafighter52531 quits [Client Quit]
12:22:13Minkafighter52531 joins
12:48:18eythian quits [Quit: http://quassel-irc.org - Chat comfortabel. Waar dan ook.]
12:48:20eythian joins
12:50:36Perk0 joins
12:50:36Perk quits [Read error: Connection reset by peer]
12:50:36Perk0 is now known as Perk
12:50:56Perk quits [Client Quit]
12:51:19Perk joins
12:57:58Minkafighter52531 quits [Client Quit]
12:58:07erkinalp joins
12:58:44Minkafighter52531 joins
13:00:46<thuban>i had no idea about torrent uploads, that's awesome
13:10:36BearFortress quits [Ping timeout: 265 seconds]
13:14:23kiryu (kiryu) joins
13:21:05BearFortress joins
13:23:38Minkafighter52531 quits [Client Quit]
13:24:00Minkafighter52531 joins
13:24:46Minkafighter52531 quits [Client Quit]
13:25:03Minkafighter52531 joins
13:25:58Minkafighter52531 quits [Client Quit]
13:26:16Minkafighter52531 joins
13:47:24Minkafighter52531 quits [Client Quit]
13:48:44Minkafighter52531 joins
13:50:41Minkafighter52531 quits [Client Quit]
13:52:24Minkafighter52531 joins
13:52:40<TheTechRobo>It's also mentioned on the wiki: https://wiki.archiveteam.org/index.php/Internet_Archive#Torrent_upload
13:52:57<TheTechRobo>Used to use it a lot before I got fibre
13:57:46Minkafighter52531 quits [Client Quit]
13:58:04Minkafighter52531 joins
14:00:43etnguyen03 (etnguyen03) joins
14:02:29Minkafighter52531 quits [Client Quit]
14:03:53Arcorann quits [Ping timeout: 252 seconds]
14:05:09Minkafighter52531 joins
14:05:31Minkafighter52531 quits [Client Quit]
14:06:31Minkafighter52531 joins
14:06:38TastyWiener95 quits [Ping timeout: 252 seconds]
14:07:40TastyWiener95 (TastyWiener95) joins
14:07:52Minkafighter52531 quits [Client Quit]
14:09:26Minkafighter52531 joins
14:15:26BigBrain_ quits [Ping timeout: 245 seconds]
14:17:53BigBrain_ (bigbrain) joins
14:18:42dumbgoy joins
14:44:35toss quits [Ping timeout: 252 seconds]
14:50:38HugsNotDrugs quits [Ping timeout: 252 seconds]
14:55:05HugsNotDrugs joins
14:55:58treora quits [Ping timeout: 265 seconds]
14:56:11treora joins
15:20:46<manu|m>hi, so there is this publisher that recently posted about possibly being insolvent, depending on how much donations they will receive in the next two weeks. they have a magazine, a shop, etc.—6 domains in total (that i’m aware of). would you be willing to run that through the archivebot? (repost here so it doesn't get lost in #archivebot)
15:20:55<manu|m>this is their post, but it's german only :/ https://katapult-magazin.de/de/artikel/katapult-ist-insolvent
15:23:01<thuban>manu|m: thanks for the report! that looks archivebot-able. can you list the relevant domains?
15:25:28<manu|m>sure: https://katapult-verlag.de/ - https://katapult-magazin.de/ - https://katapult-mv.de/ - https://katapult-ukraine.com/ - https://www.katapult-shop.de/ - https://katapult.link/
15:26:35<manu|m>katapult-magazin.de has a bunch of subdomains, but their all either some internal service with a login at the front, not reachable, and two were some one-pagers that I sent to IA via browser extension
15:28:59<manu|m>I don't think katapult.link needs to process external links, looks like they're only linking to their own stuff on other domains
15:34:40<thuban>having looked at the link structure between those domains, i think it'll be best to run a separate archivebot job for each
15:35:12<thuban>(possibly with --no-offsite-links on the katapult.link job to avoid duplication)
15:38:25<thuban>someone with bot privileges should queue them up shortly, i believe
15:44:49<manu|m>thanks, i appreciate it :)
15:48:34nicolas17 joins
15:49:29etnguyen03 quits [Ping timeout: 252 seconds]
15:57:56<h2ibot>That lurker edited GitHub (-44, ghtorrent.org domain is used for casino…): https://wiki.archiveteam.org/?diff=50737&oldid=50426
16:00:04Bleo quits [Client Quit]
16:33:09Bleo joins
17:07:48jacksonchen666 (jacksonchen666) joins
17:09:41taaffeite joins
17:13:14qwertyasdfuiopghjkl (qwertyasdfuiopghjkl) joins
17:13:24Minkafighter52531 quits [Client Quit]
17:13:27Minkafighter52531 joins
17:20:53etnguyen03 (etnguyen03) joins
17:32:16Minkafighter52531 quits [Client Quit]
17:33:39Minkafighter52531 joins
17:34:00Minkafighter52531 quits [Client Quit]
17:34:14Minkafighter52531 joins
17:38:07Minkafighter52531 quits [Client Quit]
17:44:36erkinalp quits [Remote host closed the connection]
17:45:01Minkafighter52531 joins
18:06:24Earendil7 quits [Ping timeout: 265 seconds]
18:06:29Minkafighter52531 quits [Client Quit]
18:06:45Minkafighter52531 joins
18:14:08etnguyen03 quits [Ping timeout: 252 seconds]
18:14:25<@arkiver>wondering - how are we on resource for #archivebot nowadays? seems like all jobs are starting immediately. or is there something else that shows we would need more resources?
18:16:35<pokechu22>Right now we seem to be OK - it looks like all of the stuff I queued just now has filled all pipelines (I know because !status says 205 in progress, and around 200 is the limit - the exact number varies because some pipelines only accept specific jobs though), but that stuff should finish shortly. It was struggling a bit when there was a ton of gabon stuff going on but I
18:16:37DogsRNice joins
18:16:38<pokechu22>think it's OK at the moment
18:18:25<Barto>as my operating system teacher used to say, disks are more often full than empty, and are meant to be filled up :-) I think we're okay too. If I want to put a lot of subdomain it will definitely be queued, and if socialbot comes back for twitter, the same will apply.
18:19:58<pokechu22>(though also it's worth noting that the 200 in progress is misleading as that includes stuff on the cybercontrol pipelines, which have been offline for over a year (see http://archivebot.com/pipelines))
18:20:34<Barto>certainly, also how about those jobs there?
18:21:57<pokechu22>I'm not entirely sure what their status is and if they could be resumed if/when the pipelines return
18:50:13Minkafighter|m joins
18:51:32taaffeite quits [Ping timeout: 252 seconds]
18:55:27taaffeite joins
18:57:36Island joins
19:05:50taaffeite quits [Ping timeout: 252 seconds]
19:12:59lennier1 quits [Ping timeout: 252 seconds]
19:21:45lennier1 (lennier1) joins
19:25:08Minkafighter52531 quits [Client Quit]
19:29:54icedice (icedice) joins
19:32:10<icedice>https://twitter.com/tanks404/status/1700171142642757657
19:32:10<eggdrop>nitter: https://nitter.net/tanks404/status/1700171142642757657
19:32:15<icedice>"Come end of Oct, Tokyo Lab, one of the largest holders of film material in Japan, will be discarding all film left unclaimed by rights holders. It's a foregone conclusion that anything stuck in "licensing hell" is done-for. This is bad."
19:32:29<icedice>https://twitter.com/retroanimechris/status/1700512775536042073
19:32:30<eggdrop>nitter: https://nitter.net/retroanimechris/status/1700512775536042073
19:32:43<icedice>"> @nappasan @NFAJ_PR The negatives will be looked into by Councillor Ken Akamatsu. We have not received a commitment to resolve the issue, so please keep an eye on the progress. We would also like to continue to ask for effective approaches from all parties."
19:32:45etnguyen03 (etnguyen03) joins
19:33:09<@Sanqui>arkiver: re: archivebot, we have gotten pretty good at estimating the limit and backing off when we hit it, but that doesn't mean we couldn't use more resources, especially with a more intelligent queuing system, or a proper ability to suspend jobs
20:37:51andrew quits [Quit: Ping timeout (120 seconds)]
20:41:26AmAnd0A quits [Read error: Connection reset by peer]
20:41:45AmAnd0A joins
20:42:46andrew (andrew) joins
20:54:11etnguyen03 quits [Ping timeout: 252 seconds]
21:13:26lun4 quits [Client Quit]
21:13:46lun4 (lun4) joins
21:15:11<nicolas17>arkiver: note that there may be some low-priority stuff that people *aren't* adding to archivebot because they know of the issues uploading to IA
21:16:17<@arkiver>Sanqui: pokechu22: thank you! i do not have anything to offer right now, but wanted to know what the current situation is
21:16:21Barto quits [Ping timeout: 265 seconds]
21:16:35<@arkiver>nicolas17: i think it's fine for people to queue stuff anyway, we can always start using the offload space if really needed
21:16:39<@arkiver>also JAA ^
21:16:58<nicolas17>I mean where is archivebot data going now?
21:17:10<nicolas17>are we uploading to IA, maybe at reduced speed?
21:17:56<pokechu22>I guess I should also mention that there have been issues where one of the pipelines (I can't remember if it was hel3 or hel4) had issues uploading to IA, while the other one of the two was fine (and it also was able to transfer data to the other pipeline and upload it quickly). I'm not actually sure who's responsible for those pipelines (probably AK because
21:17:58<pokechu22>"ak-was-here-hel4") but it's something weird that's happening
21:21:15Barto (Barto) joins
21:21:38<AK>(can't remember which one it is either). Iirc JA_A did some digging and couldn't see an obvious reason. It also didn't seem to happen all the time
21:22:12<AK>(I pay the bill, JAA does all the managing of the AB stuff on the top because I don't know how and they're super nice <3)
21:22:18<@arkiver>it's being uploaded
21:22:22<@arkiver>i believe not at reduced speeds
21:22:28<nicolas17>ah
21:22:38<@arkiver>it's going in, and it's anyway a small portion of what we normally put through to IA so not a huge problem
21:22:47<nicolas17>I'm more familiar with how warrior stuff works and largely blind to archivebot :)
21:22:54<pokechu22>It's worth noting that the issue with hel3 or hel4 is a longstanding one that wasn't caused by the more recent IA issues; it's just mysterious
21:23:42<@arkiver>i bet rewby would have an idea about that, rewby is the expert when it comes to pushing data to IA
21:28:18erkinalp joins
21:36:02<h2ibot>Myusernameisanything edited ISP Hosting (+3, I archived the scraped URLS for Claranet…): https://wiki.archiveteam.org/?diff=50738&oldid=50486
21:36:03<h2ibot>Myusernameisanything edited Claranet Netherlands Personal Web Pages (+3, I archived the scraped URLS and now they are in…): https://wiki.archiveteam.org/?diff=50739&oldid=47495
21:37:03<h2ibot>That lurker edited Valhalla (+483, Cerabytes product is like ment for archiving so…): https://wiki.archiveteam.org/?diff=50740&oldid=48930
21:41:24lunik173 quits [Quit: :x]
21:41:50lunik173 joins
21:47:51DogsRNice_ joins
21:48:17etnguyen03 (etnguyen03) joins
21:52:07DogsRNice quits [Ping timeout: 265 seconds]
22:11:23<@JAA>arkiver: The hel4 upload issues are totally unrelated to anything at IA etc. That machine just has irregular transient issues towards the rsync target for some unknown reason. Nothing weird shows up in mtr etc. when it happens, but rsync transfers run at a couple hundred kB/s. Then it's fine again a few hours later. I asked a few people and nobody had any ideas what I could try to diagnose the issue
22:11:29<@JAA>either. When I'm around, I route the rsync traffic through another machine at Hetzner Helsinki, and everything works perfectly fine there.
22:12:22<@arkiver>could it be a hardware issue?
22:12:25<@JAA>On cybercontrol, I mentioned this in #archivebot recently, but it obviously got buried, so repeating for visibility: those pipelines might be a loss since the relevant person has been MIA for a long time now. If nothing happens by the end of the month, they'll be thrown out of the system.
22:12:31<@JAA>Cc pokechu22 ^
22:12:56<@arkiver>yes anything mentioned in #archivebot will probably be missed by a ton of people, please post it here too or we should have a special AB discussion channel
22:13:25<project10>#archivebot-bot
22:13:52<Barto>damn, never fun to have someone go MIA :(
22:13:58<@JAA>I mark important things with [PSA], and I advise people to set up highlights for that. A separate discussion channel has been brought up several times, there even is one, but it's completely unused, and it could kind of lead to a split brain.
22:14:20<@arkiver>which one?
22:14:23<@arkiver>let's start using it
22:15:18<@JAA>#archivebot-bs
22:16:20<Barto>for the interested: weechat.look.highlight_regex "^\[PSA\]"
22:16:30<Barto>that's what i use
22:18:00<fireonlive>i could do that with TL; though it works everywhere
22:18:07<fireonlive>add that to your reasons to hate TL JAA :p
22:25:44Doranwen quits [Quit: bbl]
22:25:51Doranwen (Doranwen) joins
22:28:47wickedplayer494 quits [Ping timeout: 252 seconds]
22:32:29<thuban>is #archivebot-bs official, then?
22:32:45<@arkiver>no
22:32:47<@JAA>There's just been discussion about that in there. :-P
22:34:17<thuban>so are we going to start using it or no ?_?
22:34:38<@JAA>My point of view is: discussions in #archivebot are mostly about immediate action, e.g. ignores or starting jobs, and dev stuff can already happen in #archiveteam-dev. PSAs like the above are rare enough that I'm not sure a separate channel makes sense, and I can repeat them here in the future for visibility.
22:35:22<@JAA>I'll also note that the above was merely a 'this might happen' comment, not a real PSA. If it comes to that, I'll of course explicitly ping everyone with affected jobs, too.
22:35:43<@JAA>More important PSAs are also usually added to the channel topic.
22:35:54<@JAA>(Like the 'don't touch the 600001 jobs' one)
22:37:33<@arkiver>i hope long term discussions/planning can be outside of #archivebot
22:37:41<@arkiver>talking about the cybercontrol stuff for example
22:37:52<@JAA>Well, not much to plan there, sadly.
22:37:53<@arkiver>very jobs specific stuff can be in #archivebot i think?
22:38:27<thuban>agreed, but i think JAA is correct that it can happen here or in -dev (as appropriate)
22:39:20<@arkiver>JAA: so archiveteam-bs/dev instead of archivebot-bs ?
22:40:06BlueMaxima joins
22:41:46wickedplayer494 joins
22:42:33<@JAA>I think so, at least. If others disagree, I won't oppose a separate channel.
22:43:48<@arkiver>we could decide tomorrow
22:45:53wickedplayer494 quits [Remote host closed the connection]
22:47:17<that_lurker>JAA: Are PSA's posted on the #archiveteam channel?
22:49:55wickedplayer494 joins
22:51:42<@JAA>that_lurker: If it's something sufficiently important, I can do that, yeah.
23:20:22<h2ibot>JustAnotherArchivist edited Deathwatch (+193, /* 2023 */ Add Squat the Planet): https://wiki.archiveteam.org/?diff=50741&oldid=50713
23:23:59<that_lurker>Yeah then I woulds say the current channels and practices cover the need of a dedicated channel. But I make the ocasional memes so the decision is up to someone else :P
23:34:05<fireonlive>archivebot meme, go!
23:34:07<fireonlive>:D
23:40:09<that_lurker>https://lounge.kuhaon.fun/folder/28c3f47d3ea56b97/7yllrg.jpg
23:41:17<fireonlive>x3
23:41:38<@JAA>Alternatively: right label 'ArchiveBot', left label 'Buttflare'
23:42:46<that_lurker>https://lounge.kuhaon.fun/folder/daed62744f4581d3/7yllyz.jpg
23:43:05<@JAA>:-)

HomeSearchPrevious dayNext day