HomeSearchPrevious dayNext day

00:03:45wickedplayer494 quits [Remote host closed the connection]
00:08:52wickedplayer494 joins
00:45:59michaelblob_ (michaelblob) joins
00:47:16Craigle quits [Quit: Ping timeout (120 seconds)]
00:48:49Craigle (Craigle) joins
00:50:18wickedplayer494 quits [Ping timeout: 265 seconds]
00:50:18michaelblob quits [Ping timeout: 265 seconds]
00:50:49wickedplayer494 joins
03:04:11ThreeHM quits [Ping timeout: 265 seconds]
03:04:35ThreeHM (ThreeHeadedMonkey) joins
05:52:35<@OrIdow6^2>How long do they typically take to run?
06:00:57<Maakuth|m>so these scripts fetch stuff based on links on the index page?
06:01:58<Maakuth|m>http://flux.mbnet.fi/saku/89luettelo/ here would be one that's not discoverable by index links, but should be good stuff
06:17:29<@OrIdow6^2>Unfortunately that's kind of a limit imposed by the architecture
06:18:03<@OrIdow6^2>If we had enough time to plausibly get a tracker project up, there are some things I could do
06:18:33<@OrIdow6^2>And if AB was running it could go there
06:21:08<Maakuth|m>could we slip that one in manually somehow? I was specifically asked by somebody to ensure this one is archived
06:21:46<@OrIdow6^2>Oh yeah in that case let me come up with something
06:26:29<Maakuth|m>i'll solicit for similar cases
06:26:37<@OrIdow6^2>wget-at --recursive --level=inf --no-parent --page-requisites --content-on-error --delete-after --warc-file yourfilename http://flux.mbnet.fi/saku/89luettelo/
06:27:38<@OrIdow6^2>That's basically what the script runs, without automation
06:28:01<@OrIdow6^2>Feel free to put the into the pad
06:28:11<@OrIdow6^2>Maybe in a separate section
06:28:37<@OrIdow6^2>Honestly, there's a possibility based on the unusual way this project has been run that these will end up going to "outsider warcs" and not the Wayback machine
06:28:45<@OrIdow6^2>But at least they'll be somewhere
06:30:48<Maakuth|m>https://muistio.tieke.fi/p/mbnet i'll opened a page here in finnish, i'll put is somewhere
06:31:09<Maakuth|m>hmmh, the `rm` command fails due to someone having - in their username
06:31:56<Maakuth|m>I think the `tar` would have similar issue
06:33:16<@OrIdow6^2>Plausible, I can't do shell escaping
06:38:39<Maakuth|m>hmmh. there are two such usernames: -agathon1 and -solmu
06:39:17<Maakuth|m>i will just remove those dashes from my file to get it error out properly and let's figure out how to handle those two, ok?
06:39:29<@OrIdow6^2>This look good?:
06:39:31<@OrIdow6^2>rm -rf ./$subdomain.mbnet.fi
06:39:40<@OrIdow6^2>tar -c --numeric-owner $(find . -name "*.warc.gz")
06:39:46<@OrIdow6^2>find . -name "*.warc.gz" -delete
06:40:01<@OrIdow6^2>I am on low sleep right now but I think these are ok
06:40:59<@OrIdow6^2>What do you mean?
06:41:26<@OrIdow6^2>If you're just saying to handle them individually, that is probably a better idea than modifying the whole script, I guess
06:41:46<Maakuth|m>yeah
06:42:29<Maakuth|m>my idea is to have 'solmu' instead on '-solmu' in the file so that the line count is preserved
06:42:50<Maakuth|m>oh, there are some others. I think five in total
06:45:04<ThreeHM>Also ran into that over night, seems to only affect tar and the final rm
06:45:37<ThreeHM>`tar -c --numeric-owner -- *.warc.gz` seems to work
06:46:02<ThreeHM>Same thing for `rm -- *.warc.gz`
06:46:11<Maakuth|m>oh, nice
06:46:19<Maakuth|m>i'll do that then
06:46:56<Maakuth|m>I think it also affects the rm -rf subdomain one?
06:47:36<@OrIdow6^2>It does but it looks like that works there also
06:47:42<@OrIdow6^2>I think it does
06:47:51<Maakuth|m>yeah. I applied those
06:48:29<@OrIdow6^2>Thank you ThreeHM
07:04:29Maturion joins
07:12:08qwertyasdfuiopghjkl quits [Ping timeout: 265 seconds]
10:10:57<Maakuth|m>hmm, ThreeHM youve claimed the rest?
10:31:12<Maakuth|m>I'll start running them too in case threehm's claim is by accident
10:51:20<ThreeHM>2022-10-11 01:10:30 ThreeHM_ I've claimed a fairly big range near the end to run over night (assuming my automation works) - Feel free to reclaim them in case progress gets there before I'm back tomorrow
10:51:29<ThreeHM>Maakuth|m ^
10:52:38<Maakuth|m>ah, ok
10:53:40<ThreeHM>Was just about to add the URLs from that
10:54:08<Maakuth|m>ok.. i'm through them all very soon
11:04:36<ThreeHM>I've added mine anyways since they're already uploaded
11:08:32<Maakuth|m>nice
11:09:02<Maakuth|m>i'll do the manual ones
11:13:52<Maakuth|m>ok, those too are there
11:16:04<@HCross>Hello
11:16:08<@HCross>Is there anything I can do here?
11:50:03Maturion quits [Ping timeout: 265 seconds]
11:57:41Maturion joins
14:35:21Maturion quits [Ping timeout: 265 seconds]
14:44:09Maturion joins
16:49:10datechnoman quits [Client Quit]
16:50:43datechnoman (datechnoman) joins
17:50:40IDK quits [Quit: Connection closed for inactivity]
20:12:05<@OrIdow6^2>HCross: I guess I should have asked this before, do you have the ability to set up a tracker project? My impression was that ark iver was needed to do that and there wasn't enough time, but I'm having doubts now
20:12:33<@HCross>I probably can and do have access to it, but I haven’t a clue how
20:12:43<@OrIdow6^2>Basically we have been running wget-at manually here and using Etherpad to keep track of "items"
20:14:07<@OrIdow6^2>Oh, wow, whoever said they were doing automation wasn't kidding
20:15:15<@OrIdow6^2>So I guess what we could use is a little target thing to combine the warcs that are currently on transfer.archivete.am
20:15:44<@OrIdow6^2>And upload them somewhere
21:00:41thetechrobo_ (TheTechRobo) joins
21:02:37thetechrobo_ is now known as Pebbles
21:03:13Pebbles quits [Remote host closed the connection]
21:03:28tzt quits [Ping timeout: 265 seconds]
21:04:39tzt (tzt) joins
22:12:06Maturion quits [Ping timeout: 265 seconds]

HomeSearchPrevious dayNext day