HomeSearchPrevious dayNext day

00:05:19etnguyen03 (etnguyen03) joins
00:18:38ScarlettStunningSpace quits [Read error: Connection reset by peer]
00:30:57etnguyen03 quits [Client Quit]
00:41:32tekulvw quits [Ping timeout: 268 seconds]
00:42:12Island joins
00:44:04etnguyen03 (etnguyen03) joins
00:45:41SootBector quits [Remote host closed the connection]
00:46:49SootBector (SootBector) joins
00:54:10test4 joins
00:56:13test4 quits [Client Quit]
01:02:45etnguyen03 quits [Client Quit]
01:17:07Arcorann (Arcorann) joins
02:04:59etnguyen03 (etnguyen03) joins
02:22:07vexr leaves [User left]
02:44:52chunkynutz60 quits [Ping timeout: 268 seconds]
02:48:51<GodzFire>pokechu22 any luck with cloudflare?
02:58:37Yakov quits [Quit: The Lounge - https://thelounge.chat]
02:58:53Yakov (Yakov) joins
03:01:45chunkynutz60 joins
03:19:36tekulvw joins
03:30:08etnguyen03 quits [Remote host closed the connection]
03:30:36h|ca2 (h) joins
03:33:26crullerIRC quits [Quit: IRCNow and Forever!]
03:36:42<pokechu22>Nope, but will keep checking
03:38:06crullerIRC joins
03:39:15klea wonders if this should be happening in #wikiteam or #wikibot instead :p
03:43:26lennier2 joins
03:46:41lennier2_ quits [Ping timeout: 272 seconds]
03:48:23tekulvw quits [Ping timeout: 268 seconds]
03:55:28Island quits [Read error: Connection reset by peer]
04:06:38CYBERDEV_ joins
04:06:57CYBERDEV quits [Ping timeout: 272 seconds]
04:16:13tekulvw joins
04:21:04tekulvw quits [Ping timeout: 268 seconds]
04:44:24tekulvw joins
04:46:24tekulvw quits [Remote host closed the connection]
04:46:34tekulvw (tekulvw) joins
04:57:37tekulvw quits [Ping timeout: 272 seconds]
05:01:19tekulvw (tekulvw) joins
05:04:51n9nes quits [Ping timeout: 268 seconds]
05:07:27n9nes joins
05:10:46itachi1706 quits [Quit: Bye :P]
05:12:37benjins3 quits [Read error: Connection reset by peer]
06:16:57Wohlstand1 (Wohlstand) joins
06:19:22Wohlstand1 is now known as Wohlstand
06:24:45medecau (medecau) joins
06:25:38nexussfan quits [Quit: Konversation terminated!]
06:31:41Wohlstand quits [Client Quit]
06:45:17tekulvw quits [Ping timeout: 272 seconds]
06:57:53medecau quits [Client Quit]
07:02:08tekulvw (tekulvw) joins
07:06:57tekulvw quits [Ping timeout: 268 seconds]
07:53:04<@arkiver>justauser: can we split off the vimeo URLs into a different list for now?
07:57:15tekulvw (tekulvw) joins
09:03:21^ quits [Ping timeout: 272 seconds]
09:07:47tekulvw quits [Ping timeout: 272 seconds]
09:08:08tekulvw (tekulvw) joins
09:09:31^ (^) joins
09:13:10kuroger quits [Quit: Ping timeout (120 seconds)]
09:13:28kuroger (kuroger) joins
09:18:17GodzFire quits [Quit: Ooops, wrong browser tab.]
09:24:11ukraine_fm joins
09:25:46ukraine_fm quits [Remote host closed the connection]
09:33:45tekulvw quits [Ping timeout: 272 seconds]
09:45:44SootBector quits [*.net *.split]
09:45:44sec^nd quits [*.net *.split]
09:45:58SootBector (SootBector) joins
09:45:58sec^nd (second) joins
09:49:22CYBERDEV_ quits [*.net *.split]
09:49:22crullerIRC quits [*.net *.split]
09:49:22h|ca2 quits [*.net *.split]
09:49:22Arcorann quits [*.net *.split]
09:49:22pokechu22 quits [*.net *.split]
09:49:22cyanbox_ quits [*.net *.split]
09:49:22ATinySpaceMarine quits [*.net *.split]
09:49:22Shard111 quits [*.net *.split]
09:49:22APOLLO03 quits [*.net *.split]
09:49:22kansei- quits [*.net *.split]
09:49:22BornOn420 quits [*.net *.split]
09:49:22fetcher quits [*.net *.split]
09:49:22Bleo1826007227196234552220 quits [*.net *.split]
09:49:22chrismeller3 quits [*.net *.split]
09:49:22abirkill quits [*.net *.split]
09:49:22DopefishJustin quits [*.net *.split]
09:49:22Snivy quits [*.net *.split]
09:49:22BennyOtt quits [*.net *.split]
09:49:22mr_sarge quits [*.net *.split]
09:49:22hackbug quits [*.net *.split]
09:49:22iPwnedYourIOTSmartdog quits [*.net *.split]
09:49:22anarcat quits [*.net *.split]
09:49:22pie_ quits [*.net *.split]
09:49:22linuxgemini quits [*.net *.split]
09:49:22@Fusl quits [*.net *.split]
09:49:22cruller quits [*.net *.split]
09:49:22M--mlv|m quits [*.net *.split]
09:49:24miksters|m quits [*.net *.split]
09:49:24IceCodeNew|m quits [*.net *.split]
09:49:24mat|m1 quits [*.net *.split]
09:49:24Joy|m quits [*.net *.split]
09:49:24ampdot|m quits [*.net *.split]
09:49:24will|m quits [*.net *.split]
09:49:24username675f|m quits [*.net *.split]
09:49:24saouroun|m quits [*.net *.split]
09:49:24Passiing|m quits [*.net *.split]
09:49:24gareth48|m quits [*.net *.split]
09:49:24yarnover|m quits [*.net *.split]
09:49:24Claire|m quits [*.net *.split]
09:49:24Tyrasuki|m quits [*.net *.split]
09:49:24Valkum|m quits [*.net *.split]
09:49:24yetanotherarchiver|m quits [*.net *.split]
09:49:24hillow596|m quits [*.net *.split]
09:49:24rain|m quits [*.net *.split]
09:49:24Misty|m quits [*.net *.split]
09:49:24Cronfox|m quits [*.net *.split]
09:49:24noxious quits [*.net *.split]
09:49:24kaz__|m quits [*.net *.split]
09:49:24ram|m quits [*.net *.split]
09:49:24PhoHale|m quits [*.net *.split]
09:49:24mister_x quits [*.net *.split]
09:49:25katia|m quits [*.net *.split]
09:49:25bogsen quits [*.net *.split]
09:49:25vics quits [*.net *.split]
09:49:25starg2|m quits [*.net *.split]
09:49:25Fijxu|m quits [*.net *.split]
09:49:25supermariofan67|m quits [*.net *.split]
09:49:25trumad|m quits [*.net *.split]
09:49:25NickS|m quits [*.net *.split]
09:49:25haha-whered-it-go|m quits [*.net *.split]
09:49:25osiride|m quits [*.net *.split]
09:49:25ax|m quits [*.net *.split]
09:49:25spearcat|m quits [*.net *.split]
09:49:25Adamvoltagex|m quits [*.net *.split]
09:49:25v1cs quits [*.net *.split]
09:49:25joepie91|m quits [*.net *.split]
09:49:25nosamu|m quits [*.net *.split]
09:49:25ragu|m quits [*.net *.split]
09:49:25its_notjack quits [*.net *.split]
09:49:25nightpool quits [*.net *.split]
09:49:25triplecamera|m quits [*.net *.split]
09:49:25nano412510 quits [*.net *.split]
09:49:25mikolaj|m quits [*.net *.split]
09:49:25GhostIsBeHere|m quits [*.net *.split]
09:49:25that_lurker|m quits [*.net *.split]
09:49:25EvanBoehs|m quits [*.net *.split]
09:49:25GRBaset quits [*.net *.split]
09:49:25superusercode quits [*.net *.split]
09:49:25s-crypt|m|m quits [*.net *.split]
09:49:25jwoglom|m quits [*.net *.split]
09:49:25lasdkfj|m quits [*.net *.split]
09:49:25jackt1365|m quits [*.net *.split]
09:49:25octylFractal|m quits [*.net *.split]
09:49:25noobirc|m quits [*.net *.split]
09:49:25wrangle|m quits [*.net *.split]
09:49:25pannekoek11|m quits [*.net *.split]
09:49:25cmostracker|m quits [*.net *.split]
09:49:25CrispyAlice2 quits [*.net *.split]
09:49:25akaibu|m quits [*.net *.split]
09:49:25Nulo|m quits [*.net *.split]
09:49:25thermospheric quits [*.net *.split]
09:49:25Ruk8 quits [*.net *.split]
09:49:25madpro|m quits [*.net *.split]
09:49:25upperbody321|m quits [*.net *.split]
09:49:25Video quits [*.net *.split]
09:49:25Roki_100|m quits [*.net *.split]
09:49:25l0rd_enki|m quits [*.net *.split]
09:49:25hexagonwin|m quits [*.net *.split]
09:49:25victor_vaughn|m quits [*.net *.split]
09:49:25iCesenberk|m quits [*.net *.split]
09:49:25qyxojzh|m quits [*.net *.split]
09:49:25th3z0l4|m quits [*.net *.split]
09:49:25phaeton quits [*.net *.split]
09:49:25yzqzss quits [*.net *.split]
09:49:25coro quits [*.net *.split]
09:49:25jevinskie quits [*.net *.split]
09:49:25e2mau|m quits [*.net *.split]
09:49:25Fletcher quits [*.net *.split]
09:49:26tech234a quits [*.net *.split]
09:49:26tech234a|m-backup quits [*.net *.split]
09:49:26Thibaultmol quits [*.net *.split]
09:49:26moe-a-m|m quits [*.net *.split]
09:49:26schwarzkatz|m quits [*.net *.split]
09:49:26andrewvieyra|m quits [*.net *.split]
09:49:26finalti|m quits [*.net *.split]
09:49:26Alienmaster|m quits [*.net *.split]
09:49:26MinePlayersPEMyNey|m quits [*.net *.split]
09:49:26aaq|m quits [*.net *.split]
09:49:26masterx244|m quits [*.net *.split]
09:49:26nyuuzyou quits [*.net *.split]
09:49:26@rewby|m quits [*.net *.split]
09:49:26nstrom|m quits [*.net *.split]
09:49:26tomodachi94 quits [*.net *.split]
09:49:26Hans5958 quits [*.net *.split]
09:49:26mpeter|m quits [*.net *.split]
09:49:26MaxG quits [*.net *.split]
09:49:26flashfire42|m quits [*.net *.split]
09:49:26Cydog|m quits [*.net *.split]
09:49:26Tom|m1 quits [*.net *.split]
09:49:26Minkafighter|m quits [*.net *.split]
09:49:26alexshpilkin quits [*.net *.split]
09:49:26igneousx quits [*.net *.split]
09:49:26justauser|m quits [*.net *.split]
09:49:26theblazehen|m quits [*.net *.split]
09:49:26x9fff00 quits [*.net *.split]
09:49:26DigitalDragon quits [*.net *.split]
09:49:26Exorcism quits [*.net *.split]
09:49:26gamer191-1|m quits [*.net *.split]
09:49:26Vokun quits [*.net *.split]
09:49:26audrooku|m quits [*.net *.split]
09:49:26xxia|m quits [*.net *.split]
09:49:26@Sanqui|m quits [*.net *.split]
09:49:26mind_combatant quits [*.net *.split]
09:49:26britmob|m quits [*.net *.split]
09:49:26anon00001|m quits [*.net *.split]
09:49:26Ajay quits [*.net *.split]
09:49:58cruller (cruller) joins
09:49:58M--mlv|m joins
09:49:58miksters|m joins
09:49:58IceCodeNew|m joins
09:49:58mat|m1 joins
09:49:58Joy|m joins
09:49:58ampdot|m joins
09:49:58will|m joins
09:49:58username675f|m joins
09:49:58saouroun|m joins
09:49:58Passiing|m joins
09:49:58gareth48|m joins
09:49:58yarnover|m joins
09:49:58Claire|m joins
09:49:58Tyrasuki|m joins
09:49:58Valkum|m joins
09:49:58yetanotherarchiver|m joins
09:49:58Misty|m joins
09:49:58hillow596|m joins
09:49:58rain|m joins
09:49:58Cronfox|m joins
09:49:58noxious joins
09:49:58PhoHale|m joins
09:49:58mister_x joins
09:49:58kaz__|m joins
09:49:58ram|m joins
09:49:58Fijxu|m joins
09:49:58Video joins
09:49:58starg2|m joins
09:49:58mind_combatant (mind_combatant) joins
09:49:58Exorcism (exorcism) joins
09:49:58osiride|m joins
09:49:58justauser|m (justauser|m) joins
09:49:58its_notjack (its_notjack) joins
09:49:58Alienmaster|m joins
09:49:58e2mau|m joins
09:49:58katia|m joins
09:49:58tomodachi94 (tomodachi94) joins
09:49:58anon00001|m joins
09:49:58mikolaj|m joins
09:49:58Adamvoltagex|m joins
09:49:58supermariofan67|m joins
09:49:58vics joins
09:49:58Tom|m1 joins
09:49:58iCesenberk|m joins
09:49:58Roki_100|m joins
09:49:58jevinskie joins
09:49:58CrispyAlice2 joins
09:49:58Fletcher (Fletcher) joins
09:49:58lasdkfj|m joins
09:49:58masterx244|m (masterx244|m) joins
09:49:58audrooku|m joins
09:49:58britmob|m joins
09:49:58nightpool (nightpool) joins
09:49:58ragu|m joins
09:49:58madpro|m joins
09:49:58x9fff00 (x9fff00) joins
09:49:58jackt1365|m joins
09:49:58schwarzkatz|m joins
09:49:58haha-whered-it-go|m joins
09:49:58nstrom|m joins
09:49:58theblazehen|m joins
09:49:58andrewvieyra|m joins
09:49:58Ruk8 (Ruk8) joins
09:49:58pannekoek11|m joins
09:49:58joepie91|m joins
09:49:58ax|m joins
09:49:58victor_vaughn|m joins
09:49:58th3z0l4|m joins
09:49:58Hans5958 (Hans5958) joins
09:49:58hexagonwin|m joins
09:49:58tech234a (tech234a) joins
09:49:58triplecamera|m joins
09:49:58spearcat|m joins
09:49:58nyuuzyou joins
09:49:58upperbody321|m joins
09:49:58l0rd_enki|m joins
09:49:58nano412510 (nano412510) joins
09:49:58GhostIsBeHere|m joins
09:49:58aaq|m joins
09:49:58v1cs joins
09:49:58bogsen (bogsen) joins
09:49:58that_lurker|m joins
09:49:58Nulo|m joins
09:49:58s-crypt|m|m joins
09:49:58gamer191-1|m joins
09:49:58qyxojzh|m joins
09:49:58octylFractal|m joins
09:49:58EvanBoehs|m joins
09:49:58flashfire42|m (flashfire42) joins
09:49:58alexshpilkin joins
09:49:58coro joins
09:49:58phaeton (phaeton) joins
09:49:58Vokun (Vokun) joins
09:49:58noobirc|m joins
09:49:58cmostracker|m joins
09:49:58trumad|m joins
09:49:58nosamu|m joins
09:49:58Cydog|m joins
09:49:58jwoglom|m joins
09:49:58MaxG joins
09:49:58superusercode joins
09:49:58wrangle|m joins
09:49:58moe-a-m|m joins
09:49:58yzqzss (yzqzss) joins
09:49:58Thibaultmol joins
09:49:58finalti|m joins
09:49:58Minkafighter|m joins
09:49:58GRBaset (GRBaset) joins
09:49:58akaibu|m joins
09:49:58NickS|m joins
09:49:58igneousx (igneousx) joins
09:49:58Ajay joins
09:49:58DigitalDragon joins
09:49:58xxia|m joins
09:49:58mpeter|m joins
09:49:58thermospheric joins
09:49:58MinePlayersPEMyNey|m joins
09:49:58tech234a|m-backup (tech234a) joins
09:49:58Sanqui|m (Sanqui) joins
09:49:58rewby|m (rewby) joins
09:49:58palermo.hackint.org sets mode: +oo Sanqui|m rewby|m
09:50:32CYBERDEV_ joins
09:50:32crullerIRC joins
09:50:32h|ca2 (h) joins
09:50:32Arcorann (Arcorann) joins
09:50:32pokechu22 (pokechu22) joins
09:50:32cyanbox_ joins
09:50:32ATinySpaceMarine joins
09:50:32Shard111 (Shard) joins
09:50:32APOLLO03 joins
09:50:32kansei- (kansei) joins
09:50:32BornOn420 (BornOn420) joins
09:50:32fetcher joins
09:50:32Bleo1826007227196234552220 joins
09:50:32chrismeller3 (chrismeller) joins
09:50:32abirkill (abirkill) joins
09:50:32DopefishJustin (DopefishJustin) joins
09:50:32Snivy (Snivy) joins
09:50:32BennyOtt (BennyOtt) joins
09:50:32mr_sarge (sarge) joins
09:50:32hackbug (hackbug) joins
09:50:32iPwnedYourIOTSmartdog joins
09:50:32anarcat (anarcat) joins
09:50:32pie_ (pie_) joins
09:50:32linuxgemini (linuxgemini) joins
09:50:32Fusl (Fusl) joins
09:50:32palermo.hackint.org sets mode: +o Fusl
09:52:45TheEnbyperor_ quits [Ping timeout: 272 seconds]
09:52:45tuna quits [Ping timeout: 272 seconds]
09:53:23TheEnbyperor quits [Ping timeout: 272 seconds]
09:53:27tuna (tuna) joins
09:54:09TheEnbyperor joins
09:54:17TheEnbyperor_ (TheEnbyperor) joins
10:02:53fireatseaparks quits [Ping timeout: 272 seconds]
10:05:49fireatseaparks (fireatseaparks) joins
10:06:19<TheoH7>It's the last day https://community.jisc.ac.uk will be online. All my attempts to contact JISC (the organisation behind this website) have sadly gone unanswered.
10:06:41<TheoH7>The Cloudflare security page looks slightly different now, not sure if they're made it slightly more relaxed or if its just a different design.
10:07:45<kline>TheoH7, i didnt know you were looking for anything here. are there specific questions you had?
10:08:13<TheoH7>I asked a few days ago (maybe a week ago) if that site could be archived.
10:08:26<TheoH7>At the time people mentioned that as it was under Cloudflare under attack mode, the answer was no.
10:08:46<TheoH7>I'm just following up as the security message on the site has changed slightly, and it's being taken down today.
10:09:41<TheoH7>It's a site full of documentation to do with JISC (the UK National Academic network), eduroam technical guides, and valuable historical material.
10:12:38<TheoH7>So my question is really whether it's worth trying to take a copy of the site before shutdown, or whether security is still stopping it.
10:15:56<kline>im seeing if i still know anyone at janet, thats all
10:19:47AK (AK) joins
10:29:07<kline>ive not heard anything back yet, but grad students arent often known for being early monday risers. if i hear anything back ill say
10:41:57tekulvw (tekulvw) joins
10:42:57Dada joins
10:46:35tekulvw quits [Ping timeout: 272 seconds]
10:47:08<pabs>TheoH7: I checked both ArchiveBot and Mnbot (headless browser), no luck with either of them. SPN or archive.today might work but probably not
10:47:46<pabs>justauser might have some ideas about bypassing
11:12:16evergreen56 joins
11:15:43evergreen5 quits [Ping timeout: 272 seconds]
11:15:43evergreen56 is now known as evergreen5
11:23:34tekulvw (tekulvw) joins
11:28:25tekulvw quits [Ping timeout: 268 seconds]
11:38:19benjins3 joins
12:00:06Bleo1826007227196234552220 quits [Quit: The Lounge - https://thelounge.chat]
12:02:46Bleo1826007227196234552220 joins
12:41:51FiTheArchiver joins
12:41:53FiTheArchiver quits [Remote host closed the connection]
12:41:54<TheoH7>pabs: Thanks for trying.
12:45:21<TheoH7>Unfortunately time is quite tight now (the site is scheduled to go dark in just under 12 hours), so any more ideas very much appreciated.
12:48:57<TheoH7>pabs: archive.today doesn't seem to work, just does 403's and is stuck on the challenge page
13:00:06<Cupping1285>TheoH7 this is the direct ip https://52.31.120.68/
13:00:29<Cupping1285>You can bypass cloudflare with that
13:02:29<Cupping1285>`curl https://community.jisc.ac.uk/ --resolve community.jisc.ac.uk:443:52.31.120.68`, just use that or you can attempt to archive it manually and set it in your hosts file
13:02:46superkuh quits [Remote host closed the connection]
13:03:00superkuh joins
13:05:51Arcorann quits [Ping timeout: 268 seconds]
13:12:57<TheoH7>Cupping1285: Thank you so much. Could someone send that IP to ArchiveBot with the domain as hostname if that's possible?
13:15:21<TheoH7>Also happy to archive locally as I have a fast broadband connection, happy to take recommendations on what's best
13:28:56qw3rty_ joins
13:30:52<pabs>TheoH7: AB can't do fake DNS stuff, and fake DNS stuff can't go to the WBM anyway
13:31:15<pabs>I started an AB job for the IP, but lots of links are to the domain
13:31:44<pabs>and AB can't transform the domain links to IP links
13:31:53qw3rty quits [Ping timeout: 272 seconds]
13:36:49<pabs>might be worth doing it in grab-site too if someone has that setup and can do DNS hardcoding in it
13:45:11<Cupping1285>Also to get a list of all the ip addresses you can do that here `dig community-prod-211428478.eu-west-1.elb.amazonaws.com`, they have a domain rewrite, but only if you attempt to connect with that host. If you connect directly to the ip or the real host name then you get the actual contents.
13:52:34tekulvw (tekulvw) joins
13:57:13tekulvw quits [Ping timeout: 272 seconds]
14:30:25<cruller>For now, I'm running grab-site with a tampered hosts file.
14:31:05<cruller>However, it's very slow because I'm on a wireless connection and running it on a virtual machine.
14:35:55<TheoH7>cruller: Sounds good. If you like feel free to send me the hosts file modifications you did and I can get it running on a VM in a datacentre.
14:39:45<cruller>I simply appended “52.31.120.68 community.jisc.ac.uk” to the end. Good luck!
14:40:50<TheoH7>cruller: Thank you! Just thought it was worth checking in case you'd added any other entries.
14:41:15<TheoH7>cruller: Also, it would be great if you could save what you manage to get as well in case I'm too late.
14:42:43<cruller>TheoH7: Once you've finished crawling, share your URL list here and someone will re-substitute them with "52.31.120.68" and !ao < it.
14:42:48nexussfan (nexussfan) joins
14:52:47<TheoH7>cruller: Will do, just installing grab-site now. The VM I'm installing it on has a multi-gig uplink so I hope all will be finished by the shutdown in 9 hours.
14:57:04<pabs>cruller: are you able to upload the WARC? it won't go into the WBM but its worth having as an IA item
15:02:18<cruller>pabs: I can upload it, but it's an "illegal" WARC and should have a label or something to indicate that, right?
15:03:04<pabs>yeah, not sure what the appropriate stuff to add is though
15:03:51<justauser>In theory, the WARC files already have the server IP.
15:04:25<justauser>I'd do a line in the description + WARCZone.
15:05:59tekulvw (tekulvw) joins
15:10:41tekulvw quits [Ping timeout: 272 seconds]
15:11:22<justauser>For WikiTeam tasks, I don't even note the cases when an intermediary was removed rather than added. Probably a bad idea for WARCs, though.
15:13:28<TheoH7>Currently struggling as grab-site is installed, but choosing the Cloudflare IP's over the override in /etc/hosts. My override works given curl and ping, but grab-site seems to ignore it.
15:16:11<cruller>TheoH7: Hmm, I have no idea about that.
15:16:18<cruller>The crawl isn't finished yet, but I'm sending a dump of wpull.db for now: https://transfer.archivete.am/UeTcD/url_strings.txt
15:16:18<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/UeTcD/url_strings.txt
15:18:55nexussfan quits [Read error: Connection reset by peer]
15:23:53<justauser>TheoH7: There is some option to grab-site.
15:25:01<justauser>"--wpull-args=--no-skip-getaddrinfo" makes it care about /etc/hosts.
15:25:13nexussfan (nexussfan) joins
15:25:21<justauser>https://transfer.archivete.am/15N8yO/blogs.sapo.pt_dnshistory.org_A_64.145.13.213.txt - surprisingly small.
15:25:21<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/15N8yO/blogs.sapo.pt_dnshistory.org_A_64.145.13.213.txt
15:32:06<justauser>This included custom domains (still supported), but also defunct blogs.sapo.{cv,tl,etc}.
15:39:23nexussfan quits [Read error: Connection reset by peer]
15:44:01<justauser>CourtsDesk doesn't seem to be public.
15:47:08<kline>ok
15:47:11<kline>pros and cons to that
15:49:19<TheoH7>cruller: Managed to get it to work. Ended up using dnsmasq to override /etc/hosts entirely. Its crawling now.
15:59:17<TheoH7>Just checking: will grab-site save linked documents on the domain as well, such as PDF's?
16:07:19tekulvw (tekulvw) joins
16:11:29arch quits [Ping timeout: 272 seconds]
16:12:05tekulvw quits [Ping timeout: 268 seconds]
16:12:20arch (arch) joins
16:12:24tekulvw (tekulvw) joins
16:17:01tekulvw quits [Ping timeout: 268 seconds]
16:18:50nexussfan (nexussfan) joins
16:25:09pedantic-darwin joins
16:26:11tekulvw (tekulvw) joins
16:31:12tekulvw quits [Ping timeout: 268 seconds]
16:54:18tekulvw (tekulvw) joins
17:05:23<klea>2026-02-16 15:15:53 <kline> https://www.legalcheek.com/2026/02/ministry-of-justice-orders-deletion-of-the-uks-largest-court-reporting-database/
17:05:23<klea>2026-02-16 15:16:47 <kline> im not sure if this is open or has worthwhile info, im not currently in a good position to check
17:05:23<klea>Probably important (from #archiveteam)
17:05:49<kline>klea, someone checked, it appears to not be open to the public
17:05:54<klea>oh
17:06:25<kline>justauser, i meant to say thank you earlier, but i was in a rush
17:07:13tekulvw quits [Ping timeout: 272 seconds]
17:07:33Webuser656324 joins
17:08:35Webuser656324 quits [Client Quit]
17:11:24tekulvw (tekulvw) joins
17:15:09Island joins
17:15:28petrichor (petrichor) joins
17:17:29fuzzy80211 quits [Killed (NickServ (GHOST command used by fuzzy8021))]
17:17:36fuzzy80211 (fuzzy80211) joins
17:20:31tekulvw quits [Ping timeout: 272 seconds]
17:23:48fuzzy80211 quits [Read error: Connection reset by peer]
17:26:46fuzzy80211 (fuzzy80211) joins
17:27:09fuzzy80211 quits [Excess Flood]
17:27:46fuzzy80211 (fuzzy80211) joins
17:28:44fuzzy80211 quits [Read error: Connection reset by peer]
17:29:51fuzzy80211 (fuzzy80211) joins
17:30:52fuzzy80211 quits [Killed (NickServ (GHOST command used by fuzzy8021))]
17:31:04fuzzy80211 (fuzzy80211) joins
17:31:24fuzzy80211 quits [Excess Flood]
17:32:14fuzzy80211 joins
17:32:14fuzzy80211 quits [Changing host]
17:32:14fuzzy80211 (fuzzy80211) joins
17:32:33fuzzy80211 quits [Excess Flood]
17:33:04fuzzy80211 joins
17:33:04fuzzy80211 quits [Changing host]
17:33:04fuzzy80211 (fuzzy80211) joins
17:33:29fuzzy80211 quits [Excess Flood]
17:33:51fuzzy80211 (fuzzy80211) joins
17:38:28<TheoH7>My crawl has now finished and I have a warc file. Linked below are all the files in the folder grab-site created, including the warc.
17:38:31<TheoH7>https://www.swisstransfer.com/d/0683ec49-3b87-4cc9-8333-1e5a9df119e3
17:39:51<TheoH7>As was mentioned earlier it would be great if this warc could be added to the systems mentioned (even if not WBM)
17:41:20tekulvw (tekulvw) joins
17:45:12APOLLO03 quits [Ping timeout: 268 seconds]
17:48:54tekulvw quits [Ping timeout: 268 seconds]
17:50:23<@arkiver>TheoH7: please upload it to IA
17:50:29<@arkiver>but it will not go into the Wayback Machine
17:50:35<@arkiver>part of the reason is overriding /etc/hosts
17:51:05<@arkiver>but also because not just any contributed WARCs can go into the Wayback Machine
17:55:40<TheoH7>I think the Wayback Machine does have at least some parts of the site archived, maybe because it as an exception to the Cloudflare security layer.
17:56:00TheEnbyperor_ quits [Read error: Connection reset by peer]
17:56:00TheEnbyperor quits [Read error: Connection reset by peer]
17:56:35<TheoH7>What are the criteria exactly? I assume using DNS only as intended is one.
17:58:18TheEnbyperor (TheEnbyperor) joins
17:59:00TheEnbyperor_ joins
18:10:49<steering>TheoH7: my understanding is that they need to come from a vetted, trusted organization in order to be processed into WBM.
18:11:10<steering>(i.e. Archive Team, or Alexa back in the day)
18:16:56<Guest><https://wiki.archiveteam.org/index.php/Frequently_Asked_Questions#I_uploaded_a_WARC_file_but_why_doesn't_it_show_up_in_Wayback_Machine?>
18:21:26bronsen (bronsen) joins
18:22:11Ointment8862 quits [Quit: Lost terminal]
18:25:09<TheoH7>Well anyway, it seems that crawl wasn't finished despite being shown as finished via web status. I'll hold off uploading till IA till it finishes.
18:25:19tekulvw (tekulvw) joins
18:30:52APOLLO03 joins
18:32:41tekulvw quits [Ping timeout: 268 seconds]
18:36:36<Cupping1285>TheoH7 thanks for crawling though :D
18:37:36tekulvw (tekulvw) joins
18:38:05kansei- quits [Quit: ZNC 1.10.1 - https://znc.in]
18:40:31kansei (kansei) joins
18:42:13tekulvw quits [Ping timeout: 272 seconds]
18:46:24<TheoH7>Cupping1285: Well in that WARC if you go to replayweb.page and try and access it some of the links on the "Library page" point to community.ja.net, which then has a "please wait this page will reload after authentication" thing
18:46:36<TheoH7>I really hope I don't have to re-run it with community.ja.net as another dns override
18:47:25<TheoH7>Do you think that's needed?
18:53:10<Cupping1285>As far as I can tell, that domain and community.jisc.ac.uk is just the same page with another domain name. Up to do, really depends what is lost.
18:58:06<TheoH7>Cupping1285: I probably will. Only issue is I'll then start getting domain HTTPS cert mismatches. CanI run grab-site in a mode where it'll ignore invalid cert warnings?
19:01:13rohvani quits [Ping timeout: 272 seconds]
19:02:47<TheoH7>Looks like it might be the --insecure option
19:02:54h|ca2 quits [Ping timeout: 268 seconds]
19:07:31<bronsen>hello! I am trying this docker thing with my rootless docker because I don't want to recompile my kernel to (dis|en)able vmx something or other. I managed to start watchtower and a warrior. But `docker logs watchtower` shows this: https://hastebin.ianhon.com/6515 Is this normal?
19:08:40<Cupping1285>TheoH7 http://community.jisc.ac.uk/ is the same ips on the back side while http://community.ja.net/ is not under cloudflare.
19:09:39<Cupping1285>bronsen no
19:10:28rohvani joins
19:13:31<TheoH7>It looks like --insecure is not an option. Will it push past HTTPS mismatch errors already?
19:14:43<bronsen>what can I do to help make it normal?
19:16:12<pokechu22>TheoH7: Try --no-check-certificate
19:17:42rohvani quits [Ping timeout: 268 seconds]
19:19:15Goofybally9 joins
19:22:45Goofybally quits [Ping timeout: 272 seconds]
19:34:05rohvani joins
19:40:37<@arkiver>TheoH7: it's still very welcome at IA :)
19:44:08Webuser879359 joins
19:44:19Webuser879359 quits [Client Quit]
19:44:48<TheoH7>archiver: Currently just trying to get as much as I can before shutdown. I'm storing it all and will upload (likely tomorrow)
19:48:07<klea>s/archiver/arkiver/
19:48:53<klea>TheoH7: Does SwissTransfer let you add more content after the fact?
19:49:41<TheoH7>Before I start crawl number 2, is there a way to make it do more parallel connections?
19:50:00<TheoH7>I have limited time before shutdown, which is apparently in about 4 hours
19:50:43<TheoH7>klea: no it doesn't, it was just a quick way to post that warc here (which turned out to still be WIP). Will upload to IA tomorrow once I've got what I can get today.
19:55:11<klea>TheoH7: concurrency apparently?
19:55:19<klea>It seems to be a file, so I suppose you could change that.
19:55:34<klea>But also keep in mind this: <https://github.com/ArchiveTeam/wpull/issues/397>
19:56:38<klea>IPs logged here for posterity: https://transfer.archivete.am/IqTCL/dns-community.ja.net.txt
19:56:39<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/IqTCL/dns-community.ja.net.txt
19:57:31<klea>Also, I asked because I'll keep a local dump of the swisstransfer thing, so it can be uploaded if something were to happen to you.
19:57:50bronsen leaves [WeeChat 4.6.3]
19:58:56<TheoH7>klea: So can I just edit that file after I start? It doesn't seem to exist until I start it.
19:59:17<klea>I believe yes.
19:59:26<klea>The same as ignores iirc.
19:59:57mr_sarge quits [Read error: Connection reset by peer]
19:59:59petrichor quits [Client Quit]
20:00:27<masterx244|m>thats why that file exists. before crawl start you can set the initial value with a parameter. the file is there so you can tweak it at runtime since wpull doesnt have a terminal UI for tweaking stuff
20:01:42Webuser335519 joins
20:02:01kansei quits [Client Quit]
20:02:24kansei (kansei) joins
20:02:33<TheoH7>--no-check-certificate seems not to work for me.
20:02:51<TheoH7>Is that the right argument to bypass cert checking? Once I have that I should be all set to start.
20:05:37petrichor (petrichor) joins
20:06:00tekulvw (tekulvw) joins
20:08:46<TheoH7>Should it be --no-certificate-check instead?
20:08:55<pokechu22>There's also --no-strong-crypto but I don't think that's what you need here. Archivebot uses those two
20:10:53tekulvw quits [Ping timeout: 272 seconds]
20:12:33cipherrot (petrichor) joins
20:12:57<TheoH7>It seems neither no-certificate-check and no-check-certificate both come up as invalid argument still unfortunately :(
20:13:49petrichor quits [Ping timeout: 268 seconds]
20:17:55<pokechu22>Oh, I guess I'm checking parameters used by wpull, not grab-site
20:18:45<pokechu22>Try --wpull-args="--no-certificate-check --no-strong-crypto"
20:19:06<pokechu22>(looks like --no-skip-getaddrinfo also goes in --wpull-args)
20:27:34h|ca2 (h) joins
20:27:38PC quits [Remote host closed the connection]
20:28:01PC (PC) joins
20:28:01<TheoH7>Currently getting the following when using --wpull-args:grab-site: error: unrecognized arguments: --no-certificate-check
20:31:33<TheoH7>Got success with no-check-certificate (words wrong way round). will report back on progress
20:54:55corentin quits [Quit: Ping timeout (120 seconds)]
20:55:43corentin joins
20:56:17<klea>Btw, how should I deal with archiving links like https://limewire.com/d/7xNKB#NfXjrIqBWo ?, I've done a warcprox download with a headfull browser, but I think it'd be best to upload the original file too along with the warc, even if that duplicates the content, as it'd make it easier for others to get, since my content is not indexed in WBM.
21:19:55tekulvw (tekulvw) joins
21:24:44tekulvw quits [Ping timeout: 268 seconds]
21:40:09lukash984 quits [Quit: The Lounge - https://thelounge.chat]
21:45:47lukash984 joins
21:46:49lukash984 quits [Client Quit]
21:56:07lukash984 joins
21:59:17tekulvw (tekulvw) joins
22:04:05Arcorann (Arcorann) joins
22:04:15tekulvw quits [Ping timeout: 272 seconds]
22:18:11Wohlstand (Wohlstand) joins
22:21:36Webuser335519 quits [Quit: Ooops, wrong browser tab.]
22:39:13sec^nd quits [Remote host closed the connection]
22:39:44sec^nd (second) joins
22:43:27<TheoH7>Crawls are coming along well. I will upload both WARC's I am creating to IA tomorrow, not sure if they could be merged.
22:46:43tekulvw (tekulvw) joins
22:47:06etnguyen03 (etnguyen03) joins
22:47:48h|ca2 quits [Client Quit]
22:51:22<klea>No need to merge them.
22:51:45tekulvw quits [Ping timeout: 272 seconds]
23:03:02<cruller>RE: community.jisc.ac.uk, My crawl is almost complete. I hasten to send the fake CDX generated by cdxj-index (https://transfer.archivete.am/5Ut6w/community.jisc.ac.uk_11.cdx.zst) and final ignores (https://transfer.archivete.am/GUZ4E/ignores).
23:03:03<eggdrop>inline (for browser viewing): https://transfer.archivete.am/inline/5Ut6w/community.jisc.ac.uk_11.cdx.zst) https://transfer.archivete.am/inline/GUZ4E/ignores).
23:03:30h|ca2 (h) joins
23:12:48Wohlstand quits [Client Quit]
23:19:28<TheoH7>Thank you all for your help in preserving https://community.jisc.ac.uk It is a resource I have enjoyed reading for many years, and am glad it will be available in archive form for the years to come.
23:20:21<TheoH7>I also have two crawls: 1 with a WARC of about 1.2 GB which was at 100 concurrency and completed, and one that's still going at 2 concurrency which I think is even larger. The 100 concurrency one ran into some DNS issues for external (non community.jisc.ac.uk) pages at the end but I do not think those are significant.
23:20:22Wohlstand (Wohlstand) joins
23:20:32Dada quits [Remote host closed the connection]
23:20:50<TheoH7>I'm pleased to report that in that WARC I created, the key links which were broken before now work.
23:24:12<klea>I wonder, should I dump the ban lists someone is pushing to a ftp server I manage into IA periodically? I guess probably not since it's not user-generated content, and it's likely just a list of a bunch of ASN's IPs.
23:27:01Wohlstand quits [Client Quit]
23:31:31<Cupping1285>TheoH7 YAY!
23:48:42<TheoH7>cruller: Once complete it would be useful to have the WARC from your crawl so I can compare.
23:49:08<TheoH7>Is it possible to merge multiple WARC's together, or is it better to treat them as separate copies even if some are incomplete?
23:49:36<klea>In this case, I'd say treat them as separate warcs.
23:50:00<TheoH7>The scheduled shutdown is in 11 minutes, let's see if that's accurate or it ends up living through to the next UK working day
23:50:28<klea>(AT's SOP is separate warcs with AB jobs like wpull makes, but join into megawarcs with special tooling for warrior warcs, I believe)
23:51:23<TheoH7>Maybe once all crawls are done a megawarc could be created? Or is that not needed here?
23:52:13<klea>Nah, at least with AB, it doesn't make megawarcs, I believe.
23:53:07<klea>It's mostly with warriors because they make *very* small warcs (a warc for a single work item I believe), so then it runs over the IA item limit (is there such a thing?), so they have to make megawarcs.
23:56:03tekulvw (tekulvw) joins
23:58:45<TheoH7>When it comes time to upload to IA, should I just upload the two WARC's or the .cdx, 4MB WARC with meta in the filename, and all the other miscellaneous files?

HomeSearchPrevious dayNext day