HomeSearchPrevious dayNext day

00:05:17DigitalDragons quits [Client Quit]
00:05:35DigitalDragons (DigitalDragons) joins
00:09:19etnguyen03 (etnguyen03) joins
00:38:26nexusxe joins
00:40:11<nexusxe>in211.communityos.org/apssreadonly/render/id/0 through 10502 have no coverage - these are web pages for Indiana community resources detailing things like township assistance
00:40:39chunkynutz60 joins
00:41:27chunkynutz6 quits [Read error: Connection reset by peer]
00:41:27chunkynutz60 is now known as chunkynutz6
00:44:08<nexusxe>it appears that many smaller townships' only web presence is one of these entries, so imo they should be archived
00:44:31<nexusxe>based on my scattershot check of a few of them, it seems that none of them are archived
00:44:57<nexusxe>oh, i got responded to in #archivebot, nvm lol
00:57:43DartRetaliator_ joins
01:14:44Wohlstand quits [Ping timeout: 260 seconds]
01:30:07abirkill- (abirkill) joins
01:32:41abirkill quits [Ping timeout: 276 seconds]
01:32:42abirkill- is now known as abirkill
01:32:52etnguyen03 quits [Client Quit]
01:41:02etnguyen03 (etnguyen03) joins
01:57:36dabs quits [Read error: Connection reset by peer]
02:14:49shinon71 quits [Ping timeout: 260 seconds]
02:16:08BornOn420 quits [Remote host closed the connection]
02:16:46BornOn420 (BornOn420) joins
02:24:45shinon71 joins
02:25:34sec^nd quits [Ping timeout: 264 seconds]
02:28:15sec^nd (second) joins
02:34:18tzt quits [Remote host closed the connection]
02:34:37tzt (tzt) joins
02:38:09DartRetaliator_ quits [Ping timeout: 260 seconds]
02:42:56etnguyen03 quits [Remote host closed the connection]
02:50:32Exorcism0666 quits [Client Quit]
02:50:32DigitalDragons quits [Client Quit]
02:50:44Exorcism0666 (exorcism) joins
02:50:47DigitalDragons (DigitalDragons) joins
02:53:54nexusxe quits [Ping timeout: 260 seconds]
02:56:43Sokar joins
03:12:43DigitalDragons quits [Client Quit]
03:13:02DigitalDragons (DigitalDragons) joins
03:15:57nexusxe joins
03:19:14cuphead2527480 quits [Quit: Connection closed for inactivity]
03:30:23gosc joins
03:45:23nexusxe quits [Remote host closed the connection]
03:47:00nexusxe joins
03:57:33gosc quits [Client Quit]
04:14:09hackbug quits [Remote host closed the connection]
04:18:51hackbug (hackbug) joins
04:26:57chunkynutz6 quits [Quit: The Lounge - https://thelounge.chat]
04:27:17chunkynutz60 joins
05:08:55Webuser007984 joins
05:08:57Webuser007984 quits [Client Quit]
06:07:49DogsRNice quits [Read error: Connection reset by peer]
06:24:21awauwa (awauwa) joins
06:26:01<pabs>c3manu: re Anubis, try different UAs on these https://wiki.archiveteam.org/index.php/Anubis/uncategorized
06:41:26Juest quits [Ping timeout: 276 seconds]
06:42:52Juest (Juest) joins
06:48:07ArchivalEfforts quits [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
06:48:22ArchivalEfforts joins
06:48:40ArchivalEfforts quits [Client Quit]
06:48:48ArchivalEfforts joins
06:51:22Webuser417748 joins
06:54:20Juesto (Juest) joins
06:55:05Juest quits [Ping timeout: 276 seconds]
06:55:18Juesto is now known as Juest
06:56:29chrismeller8 quits [Read error: Connection reset by peer]
06:56:50chrismeller8 (chrismeller) joins
06:57:33Webuser417748 quits [Client Quit]
06:58:54DartRetaliator_ joins
07:00:29Juesto (Juest) joins
07:01:49Juest quits [Ping timeout: 260 seconds]
07:01:49Juesto is now known as Juest
07:05:58Exorcism0666 quits [Quit: Ping timeout (120 seconds)]
07:06:10Exorcism0666 (exorcism) joins
07:06:11DigitalDragons quits [Quit: Ping timeout (120 seconds)]
07:06:25DigitalDragons (DigitalDragons) joins
07:07:26Juest quits [Ping timeout: 276 seconds]
07:07:59Juest (Juest) joins
07:24:40Exorcism0666 quits [Client Quit]
07:24:56Exorcism0666 (exorcism) joins
07:25:03DigitalDragons quits [Client Quit]
07:25:21DigitalDragons (DigitalDragons) joins
08:48:09pixel (pixel) joins
08:51:28Dada joins
08:55:58Wohlstand (Wohlstand) joins
09:21:59DigitalDragons quits [Client Quit]
09:22:16DigitalDragons (DigitalDragons) joins
09:24:27<@arkiver>i'm going to set --warc-tempdir on various projects now to point at the temporary /data dr
09:24:29<@arkiver>data/*
09:31:53Island quits [Read error: Connection reset by peer]
09:34:54<pabs>OrIdow6++
09:34:54<eggdrop>[karma] 'OrIdow6' now has 8 karma!
09:38:29<@arkiver>change for the temporary WARC dir for all long term running projects
09:40:21<@arkiver>thanks for bringing up the temporary files earlier fuzzy8021 , else would not have noticed this change should be made
09:40:43<@arkiver>now, we could still prevent the extra write of the temporary WARC file
09:41:09<@arkiver>but putting it in data/ together with the rest already saves a lot for those using RAM for data/
09:42:06<@arkiver>not yet setting a minimum version in the tracker, it's not a vital change, so just allowing people to update whenever
09:48:44cuphead2527480 (Cuphead2527480) joins
09:51:34<@arkiver>chfoo: do you see possible problems with the warrior VM and moving the WARC temp dir to data/ ?
09:57:18<@arkiver>there is a significant drop in items completing on various projects since that change, will give it a while to see what happens
09:57:53<@arkiver>maybe just updates happening slowly, but we'll see
09:59:57<awauwa>been just getting instant failures now
10:00:26<awauwa>on goo-gl
10:01:10<@arkiver>awauwa: do you have a log for me?
10:02:33<@arkiver>something is off indeed, but i don't have great insight into it, need some logs from people who see the problems
10:02:46<awauwa>if you can tell me where to find them sure :D
10:03:26<awauwa>Because in the web management I just see Failed WgetDownload
10:13:56<that_lurker>spinning up a test warrior
10:19:11TheEnbyperor quits [Ping timeout: 276 seconds]
10:19:34TheEnbyperor_ quits [Ping timeout: 260 seconds]
10:26:50TheEnbyperor (TheEnbyperor) joins
10:27:23<that_lurker>arkiver: Yeah Warrior does not like the temp dir change. It gives no error
10:27:51<that_lurker>other than the mentioned insta fail on wget
10:27:58TheEnbyperor_ joins
10:28:35<awauwa>ah good, so it's not just me not finding any logs :'D
10:36:17<@arkiver>yeah
10:36:22<@arkiver>well turning this back
10:36:26<@arkiver>will test myself
10:37:32<that_lurker>warrior seems to be hard coded to use the /data/data it creates, so most likely warrior needs a code change as well
10:44:30<@arkiver>yeah i'll just do some testing myself first
10:44:35@arkiver thought it was an easy change
10:46:15<@arkiver>all changed back
10:46:23<@arkiver>should recover over the next 30 minutes
10:58:53VerifiedJ quits [Quit: The Lounge - https://thelounge.chat]
10:59:21LddPotato_ joins
10:59:26VerifiedJ (VerifiedJ) joins
11:02:09LddPotato quits [Ping timeout: 260 seconds]
11:02:12LddPotato_ is now known as LddPotato
11:34:11camrod636 (camrod) joins
12:09:03fuzzy8021 is now known as fuzzy80211
12:10:35<fuzzy80211>thanks for working on this arkiver
12:11:57<fuzzy80211>credit should go t9 nicolas17 for tracking down the tmp issue
12:12:26<@arkiver>nicolas17: just checking was your https://transfer.archivete.am/inline/x9qyW/updates.cdn-apple.com-xcode-simulators.txt archived?
12:12:48<@arkiver>nulldata: do you and how do you want me to credit you for the windows update drivers list?
12:13:59egallager joins
12:16:19<@arkiver>windows update project starting in a bit
12:16:28<@arkiver>72 TB from nulldata that we can take in very fast likely
12:17:39<@arkiver>imer: could we get a target for windowsupdatedrivers? it's 72 TB, but unclear deadline... so the faster we can take it in, the better
12:17:44<@arkiver>but upload speed to IA are good nowadays
12:17:47<@arkiver>this owuld be
12:17:57<@arkiver>Archive Team Windows Update Drivers:
12:17:58<@imer>sure thing
12:18:06<@arkiver>archiveteam_windowsupdatedrivers_
12:18:10<@arkiver>windowsupdatedrivers_
12:18:41<@arkiver>and in the future we may do (i hope) a more general Windows Update project, just such influential/impactful stuff in there, if nulldata wants to sort out more downloads
12:18:47<@arkiver>but we'll see
12:31:04<@imer>arkiver: target is added
12:31:09<@arkiver>yay!
12:31:15<@arkiver>thanks a lot as always :)
12:31:27Guest58_ joins
12:31:32Guest58 quits [Read error: Connection reset by peer]
12:32:09Guest58 joins
12:33:40Guest58_ quits [Read error: Connection reset by peer]
12:36:24<h2ibot>Arkiver uploaded File:Windowsupdate-icon.png: https://wiki.archiveteam.org/?title=File%3AWindowsupdate-icon.png
12:50:11<Hans5958>wuaudrivers
12:50:15<Hans5958>Ah I'm too late
13:01:20<@arkiver>there's more to it than simply archiving the URLs in the file from nulldata, also web pages like https://www.catalog.update.microsoft.com/ScopedViewInline.aspx?updateid=8A23DDCC-F0EA-426A-8A5D-0001463E9165
13:01:27<@arkiver>shall we have a channel? any ideas?
13:13:30DigitalDragons quits [Quit: Ping timeout (120 seconds)]
13:13:43DigitalDragons (DigitalDragons) joins
13:16:25<masterx244|m>maybe a more generic update-files project since stuff like that happens elsewhere, too
13:16:29<@arkiver>imer: can we change it to
13:16:37<@arkiver>Archive Team Windows Update:
13:16:44<@arkiver>archiveteam_windowsupdate_
13:16:47<@arkiver>windowsupdate_
13:16:47<@arkiver>?
13:16:55<@imer>sure thing, different tracker as well or still the same?
13:17:06<@arkiver>imer: i will change the tracker to windowsupdate
13:17:12<@arkiver>masterx244|m: you mean only for windows update?
13:17:30<@arkiver>imer: created
13:29:17<egallager>https://ddosecrets.com/article/port-of-aqaba
13:32:45<@arkiver>nice seeing the firewire drivers in there :) https://www.catalog.update.microsoft.com/Search.aspx?q=firewire
13:34:54<masterx244|m>arkiver: more in general. Software update files often tend to disappear unless caught before purgination/vendor disappearing
13:35:14<masterx244|m>(firmware updates are more annoying on that topic, too)
13:35:29<@arkiver>masterx244|m: it would be nice to have a general channel for them, yeah
13:35:41<@arkiver>and if there's long term dedicated projects, to have separate channels for those
13:35:52<@imer>was poking around a bit there as well, not found a good way for searching for new updates unfortunately aside from searching for like aaa aab etc. with sort by date since it does tell you when there's more than 1k results
13:36:44<@arkiver>imer: yeah i hope we can figure out a way to somewhat reliably get everything
13:36:59<@imer>maybe someone more familiar with how windows updates work has an idea
13:37:17<@arkiver>so the name of that catalog is "Microsoft Update Catalog", but the actual updates are served by "Windows Update"
13:37:27<@arkiver>(download.windowsupdate.com)
13:37:37<@imer>oh, target should be up as well
13:37:44<@arkiver>there's also a HTTP only and HTTPS only URL for each
13:39:41<@arkiver>Windows Mobile is also in there https://www.catalog.update.microsoft.com/Search.aspx?q=windows+mobile
13:40:15lunax (lunax) joins
13:40:56<masterx244|m>annoying part is that when windows searches for updates that there is 2-way talk and not a dumb "download list" step that we could emulate.
13:41:32<@arkiver>masterx244|m: the catalog posted a message earlier seems to have everything, also the various architectures
13:42:06<masterx244|m>was mainly on a machine-readable way to rig a automation for detecting new stuff
13:42:15<@arkiver>like https://www.catalog.update.microsoft.com/Search.aspx?q=KB5063327
13:42:46<masterx244|m>wonder if we could "bruteforce" around the newest ID once we got a backfill since all updates have a KB-ID
13:42:52archiveDrill quits [Quit: The Lounge - https://thelounge.chat]
13:42:58<masterx244|m>and those are sequential incrementing
13:43:19<@arkiver>yep
13:43:51<@arkiver>they don't all have it
13:43:55<@arkiver>but many of the recent ones do
13:44:04<@arkiver>but maybe nulldata has some interesting ideas here
13:44:42<masterx244|m>much easier to track than some firmwares where the file naming scheme changed somewhere in the middle of the device lifetime. old files still on the server but unreachable with the new naming scheme. my monitoring started before that cutoff luckily so i still got those files
13:45:09<masterx244|m>(and i dumped that URL list a while ago with a request for !ao< since compared to other files its a relatively small dataset
13:45:10<masterx244|m>)
13:45:39<@arkiver>nice we can use the UUIDs
13:47:05<@arkiver>for example https://www.catalog.update.microsoft.com/ScopedViewInline.aspx?updateid=067c7e08-c540-45c8-8b97-da27bbcb208e take the updateid and search finds it https://www.catalog.update.microsoft.com/Search.aspx?q=067c7e08-c540-45c8-8b97-da27bbcb208e take a part of it, add a star and it'll still find it https://www.catalog.update.microsoft.com/Search.aspx?q=067c7e0* or take off even more characters https://www.catalog.update.microsoft.com/Search.a
13:47:05<@arkiver>spx?q=067c* and it finds a ton more
13:47:28<@arkiver>the UUIDs should be random, so we can enumerate everything through these searches
13:48:31cuphead2527480 quits [Quit: Connection closed for inactivity]
13:50:08<masterx244|m>65k searches for enum. if we catch a bucket with more than 1k Updates it needs a split but that should be reasonably automatable for sweeping in a automated way for discovery
13:50:29<@arkiver>yep
13:50:35<masterx244|m>(bucket in the sense of hashmap-bucket)
13:52:07<@arkiver>we can also scan by date https://www.catalog.update.microsoft.com/Search.aspx?q=2024+november+0caa*
13:52:56<@arkiver>https://www.catalog.update.microsoft.com/Search.aspx?q=2024+november+0c0*
13:53:59<@arkiver>not bad, so rescan ~256 searches a few times a month https://www.catalog.update.microsoft.com/Search.aspx?q=2025+july+ab*
13:55:14<@arkiver>it finds the 1968 stuff since there's some with a mention of 2025 and july
13:56:17<masterx244|m>best to thwack one short past 2nd tuesday of a month, that search is guaranteed to yield new files
13:56:17Sokar quits [Read error: Connection reset by peer]
13:56:43Sokar joins
13:56:46<masterx244|m>(the other tuesdays, too but those are the previews and other noncritical stuff)
13:57:44<@arkiver>sounds good
13:58:12<@arkiver>intel was busy with bluetooth updates here https://www.catalog.update.microsoft.com/Search.aspx?q=2025+march+abc*&p=1
13:58:14<masterx244|m>1968 is the faked driver date. windows uses the driver date as the primary version comparator and those drivers with the 1968 date or similar are fallback drivers
13:58:28<masterx244|m>(only reached if nothing newer matches)
13:59:13<@arkiver>i see
13:59:19<@arkiver>well i think we have a way to do this now
13:59:50<@arkiver>but since this is not anymore a simple "get all these URLs" project, i'll move launch to tomorrow, want to implement search too
13:59:59<@arkiver>masterx244|m: project channel ideas? :)
14:00:48<masterx244|m>windowfixer?
14:01:01<@arkiver>:P
14:01:01<katia>windowlicker
14:01:20<@arkiver>fixer sounds better, since we're preserving these updates that fix things
14:01:49<katia>https://www.youtube.com/watch?v=FATTzbm78cc
14:02:20<awauwa>windowframe :D (you know, keeping the windows together and in place :D)
14:02:29<katia>xD
14:03:05<@arkiver>let's do #windowfixer
14:03:19<@arkiver>nulldata: FYI we're continuing in #windowfixer
14:22:29<@imer>masterx244|m++
14:22:30<eggdrop>[karma] 'masterx244|m' now has 2 karma!
14:28:51<masterx244|m>first time that i managed to define a channelname
14:31:30pseudorizer quits [Quit: ZNC 1.10.1 - https://znc.in]
14:34:33<c3manu>pabs: you mean to check whether the listed domains still work with Anubis?
14:35:03pseudorizer (pseudorizer) joins
14:38:56DigitalDragons quits [Client Quit]
14:38:59<c3manu>pabs: i read that Anubis now supports no-JS challenges, but i didn't expect that to affect the exception rule, given the author came here proactively to talk to us
14:39:02<c3manu>https://anubis.techaro.lol/blog/release/v1.20.0
14:39:11DigitalDragons (DigitalDragons) joins
14:44:59anonymoususer852 quits [Ping timeout: 260 seconds]
14:46:46anonymoususer852 (anonymoususer852) joins
15:08:11FiTheArchiver joins
15:10:41FiTheArchiver1 joins
15:14:17FiTheArchiver quits [Ping timeout: 276 seconds]
15:31:23DigitalDragons quits [Client Quit]
15:31:23Exorcism0666 quits [Quit: Ping timeout (120 seconds)]
15:31:34Exorcism0666 (exorcism) joins
15:31:36DigitalDragons (DigitalDragons) joins
15:35:51dabs joins
15:39:14dave quits [Ping timeout: 260 seconds]
15:40:59Exorcism0666 quits [Client Quit]
15:41:03DigitalDragons quits [Client Quit]
15:41:10Exorcism0666 (exorcism) joins
15:41:20DigitalDragons (DigitalDragons) joins
15:46:20dave (dave) joins
15:53:49^ quits [Ping timeout: 260 seconds]
15:53:56anonymoususer852 quits [Ping timeout: 276 seconds]
15:57:03^ (^) joins
15:57:28grill (grill) joins
16:02:40anonymoususer852 (anonymoususer852) joins
16:09:32abirkill quits [Ping timeout: 276 seconds]
16:15:17<justauser|m>Are project-specific channels normally logged?
16:20:13<@imer>justauser|m: no
16:26:29cm quits [Ping timeout: 260 seconds]
16:26:41cm joins
16:48:48Webuser533499 joins
16:49:17<Webuser533499>hello, anyone here? I would like to contribute to an archiving project and I would love some help
16:49:26<Webuser533499>not really used to irc
16:56:14<justauser|m>Details please?
16:56:46<justauser|m>You can look there to start with:
16:56:47<justauser|m>https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior
17:00:01<Webuser533499>> Details please?
17:00:01<Webuser533499>it's on e621. i am writing a scraper for it cause my use case needs it. it just so happens that the data collected can be given to archive team
17:03:43<justauser|m>If you just have a data you want to give someone, just upload it to IA.
17:05:11<Webuser533499>will do
17:05:36Webuser533499 quits [Client Quit]
17:05:49i_have_n0_idea3 quits [Quit: The Lounge - https://thelounge.chat]
17:06:07i_have_n0_idea3 (i_have_n0_idea) joins
17:06:24<justauser|m>If it's in form of WARC, post there for Wayback inclusion, otherwise simply make sure to give it a good metadata(name, description, etc.)
17:11:42Exorcism0666 quits [Client Quit]
17:11:58Exorcism0666 (exorcism) joins
17:12:10DigitalDragons quits [Client Quit]
17:12:25DigitalDragons (DigitalDragons) joins
17:18:59khaoohs quits [Ping timeout: 260 seconds]
17:25:31cuphead2527480 (Cuphead2527480) joins
17:28:26<TheTechRobo>Note that WARCs from most people won't go into the Wayback Machine due to concerns of integrity.
17:29:08<TheTechRobo>Lists of stuff is really helpful though - feel free to upload to transfer.archivete.am and post here
17:30:13<h2ibot>HadeanEon edited Deaths in 2025 (+3650, BOT - Updating page: {{saved}} (143),…): https://wiki.archiveteam.org/?diff=56462&oldid=56459
17:30:14<h2ibot>HadeanEon edited Deaths in 2025/list (+199, BOT - Updating list): https://wiki.archiveteam.org/?diff=56463&oldid=56460
17:31:07<pokechu22>note that we did also do some other scraping already documented at https://wiki.archiveteam.org/index.php/E621 (based on their data dumps)
17:34:42DigitalDragons quits [Client Quit]
17:34:57DigitalDragons (DigitalDragons) joins
17:36:34<nicolas17>arkiver: xcode-simulators.txt finished archiving via archivebot, I think it hasn't yet been indexed by WBM
17:44:37DigitalDragons quits [Client Quit]
17:44:53DigitalDragons (DigitalDragons) joins
17:46:31Jens quits []
17:47:03Jens (JensRex) joins
18:16:09grill quits [Ping timeout: 260 seconds]
18:22:36grill (grill) joins
18:52:19midou quits [Ping timeout: 260 seconds]
19:02:22awauwa quits [Quit: awauwa]
19:02:33midou joins
19:16:45Wohlstand quits [Quit: Wohlstand]
19:17:00Wohlstand (Wohlstand) joins
19:20:37<nicolas17>time to once again download 80GB myself and upload the warcs because we have no good tool for deduplicated archiving~
19:23:27khaoohs joins
19:38:31cuphead2527480 quits [Client Quit]
20:13:49DigitalDragons quits [Client Quit]
20:14:03DigitalDragons (DigitalDragons) joins
20:23:11notarobot17 quits [Quit: Ping timeout (120 seconds)]
20:23:28notarobot17 joins
20:23:41DigitalDragons quits [Client Quit]
20:23:58DigitalDragons (DigitalDragons) joins
20:45:37Exorcism0666 quits [Quit: Ping timeout (120 seconds)]
20:45:51Exorcism0666 (exorcism) joins
20:45:55DigitalDragons quits [Client Quit]
20:46:11Webuser053605 joins
20:46:13DigitalDragons (DigitalDragons) joins
20:46:42<Webuser053605>Hello
20:47:04<Webuser053605>I'm trying to find posts from a certain AskFM account but I don't have 2 terrabytes of storage space on my computer
20:47:22<Webuser053605>Is there any way I could just download part of the AskFM archive?
21:02:16Webuser053605 quits [Client Quit]
21:09:50grill quits [Ping timeout: 276 seconds]
21:17:06etnguyen03 (etnguyen03) joins
21:38:08Dada quits [Remote host closed the connection]
21:49:49Sokar quits [Read error: Connection reset by peer]
22:21:44etnguyen03 quits [Client Quit]
22:22:04etnguyen03 (etnguyen03) joins
22:23:47DigitalDragons quits [Client Quit]
22:24:03DigitalDragons (DigitalDragons) joins
22:44:20cuphead2527480 (Cuphead2527480) joins
22:45:58DigitalDragons quits [Client Quit]
22:46:14DigitalDragons (DigitalDragons) joins
22:47:48Island joins
22:49:56andrewnyr quits [Read error: Connection reset by peer]
22:57:21Wohlstand quits [Quit: Wohlstand]
23:04:53kedihacker quits [Ping timeout: 276 seconds]
23:26:34<egallager>https://ddosecrets.com/article/american-golf
23:30:37etnguyen03 quits [Client Quit]
23:44:56DigitalDragons quits [Client Quit]
23:45:13DigitalDragons (DigitalDragons) joins
23:49:44nine quits [Quit: See ya!]
23:49:57nine joins
23:49:57nine quits [Changing host]
23:49:57nine (nine) joins
23:54:13APOLLO03 quits [Remote host closed the connection]
23:54:25APOLLO03 joins
23:56:05kansei quits [Quit: ZNC 1.10.1 - https://znc.in]
23:58:14kansei (kansei) joins

HomeSearchPrevious dayNext day