00:07:22etnguyen03 quits [Client Quit]
00:13:48sheldon joins
00:19:23Kabaya quits [Client Quit]
00:19:37ats quits [Quit: kernel]
00:24:47ats (ats) joins
00:31:40Kabaya joins
00:32:37sheldon quits [Client Quit]
00:41:07beastbg8 joins
00:43:52Webuser229683 joins
00:44:37<Webuser229683>So, basically, it is not possible to get the original archive link of a tweet if a user changed their username?
00:45:41<pokechu22>If you don't know the original username, it probably would be difficult to do it, yeah
00:48:28Webuser229683 quits [Client Quit]
00:52:42Webuser818448 joins
00:52:52<Webuser818448>man, you used to be able to do it with the "i" trick
00:53:34<Webuser818448>I suppose looking through the replies of a twitter account and lookup up the archive links of the people replying to them (if they didnt change their username) would still work, however
01:00:28Webuser818448 quits [Client Quit]
01:04:21agtsmith quits [Ping timeout: 265 seconds]
01:05:45zwzw4132 joins
01:14:12etnguyen03 (etnguyen03) joins
01:36:46zwzw4132 quits [Client Quit]
01:38:43agtsmith joins
01:40:52zwzw4132 joins
01:41:00etnguyen03 quits [Ping timeout: 256 seconds]
01:49:56etnguyen03 (etnguyen03) joins
02:23:53polygon6a673c quits [Ping timeout: 260 seconds]
02:24:24polygon6a673c (polygon6a673c) joins
02:30:28h|ignition quits [Remote host closed the connection]
02:31:04h|ignition (h) joins
02:34:21h|ignition quits [Remote host closed the connection]
02:34:38h|ignition (h) joins
02:34:51NoNamo quits [Quit: Leaving]
02:38:17h|ignition quits [Remote host closed the connection]
02:38:37h|ignition (h) joins
02:45:33zwzw4132 quits [Client Quit]
03:31:23zwzw4132 joins
03:35:38grill quits [Ping timeout: 260 seconds]
03:36:23grill (grill) joins
03:59:58lukash984 quits [Quit: The Lounge - https://thelounge.chat]
04:05:04lukash984 joins
04:25:49DogsRNice quits [Read error: Connection reset by peer]
05:09:48etnguyen03 quits [Remote host closed the connection]
05:13:38Webuser473519 joins
05:20:48Webuser473519 quits [Client Quit]
05:27:56halloy8426 joins
05:32:41halloy8426 quits [Client Quit]
06:04:03Green116 joins
06:06:25Green11 quits [Ping timeout: 265 seconds]
06:06:26Green116 is now known as Green11
06:26:07d02 joins
06:32:05Green11 quits [Client Quit]
06:37:28MPThLee quits [Quit: bye]
06:39:02MPThLee (MPThLee) joins
06:50:55valdikss_ quits [Quit: Bye!]
06:51:47valdikss joins
06:54:22valdikss quits [Read error: Connection reset by peer]
06:56:26valdikss joins
07:00:07nexussfan quits [Remote host closed the connection]
07:20:51Umbire quits [Ping timeout: 265 seconds]
07:21:31zwzw4132 quits [Quit: Ooops, wrong browser tab.]
07:32:49Green11 joins
07:47:43halloy7171 joins
08:14:58<h2ibot>Brad edited IFTTT (+120, Changes to represent there name change from…): https://wiki.archiveteam.org/?diff=64964&oldid=59540
08:19:59<h2ibot>Brad edited Finding subdomains (+181, Added hacker target): https://wiki.archiveteam.org/?diff=64965&oldid=64869
08:22:04<h2ibot>Brad edited Finding subdomains (+31, corrected hacker target free api limit): https://wiki.archiveteam.org/?diff=64966&oldid=64965
08:23:12halloy7171 quits [Ping timeout: 265 seconds]
08:30:47awauwa (awauwa) joins
08:39:46Hackerpcs quits [Remote host closed the connection]
08:40:41Hackerpcs (Hackerpcs) joins
08:50:31Umbire joins
08:52:07<h2ibot>Brad edited ArchiveBot/Yellow Vests/list (+46, Added notes for Age-restriction and offline): https://wiki.archiveteam.org/?diff=64967&oldid=48243
08:55:18Umbire quits [Ping timeout: 260 seconds]
09:20:12<h2ibot>Manu edited Distributed recursive crawls (+53, Candidates: Add https://policingequity.org/): https://wiki.archiveteam.org/?diff=64968&oldid=64855
09:23:24ib4n quits [Ping timeout: 256 seconds]
09:36:14<h2ibot>Manu edited Discourse/active (+47, Found https://org-roam.discourse.group/): https://wiki.archiveteam.org/?diff=64969&oldid=64955
10:00:25Bleo182600722719623455222011079 quits [Quit: The Lounge - https://thelounge.chat]
10:03:08Bleo182600722719623455222011079 joins
10:04:21evergreen quits [Quit: Bye]
10:04:49evergreen joins
10:11:10NotKlea joins
10:11:36NotKlea is now known as RJHacker12799
10:29:24haiseiko (Haiseiko) joins
10:30:39yano quits [Quit: WeeChat, https://weechat.org/]
10:35:14yano (yano) joins
10:44:07<haiseiko>my github account nuked recenntly.is there any way to contribute to url source?
10:44:43<haiseiko>@c3manu:
10:47:03<haiseiko>@Ryz:
11:04:57nune quits [Remote host closed the connection]
11:05:00nune (nune) joins
11:11:24RedCat quits [Ping timeout: 265 seconds]
11:20:26<h2ibot>YFdyh000 edited Finding subdomains (-1, fix #): https://wiki.archiveteam.org/?diff=64971&oldid=64966
11:34:30<h2ibot>VoynichCr uploaded File:Atlocal.jpg: https://wiki.archiveteam.org/?title=File%3AAtlocal.jpg
11:36:31<h2ibot>KleaBot edited Main Page/Current Warrior Project (-151, Default projects are now metaadlibrary (weight…): https://wiki.archiveteam.org/?diff=64973&oldid=64958
11:49:32<h2ibot>Manu edited Distributed recursive crawls (+41, Candidates: Add https://psk.sk/): https://wiki.archiveteam.org/?diff=64975&oldid=64968
11:56:58pendulum quits [Remote host closed the connection]
11:57:12pendulum (pendulum) joins
12:04:33<h2ibot>Manu edited Webring/XXIIVV (-33, jobs finished): https://wiki.archiveteam.org/?diff=64976&oldid=64952
12:12:36<h2ibot>Manu edited Webring/XXIIVV (+450, Queued more websites): https://wiki.archiveteam.org/?diff=64977&oldid=64976
12:24:50murb quits [Quit: moo]
12:40:39<h2ibot>Cruller edited Deathwatch (+173, /* 2027 */ Add Misskey.art): https://wiki.archiveteam.org/?diff=64978&oldid=64923
13:00:40<h2ibot>YFdyh000 edited URLTeam/Dead (+39, short.im changed): https://wiki.archiveteam.org/?diff=64979&oldid=60122
13:07:41<h2ibot>Manu edited Distributed recursive crawls (+66, Candidates: Add…): https://wiki.archiveteam.org/?diff=64980&oldid=64975
13:12:41<h2ibot>Manu edited Backup Sachsen-Anhalt (+224, Queued more websites): https://wiki.archiveteam.org/?diff=64981&oldid=64957
13:13:20etnguyen03 (etnguyen03) joins
13:14:18murb (murb) joins
13:15:24pendulum quits [Remote host closed the connection]
13:15:53pendulum (pendulum) joins
13:29:22d02 quits [Remote host closed the connection]
13:29:52d02 joins
13:43:00<haiseiko>@c3manu my github account nuked recenntly. is there any way to contribute to url source?
13:45:10<c3manu>haiseiko: stop pinging me
13:45:43<c3manu>haiseiko: maybe try the project channel
13:55:28<haiseiko>sorry. i can't find other mod.s who still work
13:55:43<haiseiko>what do you mean?
14:04:56<c3manu>i'm not a mod
14:05:44<c3manu>https://wiki.archiveteam.org/index.php/URLs
14:10:46etnguyen03 quits [Client Quit]
14:58:17<Dango360>haiseiko: join #// for the url project chat
15:00:12<Dango360>and you can use https://transfer.archivete.am/ to upload text files with urls (just get a token from #archiveteam-transfer )
15:00:15RedCat joins
15:02:45nexussfan (nexussfan) joins
15:09:53<h2ibot>Manu edited Backup Sachsen-Anhalt (-10, jobs finished): https://wiki.archiveteam.org/?diff=64982&oldid=64981
15:15:55<h2ibot>Manu edited Backup Sachsen-Anhalt (+240, Queued more websites): https://wiki.archiveteam.org/?diff=64983&oldid=64982
15:17:56<h2ibot>Dango360 edited Template:CTA URL lists (+393, Add info about transfer upload tokens): https://wiki.archiveteam.org/?diff=64984&oldid=57536
15:22:56<h2ibot>Manu edited Backup Sachsen-Anhalt (+8, no working pipeline for…): https://wiki.archiveteam.org/?diff=64985&oldid=64983
15:25:56<h2ibot>Manu edited Backup Sachsen-Anhalt (-57, Jobs finished): https://wiki.archiveteam.org/?diff=64986&oldid=64985
15:38:57<h2ibot>Manu edited Backup Sachsen-Anhalt (+525, Queued more websites): https://wiki.archiveteam.org/?diff=64987&oldid=64986
15:41:58<h2ibot>Manu edited Backup Sachsen-Anhalt (+146, Queued more websites): https://wiki.archiveteam.org/?diff=64988&oldid=64987
15:57:59<h2ibot>Manu edited Backup Sachsen-Anhalt (-5, https://shop.hvb-harz.de/ finished): https://wiki.archiveteam.org/?diff=64989&oldid=64988
16:07:15h|ignition quits [Remote host closed the connection]
16:08:00<h2ibot>Manu edited Backup Sachsen-Anhalt (-5, https://amg.sachsen-anhalt.de/ finished): https://wiki.archiveteam.org/?diff=64990&oldid=64989
16:09:09haiseiko quits [Quit: Connection closed for inactivity]
16:12:54h|ignition (h) joins
16:23:44<cruller>I'm doing some research on firestorage, which will be shut down on November 24th at 0:00 (UTC).
16:24:10<cruller>Extracting URLs containing 40 alphanumeric characters from WBM CDX and excluding those that redirect to https://firestorage.jp/doc/newphotoinfo/ leaves 33,210. Of those, 29,463 have the prefix https://firestorage.jp/download/
16:24:44<cruller>After examining 100 of these, I found that 87 had been deleted and 1 was password protected. I expect the number of URLs requiring archiving to be fewer than 10,000.
16:32:40<klea>Should we make a channel for it?
16:32:51<klea>datafiring?
16:33:57<justauser>10KURLs sounds like a single AB job.
16:34:35<cruller>Here is probably enough?
16:34:45<cruller>I believe URLs starting with https://firestorage.jp/ja/d/ are the new version that is not subject to deletion this time, but I have no proof.
16:35:51<klea>Yeah, sounds good here tbh.
16:36:23<klea>cruller: Do you know if they need POST or some other special handling that AB can't do?
16:36:42etnguyen03 (etnguyen03) joins
16:38:27<klea>I'm going to be updating the wiki bot code to only show the project names without weights.
16:45:15BornOn420 quits [Remote host closed the connection]
16:50:08<h2ibot>Klea edited Template:CurrentWarrior (-136, Remove the weight info): https://wiki.archiveteam.org/?diff=64991&oldid=63009
16:50:42midou quits [Ping timeout: 265 seconds]
16:55:08<h2ibot>KleaBot edited Main Page/Current Warrior Project (-59, Default projects are now metaadlibrary,…): https://wiki.archiveteam.org/?diff=64992&oldid=64973
16:55:31BornOn420 (BornOn420) joins
16:55:52<cruller>POST isn't required, but a download link won't be displayed if Javascript is blocked. There is also a 10-minute expiration and a generous rate limit -> https://firestorage.jp/status-attack.html
16:57:07<klea>"if Javascript is blocked" sounds like perhaps AB won't be able to get trough.
16:57:13<cruller>Example: https://firestorage.jp/download/011b21c8c94574f67fa8585e4c1ba67b102250fe
16:58:46<klea>oh, nvm, it seems the link is in the HTML, so perhaps it can be done.
16:59:56Umbire joins
17:00:42Umbire quits [Remote host closed the connection]
17:01:01<justauser>arkiver: https://en.metapedia.org/ has something.
17:01:07Umbire joins
17:01:14<klea>(every other subdomain too?)
17:06:08nathang218438 joins
17:08:12RJHacker12799 quits [Quit: Quit]
17:10:02<cruller>After continuously wpulling with "--delay=250-375", I encountered another rate limit. https://alert.firestorage.jp/status-attack.cgi?ip=XXX&sessionkey=XXX -> https://firestorage.jp/recaptchav3.cgi -> https://firestorage.jp/access_deny/
17:11:28<cruller>--concurrency=1
17:12:06<klea>hmm.
17:12:14BornOn420 quits [Ping timeout: 250 seconds]
17:12:42<klea>cruller: Are you able to complete a captcha and continue archiving with wpull?
17:13:58<cruller>Yes!
17:17:41<klea>hmm, and if you don't do the captcha, you can't continue, right?
17:18:22midou joins
17:19:00kansei- (kansei) joins
17:20:11kansei quits [Ping timeout: 265 seconds]
17:20:19BornOn420 (BornOn420) joins
17:22:41<cruller>Yeah I was redirected to https://alert.firestorage.jp/status-attack.cgi?ip=XXX&sessionkey=XXX until I solved the CAPTCHA.
17:31:10Wuzado joins
17:42:51ib4n joins
17:49:03<klea>cruller: I assume you're already archiving firestorage.jp entirely, and will upload it to IA, right?
17:49:26<klea>(since the CAPTCHA likely won't be able to be completed in AB unless we have JAA available and checking every so often.)
17:50:48Umbire quits [Client Quit]
17:57:04<cruller>klea: Of course, it would be ideal if ArchiveTeam could handle the archiving, but if that's not possible, I intend to do so. I'm still n the investigation stage.
17:59:18<klea>Ah, makes sense.
17:59:25desmosanddeimos joins
17:59:58kansei- quits [Client Quit]
18:02:20desmosanddeimos quits [Client Quit]
18:02:28<cruller>I executed "grab-site --delay=1500-2500 --concurrency=1 --no-offsite-links --no-sitemaps --wpull-args=-D=firestorage.jp https://firestorage.jp/download/011b21c8c94574f67fa8585e4c1ba67b102250fe",but the link from https://firestorage.jp/download/{46_characters_ID} to https://firestorage.jp/index.cgi?act=download&key={46_characters_ID}&comoff=XXXXX wasn't followed.
18:03:43<cruller>Is it because of --no-parents?
18:04:47kansei (kansei) joins
18:05:09<justauser>Looks like it.
18:11:02<cruller>In that case, AB probably has the same issue, but could someone try running "!a https://firestorage.jp/download/011b21c8c94574f67fa8585e4c1ba67b102250fe"?
18:20:02etnguyen03 quits [Ping timeout: 256 seconds]
18:29:31<justauser>The job isn't finished yet, but it doesn't look like it worked.
18:29:38<justauser>cruller: ^
18:29:56msfjarvis quits [Quit: Lurker 2.6.3 https://lurker.chat]
18:30:55msfjarvis joins
18:31:16<h2ibot>Brad edited Deathwatch (+353, Added Microsoft To Do): https://wiki.archiveteam.org/?diff=64993&oldid=64978
18:33:27BornOn420 quits [Remote host closed the connection]
18:35:09<cruller>justauser: Thank you! There are many junk urls...
18:41:51balrog quits [Quit: Bye]
18:46:51BornOn420 (BornOn420) joins
18:47:04balrog (balrog) joins
19:03:53Umbire (Umbire) joins
19:21:58BornOn420 quits [Remote host closed the connection]
19:33:08BornOn420 (BornOn420) joins
19:35:43Webuser010659 joins
19:35:57d02 quits [Quit: Leaving]
19:37:22DogsRNice joins
19:41:26Webuser010659 quits [Client Quit]
19:42:13DogsRNice quits [Ping timeout: 260 seconds]
19:44:34<cruller>In the end, I decided to specify only the necessary URLs directly using --accept-regex. It's a crude method, but I hope it works.
19:45:09Webuser425287 joins
19:45:27Webuser425287 quits [Client Quit]
19:49:40HP_Archivist quits [Quit: Leaving]
19:50:03sg-72 quits [Quit: Leaving]
20:40:46etnguyen03 (etnguyen03) joins
21:13:31<h2ibot>YFdyh000 edited URLTeam (+6461, /* Alive */ Add many. no strict checks, no…): https://wiki.archiveteam.org/?diff=64994&oldid=64876
21:15:35awauwa quits [Quit: awauwa]
21:27:22<jmcdowan17384|m>koichi: Do you know how Niconico video description data was obtained for the uploaderjp project? I think it would also be a good source of URLs, but I do not know the details.
21:35:27Island quits [Read error: Connection reset by peer]
21:37:46Island joins
21:45:34<h2ibot>YFdyh000 edited URLTeam (+2022, add more in Probably Dead or Broken; no sorting): https://wiki.archiveteam.org/?diff=64995&oldid=64994
21:53:35<h2ibot>YFdyh000 edited URLTeam (+386, /* Probably Dead or Broken */ add c1n.cn): https://wiki.archiveteam.org/?diff=64996&oldid=64995
21:56:54Soul_ joins
21:56:54Soulflare quits [Read error: Connection reset by peer]
21:58:35<h2ibot>YFdyh000 edited URLTeam (+33, /* Alive */ fix tinu.be; fix typo): https://wiki.archiveteam.org/?diff=64997&oldid=64996
22:05:36<h2ibot>IDKhowToEdit edited ArchiveBot (+10, Update on https://control.archiveteam.net/): https://wiki.archiveteam.org/?diff=64998&oldid=64495
22:06:36<h2ibot>IDKhowToEdit edited ArchiveBot (+11, fix): https://wiki.archiveteam.org/?diff=64999&oldid=64998
22:11:41HP_Archivist (HP_Archivist) joins
22:18:02sepro quits [Ping timeout: 256 seconds]
22:21:04sepro (sepro) joins
22:23:07haiseiko (Haiseiko) joins
22:24:41sepro7 (sepro) joins
22:26:32sepro quits [Ping timeout: 256 seconds]
22:26:32sepro7 is now known as sepro
22:26:40<h2ibot>Petchea edited Tapatalk (+1741): https://wiki.archiveteam.org/?diff=65000&oldid=59951
22:27:40<h2ibot>YFdyh000 edited URLTeam (+309, add v.douyin.com; refine url.cn): https://wiki.archiveteam.org/?diff=65001&oldid=64997
22:35:49MrBlizzard joins
22:40:15<MrBlizzard>I wonder would it be useful to try adding discord like platforms as something that should be archived?
22:40:25<nicolas17>adding where?
22:40:57<MrBlizzard>well I mean like adding it as a site to consider archiving
22:41:11<nicolas17>https://wiki.archiveteam.org/index.php/Discord
22:41:29<MrBlizzard>not discord
22:41:31<MrBlizzard>discord like
22:41:32<MrBlizzard>https://fluxer.com/
22:41:33<MrBlizzard>example
22:41:50<nicolas17>oh
22:41:56<nicolas17>I interpreted it as
22:41:58<MrBlizzard>discord alternatives would of been a better way to say it
22:42:14<nicolas17>adding "discord" like a platform to archive
22:42:17<nicolas17>not discord-like :D
22:42:29<MrBlizzard>yea thats slightly on my part
22:42:49<@JAA>cruller: Any size estimate?
22:42:57<MrBlizzard>fluxer.com has over 500K users
22:42:59<@JAA>cruller: And is that captcha IP-based whitelisting or cookies?
22:43:31<MrBlizzard>not entirely sure
22:43:50<klea>I assume IP-based, rather than cookies.
22:43:52<MrBlizzard>fluxer.com has no vital signs but all file attachments uploaded do expire after 3 years
22:44:46<nicolas17>"no vital signs" so it's dead? :D
22:44:49<MrBlizzard>oh
22:44:55<MrBlizzard>no its alive
22:44:57<MrBlizzard>😭
22:45:16<MrBlizzard>I meant no signs of danger of the site going down
22:46:25<MrBlizzard>I'd probably also add other discord alternatives/competitors to that list since while a majority aren't likely to die many can end up getting lost
22:47:35<MrBlizzard>unfortunately most of these will be a bit of a pain to archive and will probably require remaking tools similar to discordchatexporter
22:51:30MrBlizzard quits [Client Quit]
22:52:17MrBlizzard joins
23:11:43<h2ibot>IDKhowToEdit edited Deathwatch (+169): https://wiki.archiveteam.org/?diff=65002&oldid=64993
23:13:25<klea>IDK: (re sonic.com) → #webroasting
23:13:34myuar quits [Ping timeout: 256 seconds]
23:13:51MrBlizzard quits [Client Quit]
23:16:21myuar (myuar) joins
23:24:45<h2ibot>YFdyh000 edited URLTeam (+548): https://wiki.archiveteam.org/?diff=65003&oldid=65001
23:58:48<h2ibot>IDKhowToEdit edited Web Roasting/ISP Hosting (+113): https://wiki.archiveteam.org/?diff=65004&oldid=63578
23:59:25midou quits [Ping timeout: 265 seconds]